D-Matrix Inc., a hardware startup based in Santa Clara, California, has unveiled its first AI processor, Corsair, which aims to redefine AI inference. This innovative product does away with traditional GPUs and costly high-bandwidth memory (HBM), offering a more efficient and cost-effective solution.
Supported by Microsoft and embracing cutting-edge advancements, Corsair is available to early-access customers and is expected to reach broader availability by the second quarter of 2025.
What makes Corsair stand out?
Corsair is purpose-built to tackle demanding AI inference tasks, particularly those involving generative AI models. The processor achieves remarkable speeds, handling 60,000 tokens per second at 1 millisecond per token when running models like Llama3 8B on a single server.
For larger-scale applications, such as Llama3 70B, Corsair achieves 30,000 tokens per second at 2 milliseconds per token on a single rack. This performance significantly lowers energy consumption and operational costs compared to traditional GPU-based systems.
Built on d-Matrixโs Nighthawk and Jayhawk II tiles, Corsair utilises a 6nm manufacturing process. Each Nighthawk tile has four neural cores and a RISC-V CPU, optimised for large-model inference. It incorporates digital in-memory computation (DIMC) and supports versatile datatype processing, including block floating point (BFP).
Corsairโs chiplet packaging integrates memory and computation to boost efficiency. It adheres to the PCIe Gen5 full-height, full-length card form factor, making it compatible with DMX Bridge cards for scalable performance. The card is powered by 2400 TFLOPs of 8-bit peak computing power, 2GB of integrated performance memory, and 256GB of off-chip memory capacity.
A partnership with Nvidiaโs key ally
Micron Technology, a significant partner of Nvidia, is collaborating with d-Matrix to support Corsairโs development and growth.
Although initially slated for release in late 2023, d-Matrix restructured its architecture to align with the growing demand for generative AI. This shift allowed Corsair to incorporate advanced features tailored for transformer models, agentic AI, and emerging applications like interactive video generation.
โOur vision for d-Matrix was to address the massive computing challenges of generative AI and transformers,โ said Sid Sheth, co-founder and CEO of d-Matrix. โCorsair is a groundbreaking platform, delivering blazing-fast token generation for interactive AI applications, making generative AI commercially viable.โ
Corsairโs focus on cost efficiency, scalability, and high-performance positions it as a promising solution in the rapidly evolving AI landscape.
