Wednesday, 1 January 2025
24.6 C
Singapore

Microsoft supports new startups as it launches an AI processor that skips GPUs and expensive memory

Discover how d-Matrix's new AI processor, Corsair, backed by Microsoft, redefines AI inference with a GPU-free and cost-effective solution.

D-Matrix Inc., a hardware startup based in Santa Clara, California, has unveiled its first AI processor, , which aims to redefine AI inference. This innovative product does away with traditional GPUs and costly high-bandwidth memory (HBM), offering a more efficient and cost-effective solution.

Supported by Microsoft and embracing cutting-edge advancements, Corsair is available to early-access customers and is expected to reach broader availability by the second quarter of 2025.

What makes Corsair stand out?

Corsair is purpose-built to tackle demanding AI inference tasks, particularly those involving models. The processor achieves remarkable speeds, handling 60,000 tokens per second at 1 millisecond per token when running models like Llama3 8B on a single server.

For larger-scale applications, such as Llama3 70B, Corsair achieves 30,000 tokens per second at 2 milliseconds per token on a single rack. This performance significantly lowers energy consumption and operational costs compared to traditional GPU-based systems.

Built on d-Matrix’s Nighthawk and Jayhawk II tiles, Corsair utilises a 6nm manufacturing process. Each Nighthawk tile has four neural cores and a RISC-V CPU, optimised for large-model inference. It incorporates digital in-memory computation (DIMC) and supports versatile datatype processing, including block floating point (BFP).

Corsair’s chiplet packaging integrates memory and computation to boost efficiency. It adheres to the PCIe Gen5 full-height, full-length card form factor, making it compatible with DMX Bridge cards for scalable performance. The card is powered by 2400 TFLOPs of 8-bit peak computing power, 2GB of integrated performance memory, and 256GB of off-chip memory capacity.

A partnership with Nvidia’s key ally

Micron Technology, a significant partner of Nvidia, is collaborating with d-Matrix to support Corsair’s development and growth.

Although initially slated for release in late 2023, d-Matrix restructured its architecture to align with the growing demand for generative AI. This shift allowed Corsair to incorporate advanced features tailored for transformer models, agentic AI, and emerging applications like interactive video generation.

โ€œOur vision for d-Matrix was to address the massive computing challenges of generative AI and transformers,โ€ said Sid Sheth, co-founder and CEO of d-Matrix. โ€œCorsair is a groundbreaking platform, delivering blazing-fast token generation for interactive AI applications, making generative AI commercially viable.โ€

Corsair’s focus on cost efficiency, scalability, and high-performance positions it as a promising solution in the rapidly evolving AI landscape.

Hot this week

LG reveals 2025 gaming monitors with innovative bendable 5K2K OLED

LG unveils its 2025 gaming monitors, led by a 5K2K bendable OLED. These monitors boast 21:9 aspect ratios and advanced features for immersive gaming.

YouTube tests a new feature to help you decide what to watch

YouTube tests a "Play something" button to help users decide what to watch next, offering personalised video recommendations through the Shorts interface.

Nintendo Switch 2: Rumoured upgrades that could make it a must-buy

Discover the rumoured Nintendo Switch 2 upgrades, including better performance, revamped Joy-Cons, and a familiar design, possibly launching soon!

Hackers exploit Russian domains for phishing attacks

Hackers are bypassing email security by exploiting Russian domains and advanced phishing tactics, including RATs and malicious Office documents.

Marriott and Starwood hotels urged to strengthen data security measures

The FTC ordered Marriott and Starwood to improve data security after breaches exposed the information of 344M customers with new policies and transparency.

China tightens rules on cryptocurrency and forex trading

China tightens forex rules, targeting cryptocurrency trades to prevent risky transactions and strengthen financial oversight.

Alipay introduces AI image search to strengthen its role as a super app

Alipay launches an AI image search tool, enhancing its super app role and intensifying competition with Tencent's WeChat in China.

Volkswagen data leak reveals sensitive driver locations and details

A data leak exposed the locations and personal details of 800,000 electric Volkswagen vehicles, raising serious privacy concerns.

US Treasury Department faces major security breach

The US Treasury Department faced a cyberattack linked to a China-state-sponsored hacker exploiting third-party software. No ongoing access was found.

Related Articles