Microsoft Targets Nvidia's Inference Business With Maia 300 Chip

Microsoft drew a cleaner line against Nvidia than it ever has before when it unveiled the Maia 300 at the OCP Global Summit on October 12. The company didn't just introduce another in-house accelerator to sit alongside its GPU fleet. It said Maia 300 will become the primary inference system for Microsoft's own products and services, with production deployments starting in 2027.

The scope matters here, so it's worth being precise about it. <cite index="19-12">Microsoft's plans to make Maia its primary inference platform apply specifically to Microsoft's own services, not necessarily to third-party Azure customers, including OpenAI.</cite> That's a narrower commitment than the biggest headlines suggest, but it's still a significant architecture decision: the company is betting its own Copilot stack, Teams AI features, and cloud AI workloads on silicon it controls rather than silicon it buys from a single dominant vendor.

<cite index="28-4,28-5">That is a notable shift in tone. Microsoft has built custom silicon since 2023, but it has mostly talked about Maia chips as a supplement to Nvidia and AMD hardware inside Azure data centers.</cite> The performance claims attached to Maia 300 are specific enough to test: <cite index="22-2">the company says Maia 300 provides 40% more tokens per megawatt than "leading GPU-based systems," and that a 96-chip rack offers seven times the rack-scale token throughput of Microsoft's Maia 200.</cite>

The underlying architecture choices, reported in a Microsoft post reviewed by CNBC, show where the design priorities actually landed. <cite index="27-6">The chip delivers 36% more compute FLOPs, 2.5 times HBM bandwidth and 14% more scale-up bandwidth than Maia 200, with an accelerator able to send up to 750 GB/s to a peer under certain cross-rack job configurations.</cite> Those bandwidth numbers are relevant because memory-to-compute data movement is the real bottleneck in inference, not raw FLOP count.

<cite index="27-5">Maia 300 can scale to a 12,288-accelerator superpod across 128 racks</cite>, which means Microsoft is building toward datacenter-scale homogeneous inference clusters rather than mixing vendor hardware in each rack. That's the kind of supply chain simplification that becomes valuable when you're running hundreds of millions of daily inference calls across enterprise products.

<cite index="27-8,27-9">Reuters reports that Maia 300 splits functions across process nodes, with compute functions using 2 nm technology while data-movement components use 3 nm silicon, though Microsoft's own specifications identify TSMC N2P for the compute chiplets but do not provide that breakdown.</cite> The process node choices weren't confirmed in full by Microsoft directly, so treat the Reuters reporting on the split-node approach as unconfirmed until Microsoft publishes its complete architecture documentation.

What Microsoft was careful not to say is that it's walking away from Nvidia. <cite index="28-7,28-8">None of this means Microsoft is abandoning Nvidia or AMD. The company said its own silicon will mainly handle inference requests, which leaves room for GPUs to continue handling training and other workloads.</cite> Training at the frontier still runs on Nvidia, and nothing in the Maia 300 announcement changes that in the near term.

<cite index="19-1">Microsoft's plan to power its AI products with custom silicon shows how every major Nvidia customer is now a competitor, seeking lower costs, improved performance and more control</cite>, as Axios noted in its coverage of the announcement. Google has Tensor Processing Units. Amazon has Trainium and Inferentia. Now Microsoft is moving Maia from supplemental to primary for at least the inference half of its workload.

The missing piece is independent benchmark data. Microsoft's efficiency claims compare Maia 300 to unnamed "leading GPU-based systems" without specifying which Nvidia part is the baseline. Until third-party benchmarks run Maia 300 against H100 and Blackwell hardware on representative inference workloads, the 40% efficiency claim is a target, not a verified result. Production deployments start in 2027, so the first real-world data is still months away.

Sources cited:
- CNBC (https://www.cnbc.com/2026/10/12/microsoft-will-mainly-use-its-maia-chips-for-ai-inference.html)
- Axios (https://www.axios.com/2026/10/13/big-tech-ai-chips-nvidia-rivals)
- WindowsForum (Reuters and Microsoft Azure sourced) (https://windowsforum.com/news/microsoft-maia-300-ai-chip-targets-copilot-inference-in-2027-while-nvidia-remains-key-megathread.448034/)
- Shattered.io (https://shattered.io/microsoft-maia-300-ai-chip-2026-2/)