Meet Maia and Cobalt, Microsoft’s New Custom Chips Designed for AI Supremacy

Microsoft, known more for its software than hardware, today announced two new chips: the Azure Maia AI Accelerator and the Azure Cobalt CPU. The move is a notable expansion of the company’s technological capabilities, with Maia in particular positioned as the tip of the spear for the Redmond-based tech giant’s AI strategy.

Maia is specifically designed for AI tasks and generative AI in the data center environment.

“We are reimagining every aspect of our data centers to meet the needs of our customers,” wrote Scott Guthrie, Executive Vice President of Microsoft’s Cloud + AI Group, in an official blog post, explaining that adding custom-designed chips into their ecosystem aligns with Microsoft’s broader vision for integrated and optimized AI and cloud services.

The Maia chip, developed as part of Microsoft’s recent expansion into custom silicon, has several important characteristics that distinguish it in the fields of AI and cloud computing. These include optimizations for AI tasks like model training and inferencing and generative AI tasks—creating new content from existing datasets.

Guthrie says Maia will be especially adept working with large language models, like the powerful GPT-4 Turbo developed by OpenAI. This makes it well-suited for tasks involving natural language processing and understanding.

Maia also integrates with Microsoft Azure, the company’s cloud computing service, providing customers with increased performance whether or not they are working with AI.

As a custom-designed chip, the company noted, Maia is tailored to meet specific performance requirements for Microsoft’s services rather than adjusting for the limitations of off-the-shelf silicon.

Finally, with data center power consumption raising the ire of environmental groups worldwide, Microsoft says Maia is especially energy efficient, which at scale allows for a significant reduction in energy use and costs.

Rani Borkar, Corporate Vice President for Azure Hardware Systems and Infrastructure, shed light on the company’s collaboration between hardware and software. “We are co-designing and optimizing hardware and software together,” she noted, highlighting the focus on the Maia chip to improve efficiency in cloud and AI workloads. “We have visibility into the entire stack, and silicon is just one of the ingredients.”

Recently, Microsoft revamped its Windows 11 operating system, adding AI features to enhance its capabilities. Now, an AI-powered Windows copilot replaced Cortana, and AI has been imbued into Microsoft 365 and other native programs offered by Microsoft—even Microsoft Paint.

Is Microsoft mulling a change of architecture?

Along with Maia, Microsoft introduced its new lineup of Cobalt chips, suggesting a potential shift in Microsoft’s architectural preferences. Moving away from the traditional x86/AMD64 setup, Microsoft appears to be exploring ARM architecture—likely influenced by the efficiency benchmarks set by ARM-based chips like Apple’s M series. If so, it could mean a possible new direction for Windows, particularly in the realms of AI and cloud computing efficiency.

The Cobalt CPU is built on ARM architecture, known for its energy efficiency and performance. ARM-based designs are increasingly popular in cloud computing due to their balance of power and performance.

The chip is specifically optimized for running general-purpose compute workloads in the cloud. This makes it particularly suitable for a wide range of applications in Microsoft’s cloud services.

Also, a core focus of the Cobalt CPU is energy efficiency. Just like Apple’s M-series chips, the design of the Cobalt chips aims to maximize computing power while minimizing energy consumption, which is crucial in large-scale data center operations.

The Cobalt 100 CPU evolved from ARM’s Neoverse N1 design, featuring Neoverse N2 cores. This upgrade promises a significant performance boost, with 40% higher performance compared to its predecessor, according to Semanalysis.

The announcement of both Maia and Cobalt reflects a broader industry trend where tech giants are increasingly taking chip production in-house to tailor performance to specific needs. For example, Google recently introduced its new Tensor G3 lineup and expanded its partnership with Anthropic to let it use its latest generation of TPUs to train new AI models. Besides that, Apple recently introduced its M3 lineup with more AI capabilities, Qualcomm launched its Snapdragon 8 Gen 3 with enough power to run a small local LLM, and even smaller competitors like AMD and Mediatek are expanding their AI capabilities.

Micorosft’s bet on AI appears to be paying off. The company’s stock performance in 2023 has been positively impacted by its AI initiatives, particularly its partnership with OpenAI. In 2023, Microsoft’s stock performance was notably strong, reflecting the company’s robust growth and market resilience. As of November 10, 2023, Microsoft’s stock price closed at an all-time high of $369.67, with the 52-week high stock price reaching $370.10, just 0.1% above the current share price​.

Edited by Ryan Ozawa.