Zhitong App has learned that Meta Platforms, the parent company of social media platforms Meta Platforms and Instagram, plans to deploy a new version of customized AI chips in its data centers this year to support the development of its artificial intelligence (AI) technology. This chip is part of Meta Platforms' announced &#34;second-generation internal chip production line&#34; last year, which may help reduce heavy reliance on expensive AI chips from NVIDIA, the dominant player in the AI chip market. This is also one of the reasons why Alphabet-C and Microsoft choose to develop their own AI chips. As Meta Platforms plans to launch new AI products, controlling and operating the rising costs associated with AI training/inference workloads is crucial.
As the world's largest social media company, Meta Platforms has been continuously improving its computing power to support computationally intensive and power-consuming generative AI products. This tech giant is pushing its generative AI products, developed in-house, to its global family of Meta Platforms, Instagram, WhatsApp, and other Meta Platforms applications, as well as hardware devices like Ray-Ban smart glasses. Meta Platforms has invested billions of dollars in accumulating a large number of AI chips, such as NVIDIA H100, and reconfigured its data centers to accommodate these chips.
Dylan Patel, the founder of Silicon Research Group SemiAnalysis, stated that with the operational scale of Meta Platforms, successfully deploying its own AI chips could save hundreds of millions of dollars in energy costs and billions of dollars in chip procurement costs each year.
The chips, infrastructure, and energy required to run generative AI applications like ChatGPT have become a &#34;huge sinkhole&#34; for tech companies' investments, partially offsetting the benefits brought by the excitement surrounding this technology.
Tech giants embrace self-developed AI chips
A spokesperson for Meta Platforms confirmed plans to start production in 2024, stating that the chip will work in synergy with the hundreds of thousands of off-the-shelf AI chips, including the Nvidia H100, that the company is currently purchasing. In a statement, the spokesperson said, &#34;We believe that our internally developed AI accelerator will provide the best performance and efficiency combination for Meta Platforms' specific AI workloads, complementing the commercially available AI chips.&#34;
Meta Platforms CEO Mark Zuckerberg previously announced that the company plans to have approximately 350,000 flagship AI chips, including the Nvidia H100, by the end of this year. The H100 is currently the most popular server GPU for AI workloads developed by Nvidia. He emphasized that, combined with the self-developed AI chips and potential AI chips from other suppliers, Meta Platforms will have a computing power equivalent to 600,000 H100 AI chips.
As part of this plan, deploying their own self-developed AI chips marks a positive turning point for Meta Platforms' internal AI chip project, after the company's executives decided to halt the &#34;first iteration&#34; of the chip in 2022.
Instead, the company chose to purchase Nvidia AI chips worth billions of dollars, as Nvidia has almost monopolized the &#34;training&#34; process in AI workloads, which involves feeding large datasets into models to teach them how to perform tasks. Due to the unique architecture of Nvidia AI chips, chips like the H100 are also capable of handling inference tasks, although the computational requirements for inference are much lower than for training, resulting in more competition in the inference domain.
Compared to general-purpose AI chips from Nvidia or AMD, self-developed AI chips, also known as ASICs, are often more suitable for the specific AI workload requirements of tech companies and have lower costs.
For example, cloud computing giants Microsoft and Amazon have chosen to develop their own AI chips primarily to optimize the performance and cost efficiency of specific AI computing tasks, while reducing reliance on external suppliers like Nvidia. Self-developed AI chips can be better integrated into a company's cloud computing platforms and services, providing customized solutions to meet specific business needs.
Global leading public cloud giant Amazon's AWS recently announced the launch of a new self-developed AI chip, AWS Trainium2, designed for generative AI and machine learning training. The performance of Trainium2 is four times higher than the previous generation chip, providing 65ExaFlops of supercomputing power. Microsoft also recently announced the launch of its first custom-designed self-developed CPU series, Azure Cobalt, and AI acceleration chip, Azure Maia. The latter is Microsoft's first AI chip, mainly targeting large language model training, and is expected to be launched in Microsoft Azure data centers early next year. Another major cloud giant, Alphabet-C, recently announced the launch of a new version of its TPU chip, TPU v5p, aiming to significantly reduce the time required for training large language models. V5p is an updated version of the Cloud TPU v5e, which was fully launched earlier this year.
Meta Platforms Platforms Platforms Platforms focuses on self-developed AI chips for inference
Compared to AI training, the demand for GPU parallel computing power in the field of AI inference, which involves using pre-trained models for decision-making or recognition, is much lower. CPUs, which are core processors with excellent performance in complex logic processing and control flow tasks, are sufficient to efficiently handle many inference scenarios.
From the perspective of industry development trends, the AI computing load is likely to gradually shift from training to the inference side. This means that the threshold for AI chips may significantly decrease, and chip companies covering wearable devices, electric vehicles, and the Internet of Things are expected to fully penetrate the field of AI inference chips in the future.
Morgan Stanley, a major Wall Street bank, pointed out in its top 10 investment themes for 2024 that with significant improvements in data processing, storage, and battery life in consumer edge devices, there will be more catalysts driving the catch-up of edge AI, and the development focus of the AI industry will shift from &#34;training&#34; to &#34;inference.&#34;
Edge AI refers to the technology of directly processing AI data streams on edge devices such as PCs, smartphones, IoT devices, and cars. Research firm Gartner predicts that by 2025, 50% of enterprise data will be created at the edge, spanning billions of devices. This means that the inference of AI large models (the process of applying models for decision-making or recognition) is expected to be performed in bulk on edge devices rather than on remote servers or in the cloud.
Qualcomm CEO Amon also pointed out that the main battlefield for chip manufacturers will soon shift from &#34;training&#34; to &#34;inference.&#34; Amon said in a recent interview, &#34;As AI large models become more streamlined and capable of running on devices, focusing on inference tasks, the main market for chip manufacturers will shift to 'inference,' that is, model application. Data centers are also expected to show interest in processors specifically designed for inference tasks of trained models, and everything will contribute to the inference market surpassing the training market.
According to reports, Meta Platforms' new self-developed AI chip is internally known as &#34;Artemis&#34;. Similar to its predecessor, it can only perform an artificial intelligence workload process called &#34;inference&#34;. In this process, models are required to use their algorithms to make ranking judgments and respond to user prompts.
Media reports last year indicated that Meta Platforms was developing a more ambitious chip, similar to NVIDIA's H100, that could perform both training and inference simultaneously.
The tech giant, headquartered in Menlo Park, California, previously shared details of the first-generation Meta Platforms Training and Inference Accelerator (MTIA) project. However, the announcement only described this version of the chip as a learning opportunity. The plan has not been mentioned by Meta Platforms since then.
Patel stated that despite some early challenges, AI chips in the inference field, such as those used in Meta Platforms' recommendation models, may be much more efficient than power-consuming chips like NVIDIA's H100. &#34;Therefore, a significant amount of funds and power are expected to be saved overall.&#34;

GOOGL

GOOG

Meta Platforms plans to deploy a new version of customized AI chips in its data centers this year to reduce its reliance on expensive AI chips from NVIDIA. This will help them save energy and chip procurement costs, supporting the development of their artificial intelligence technology. This move is also one of the reasons why Alphabet-C and Microsoft choose to develop their own AI chips. Meta Platforms has been improving its computing power to support its generative AI products, including Meta Platforms Platforms Platforms and Ray-Ban smart glasses. Meta Platforms has invested billions of dollars in accumulating a large number of AI chips and reconfiguring its data centers. This will to some extent offset the huge costs brought by AI technology.

Self-developed AI chips are sweeping through major companies! Meta Platforms plans to deploy self-developed chips this year, focusing on AI inference.