--- title: "From \"Training\" to \"Inference\" -- The Changing Role of the CPU" type: "Topics" locale: "en" url: "https://longbridge.com/en/topics/40196884.md" description: "The nature of AI workloads has undergone a significant shift. If the past was the era of "training," with GPUs as the absolute protagonists; then we have now entered a new phase centered on "inference" and "intelligent agents" (Agentic AI), where the role of the CPU has become unprecedentedly important. This change is primarily driven by several factors: 📈 Structural shift in computing power demand: AI development is transitioning from a focus on "model development and training" (Train) to one dominated by "practical application and inference" (Inference)..." datetime: "2026-04-25T01:10:49.000Z" locales: - [en](https://longbridge.com/en/topics/40196884.md) - [zh-CN](https://longbridge.com/zh-CN/topics/40196884.md) - [zh-HK](https://longbridge.com/zh-HK/topics/40196884.md) author: "[夏日远航](https://longbridge.com/en/profiles/18520637.md)" --- # From "Training" to "Inference" -- The Changing Role of the CPU The nature of AI workloads has undergone a significant shift. If the past was the era of "training," with GPUs as the absolute protagonists, we have now entered a new phase centered on "inference" and "Agentic AI," where the role of the CPU has become more critical than ever. This change is primarily driven by several factors: 📈 Structural Shift in Computing Demand: From "Training" to "Inference" AI development is transitioning from a focus on "model development and training" to one dominated by "practical application and inference." The logic of inference workloads places immense demand on CPUs, which are naturally suited for task scheduling and data processing. Different Task Nature: Training is an intensive batch processing task where GPUs maximize their parallel computing advantage. Inference emphasizes low-latency, high-efficiency real-time computing, requiring CPUs to play a unique role in task orchestration, data management, and control scheduling. Widening Ratio Gap: In training scenarios, a common configuration is 1 CPU paired with 7-8 GPUs. In inference scenarios, this ratio tightens to approximately 1:4, directly indicating increased demand for CPUs. 🤖 The Rise of "Agentic AI": From "Answering" to "Executing" This is the core factor driving the change in the ratio. Unlike traditional "Q&A AI," Agentic AI is a complex system capable of autonomous planning, tool invocation, and task execution. CPU as the Orchestration Core: Agentic AI needs to interact dynamically with the environment—planning tasks, invoking tools, passing data between sub-agents, and evaluating task completion. All this complex "orchestration layer" work is handled by the CPU. Quantitative Data Support: Relevant research clearly indicates that in Agentic AI scenarios, latency generated by CPU processing (e.g., interpreting Python code, database retrieval) can account for 90.6% of total latency, and the CPU energy consumption for these tasks can reach up to 44% of total energy consumption. Exponential Demand Growth: According to estimates, traditional AI data centers require about 30 million CPU cores per gigawatt (GW). In the Agentic AI era, this demand is projected to surge to 120 million cores, a fourfold increase. ⚙️ The Industrialization of "Reinforcement Learning": Simulation and Decision-Making Intensify CPU Demand Reinforcement learning technology is moving out of the lab and achieving industrial application in cutting-edge fields like autonomous driving, robotics, and precision medicine. CPU Dominates Simulation Computing: The core processes of reinforcement learning—environment stepping, control logic, search, trajectory management—are all CPU-led. Especially in high-fidelity 3D simulation environments, massive CPU computing power is needed to simulate the physical world and complex scenarios. 💰 Economic and Industrial Considerations: From "Stacking Compute" to "Pursuing Efficiency" As AI enters the stage of large-scale application, data center operators are beginning to measure computing efficiency and cost more meticulously. Avoiding Expensive Resource Idleness: GPUs are one of the most expensive resources in a data center. If CPU scheduling capability is insufficient, expensive GPUs will sit idle while waiting for tasks, causing resource waste. Increasing CPU resources to ensure GPUs are always fully loaded is key to optimizing cost-effectiveness. Shifting System Bottleneck: As AI systems become more complex, the performance bottleneck has shifted from GPU computing power to CPU scheduling capability. Simply stacking more GPUs no longer linearly improves overall performance. Facing this trend, the entire industry is rapidly adjusting. Traditional vendors like Intel and AMD are already seeing tight CPU supply and price increases, while companies like NVIDIA and Arm are also making moves and launching their own server CPU products to meet the growing CPU demand. In summary, the change in the CPU-to-GPU ratio in data centers is an inevitable result of AI moving from "showcasing skills" to "being pragmatic." When AI begins to operate as a service on a large scale, what determines its efficiency and cost is not just the peak computing power of a single chip, but the collaborative capability of the entire system. Disclaimer: This content is generated by AI. The views expressed herein represent only the output of the AI model and do not constitute any real investment advice or basis for operation. ### Related Stocks - [9984.JP](https://longbridge.com/en/quote/9984.JP.md) - [ARM.US](https://longbridge.com/en/quote/ARM.US.md) - [AMD.US](https://longbridge.com/en/quote/AMD.US.md) - [NVDA.US](https://longbridge.com/en/quote/NVDA.US.md) - [SFTBY.US](https://longbridge.com/en/quote/SFTBY.US.md) - [AMDL.US](https://longbridge.com/en/quote/AMDL.US.md) - [NVDL.US](https://longbridge.com/en/quote/NVDL.US.md) - [07788.HK](https://longbridge.com/en/quote/07788.HK.md) - [07388.HK](https://longbridge.com/en/quote/07388.HK.md) - [NVDY.US](https://longbridge.com/en/quote/NVDY.US.md) - [NVDD.US](https://longbridge.com/en/quote/NVDD.US.md) - [NVDX.US](https://longbridge.com/en/quote/NVDX.US.md) - [NVDQ.US](https://longbridge.com/en/quote/NVDQ.US.md)