Summary of Key Points
NVIDIA's introduction of the Vera CPU, which features its own custom-designed core, is not aimed at competing with the traditional CPU market. Instead, it addresses the issue where CPUs can become a bottleneck for GPUs in intelligent agent AI applications. Intelligent agents require CPUs to frequently handle tasks such as script execution, database queries, and task scheduling, which are often performed in a single-threaded manner. The multi-core design of traditional CPUs fails to overcome these limitations, leading to inefficient use of expensive GPUs that sit idle. By optimizing single-core performance and reducing latency in interconnections, Vera aims to improve GPU utilization. Essentially, NVIDIA seeks to take control of the entire AI computing system, from the CPU to the GPU, and from software to the entire hardware infrastructure, redefining the criteria for evaluating CPUs (from "number of cores per dollar" to "amount of AI output per watt of power consumed").
Detailed Analysis
1. Why Did NVIDIA Suddenly Focus on CPUs? – Intelligent Agents Make CPUs a Bottleneck
In the past, AI processes were relatively straightforward: data was fed to the GPU, which processed it and provided the results, with the CPU only performing preparatory tasks. However, intelligent agents act more like versatile assistants that need to coordinate multiple tasks, involving GPU inference, search queries, Python script execution, database interactions, and managing the collaboration of multiple agents. These processes are often serial in nature (A → B → C), so increasing the number of CPU cores does little to improve performance—it's like having ten coffee ordering windows but still waiting for slow customers. Since GPUs are expensive and power-consuming, any delay is costly. NVIDIA developed Vera to make CPUs fast enough to prevent GPUs from idling.
2. How Does Vera Differ from Traditional CPUs? – A Shift in Design Philosophy
Traditional server CPUs follow the principle of "more cores equals better performance." Cloud companies need to run multiple virtual machines on a single CPU, which is why AMD's EPYC series has hundreds of cores. However, Vera takes a different approach:
- Fewer but More Powerful Cores: With only 88 cores, each core utilizes advanced technologies like 10-way decoding and deep out-of-order execution, making it faster in single-threaded tasks.
- Reduced Latency: All cores are placed on the same chip, reducing data transfer times between them and with memory.
- High Memory Bandwidth: Supporting 1.2 TB/s of memory bandwidth allows for faster data transfer to the GPU.
NVIDIA's new metric for evaluating CPUs is "how much AI output can be generated per watt of power." After all, GPUs are the main performers, and the CPU's role is to enable them to work more efficiently.
3. NVIDIA's Real Ambition: To Be the Owner of the “AI Factory”
Vera is not sold as a standalone product; it is part of NVIDIA's AI ecosystem (e.g., the Vera Rubin NVL72 platform), which includes 36 Vera CPUs, 72 GPUs, an NVLink interconnect system, and network cards. This indicates that NVIDIA aims to sell a complete AI computing solution rather than just GPUs. Previously, it had to adapt its products to Intel/AMD CPU platforms; now, it wants to define the entire system from scratch. Traditional server manufacturers will become suppliers of components, while NVIDIA becomes the owner of the “AI factory.”
4. A Changing Market Landscape: The CPU Race Moving Towards Segmentation
With Vera, the server CPU market is no longer dominated by Intel and AMD’s x86 architecture. The future will see four main trends:
- General-Purpose x86: Intel/AMD will continue to produce CPUs compatible with all software for general business use.
- Cloud-Specific Solutions: Cloud providers like AWS and Google are developing their own custom CPUs (e.g., Graviton and Axion) to optimize cloud services.
- AI-Dedicated Solutions: CPUs designed specifically for AI, such as Vera, which are highly integrated with GPUs and focused on efficiency (e.g., output per rack of hardware).
- Industry-Specific Solutions: Customized CPUs for high-performance computing and database applications (e.g., Arm-based solutions).
The shift from general-purpose CPUs to specialized ones reflects the trend towards more tailored hardware for specific use cases, similar to how mobile and desktop chips have evolved.
5. Implications for China's Processor Industry
Chinese companies developing CPUs should focus on system integration rather than just focusing on the number of cores or chip manufacturing technology. The success of an AI system depends on the coordination between CPU, GPU, memory, interconnects, and software. For example, even if domestic AI chips have high theoretical performance, poor data preparation and task scheduling can still lead to inefficient use of GPUs. Future developments should focus on:
- High-speed connections between CPUs and accelerators.
- Energy-efficient memory systems.
- Optimized environments for intelligent agents.
- Coordinated design from chip to entire hardware infrastructure.
In summary, Vera marks a shift in the role of CPUs from general-purpose components to essential elements in AI systems. The real competition will be about which companies can create the most efficient AI solutions.
Conclusion
Vera represents more than just a revival of CPUs; it signifies a change in their role within the AI ecosystem. GPUs have become the central components, and CPUs are now playing a supporting role as “advanced managers” that optimize GPU performance. NVIDIA’s Vera is part of its strategy to strengthen its dominance in AI computing and gain control over the entire AI infrastructure. The future competition will revolve around which companies can build the most efficient AI systems.