Summary of Key Points
This news article highlights the critical trends in the domestic computing power sector: as the number of parameters in large models continues to grow (for example, Kimi’s 2.8 trillion parameters), the computing clusters used for their training and inference need to shift from simply stacking cards to achieving system-level optimization. Super nodes have become a core solution to address the coordination issues between these cards. At this year’s WAIC conference, several domestic manufacturers such as Huawei and Alibaba introduced super node products. While domestic super nodes are leading in certain technical aspects (such as orthogonal backplanes), they face challenges related to high costs and high power consumption. In the future, optical interconnectivity will be key to overcoming the limitations of current super node scalability.
Detailed Analysis
1. Why have super nodes suddenly become a necessity?
In the past, large models could be trained using just dozens of cards; however, with parameters increasing to 2.8 trillion (and expected to reach 5-6 trillion by next year), the number of required computing cards has also doubled. The increased number of cards creates a bottleneck in communication efficiency—slower communication between them results in significant waste of computational resources. Super nodes are designed to integrate multiple cards (for instance, up to 256) and make them function as if they were part of a single, highly efficient system. For example, Moore Threads’ super node technology allows 256 GPUs to work together as if they were a single unit, improving efficiency and reducing costs. This becomes even more crucial when the cluster size exceeds tens of thousands of cards; it’s impractical to rely on stacking hundreds of servers directly.
2. Domestic super node technology: Some approaches are already ahead of overseas competitors
Overseas manufacturers (such as NVIDIA) use cable connections for their super nodes, which can connect up to 128 cards. However, these cables are prone to failure and require replacement, and they limit the distance over which signals can be transmitted. Domestic manufacturers have adopted a different approach with orthogonal backplanes, which eliminate the need for cables and enable direct, 90-degree connections between computing and switching units, simplifying maintenance and production processes. Currently, no overseas company offers this solution, so we are in the lead in this area (manufacturers like Wallrun, Muxi, and Moore Threads have related products). Additionally, while NVIDIA handles both upstream and downstream components for its super nodes, domestic companies often collaborate with multiple suppliers (e.g., using GPUs from Moore Threads and CPUs from other manufacturers), which provides unique technical advantages.
3. The challenges of developing super nodes: Three major engineering hurdles
Developing super nodes is no easy task:
- High costs: A single super node can cost hundreds of millions of yuan, and the failure of a single card can render the entire device unusable.
- High power consumption: While ordinary data centers consume around 6 kW, super nodes can require up to 48 kW or even 400 kW, posing significant cooling challenges.
- Reliability: It’s essential to ensure that the failure of a single component (such as a network card or disk) does not affect the entire system’s performance, especially in clusters with tens of thousands of cards. Even NVIDIA has encountered production issues with its next-generation super nodes due to PCB-related problems, highlighting the complexity of this task.
4. The future direction: Optical interconnectivity as the key to scalability
Current electrical interconnect technologies can support a maximum of 128 cards; as model parameters continue to increase, these limitations will become more prominent. Optical interconnectivity offers a solution by enabling longer transmission distances and faster data transfer speeds. Among the available optical solutions, NPO (Near-Package Optics) is the most promising approach, as it allows GPUs and optical engines to be packaged together without the need for custom data centers. Multiple manufacturers are working on this technology, with expected widespread adoption next year and full implementation by 2028. For example, Wallrun’s new generation of super nodes will utilize NPO, enabling support for even larger-scale clusters and models with even higher parameter counts.
Conclusion
The emergence of super nodes marks a shift in the domestic computing power industry from a focus on individual card performance to system-level coordination. While domestic manufacturers have made progress in technology, there is still a need for time to overcome engineering challenges and implement optical interconnectivity solutions. Behind this development is the drive to compensate for the limitations of single-chip performance through systematic optimizations.