Summary of Key Points
Although world models have confusing names (some are called physical AI, while others are embedded in autonomous driving architectures), their essence is to create a virtual environment within the machine's “brain” that allows for reasoning and experimentation before actual action, thereby reducing reliance on real data. Currently, major internet companies (such as Alibaba, Tencent, ByteDance), automakers (like NIO, Li Auto, Xpeng), and autonomous driving suppliers (including Momenta and Horizon Robotics) are all making investments in this area, each with their own focus: large companies tend to focus on “creating worlds” that encompass language, virtual, and physical scenarios; automakers use these worlds as training grounds or testing platforms for autonomous driving systems; suppliers act as the “invisible engines” that are integrated into products. Startups have specialized advantages but lack resources, while large companies have data, computing power, and a complete ecosystem, although their organizational structures can be complex. This is not a new trend; rather, it is the inevitable outcome of the convergence of technologies such as language models, video generation, and autonomous driving in the physical world. The competition in the future will revolve around which company’s world model can truly help machines understand the real world.
Detailed Analysis
1. What Exactly Are World Models?
In simple terms, a world model is like a virtual sandbox for machines—allowing them to simulate situations in their “brains” before taking actual action. For example:
- Autonomous driving: Simulating rainy or snowy weather, as well as unexpected pedestrians, to create challenging scenarios for training the system.
- Robots: Practicing tasks (such as grasping objects and walking) in a simulated environment until they are proficient enough before going out into the real world.
- Games: Based on user input, generating interactive virtual worlds. For instance, if you request a “beach town,” the model creates a 3D world that users can explore and modify.
The core purpose of world models is to use virtual data to replace some of the real data, addressing challenges related to the difficulty and high cost of collecting real-world data (such as in extreme weather or accident scenarios), thus enabling machines to learn more efficiently and safely.
2. Internet Companies: Moving from the “Digital World” to the “Physical World”
Each major company has its own approach, but they are all striving for control over the definition of these virtual worlds:
- Alibaba: Its three models cover various scenarios: Qwen-AgentWorld serves as a language training platform; HappyOyster creates playable virtual environments based on user input; Qwen-RobotWorld helps robots rehearse their actions.
- Tencent: Focusing on 3D games, its HY-World series can generate 3D models from text or images for use in game levels and virtual filming.
- ByteDance: Leveraging its massive video data stream (1 billion videos per day) to create 4D scenarios, aiming to develop digital twins that simulate real-world phenomena.
- Huawei: Working quietly in the industry, integrating world models into its Panghu model to accelerate the training of autonomous vehicles (e.g., quickly recreating complex accident scenes).
- Baidu: Incorporating world model capabilities into its Apollo autonomous driving system, which can predict traffic behavior.
3. Automakers: Using World Models as Training Platforms for Autonomous Driving
Automakers’ goal is clear: to make autonomous driving safer and more intelligent.
- NIO: Its NWM world model enables the car to “imagine road conditions” without visual input, generating future video scenarios every 0.1 second and selecting the best path among 216 possible options. The new version of its AEB (Automatic Emergency Braking) system covers six times more scenarios than before, with reduced braking errors to once in a hundred thousand kilometers.
- Li Auto: Using its DrivingSphere model to create highly realistic 4D simulations to test autonomous driving systems, aiming to match the capabilities of Tesla’s FSD V14.
- Xpeng: Starting with a large-scale simulation approach (equivalent to 30 million kilometers per day) and planning to apply these models to robots and flying cars in the future.
- Geely: Integrating world models into its vehicles, connecting autonomous driving systems, cockpits, and chassis. The Geely Extreme Knight 8X already features this technology, allowing for adjustments like adjusting the seat heating based on user requests.
4. Suppliers and Startups: Navigating the Competitive Landscape
- Suppliers: Act as the “invisible engines” behind these systems. For example, Momenta’s R7 world model has been deployed in 900,000 vehicles and trained using data from over 12 billion kilometers of real driving. Horizon Robotics’ HorizonDrive generates driving videos that help automakers verify L3+ autonomous driving systems.
- Startups: Their advantage lies in their specialization (e.g., focusing on 3D space generation), but they face challenges with data, computing power, and mass production capabilities. Large companies have the necessary resources (e-commerce platforms, cloud services, etc.) to continuously feed their models.
5. This Is Not a New Trend, but an Evolution of Existing Technologies
World models are not something new; they represent the intersection of language models, video generation, autonomous driving, and robotics. Previously, these technologies operated independently, but now they are combined to enable machines to progress from simply “recognizing” the world (e.g., identifying a cat) to “understanding” it (e.g., knowing that cats can jump and catch mice). The future competition will not be about who can create the best model, but about whose models can be effectively implemented in real-world applications—whether in autonomous driving under extreme weather conditions or in helping robots with household tasks. Large companies have already incorporated these models into their product development processes, while startups need to find their place in this evolving landscape.
In Conclusion
World models represent a critical step in the evolution of machines from being able to “see” the world to being able to “think” about it. Those who can enable machines to truly understand the physical world will hold the key to the next era. In this race, large companies have the resources, while startups possess the innovation and drive; ultimately, it will be the ability to successfully implement these models that determines success.