Summary of Key Points
The AI company General Intuition has gained attention for its innovative approach of using game footage to train robots. By leveraging the billions of hours of data from the gaming platform Medal, which includes human-operated actions such as mouse movements and button clicks, the company has developed robots with enhanced spatial reasoning abilities. This solves the traditional problems associated with robot training, namely high costs in real-world environments and the disconnect between pure simulations and reality. Recently, General Intuition completed a $320 million Series A financing round, valuing the company at $2.3 billion—five times the amount OpenAI was willing to pay for Medal two years ago. Investors are betting on the “world models” field, which aims to enable AI to understand the physical world. The next step for the company is to open its API to verify whether this technology can move from demonstration to practical application.
1. Game Footage Contains the Essential “Instinctive Data” That Robots Lack
Robot training has always faced two challenges: either collecting data in real-world environments (e.g., having robots repeatedly walk or carry objects), which is extremely costly, or using simulation software (such as Unity). However, robots trained in simulations often struggle to adapt to the real world. For example, a smooth floor in a simulation may cause a robot to stumble when it encounters a dusty real-world surface.
General Intuition’s breakthrough lies in the fact that its game data is neither purely simulated nor entirely from the real world; rather, it serves as a bridge that incorporates human intentions. The videos from Medal are not ordinary recordings; each frame corresponds to specific player actions (such as mouse movements when jumping through windows in Fortnite or button selections for landing in Apex Heroes). These videos contain essential human instincts for navigating three-dimensional spaces—how to avoid obstacles, predict object movement, and make quick decisions in dynamic environments—precisely the skills robots need in the real world.
The founder explains, “Describing the world in words can lead to information loss, but the actions within games represent the most direct sensory data. It’s like teaching a child to recognize an apple; showing them the actual apple is much more intuitive than describing it as a ‘red, round fruit.’”
2. A Short 8-Minute Fine-Tuning Process: The Magic Jump from Virtual to Real
At their New York launch event, General Intuition demonstrated how their model could be pre-trained with game data and then fine-tuned using 8 minutes of real-world data collected by a robot on the street. As a result, the robot was able to navigate a completely unfamiliar office environment successfully.
Why is 8 minutes crucial? Traditional robots would need dozens or even hundreds of hours of data collection in the target environment to adapt, but General Intuition’s model, with its pre-trained foundation, can learn to transfer skills with just a small amount of real-world data. However, there is still an uncertainty: while the demonstration showed the transition from a street to an office, whether this method will work for more complex industrial scenarios (such as dirty floors or changing lighting in factories) remains to be seen. The company plans to release test results later this year, which will be key to verifying the practicality of their technology.
3. The Highly Competitive “World Models” Field: The Next Round of AI Competition Focuses on Physics
General Intuition’s financing is not isolated; similar companies have also received significant investments this year, such as World Labs ($1 billion), Decart ($300 million), and Odyssey ($310 million), with Amazon participating in some of these deals. These companies are all working on “world models” that aim to make AI understand the physical world (e.g., how objects fall or move when pushed). Investors believe that the next battleground for AI will not be language processing (like ChatGPT) but physical interactions (e.g., robots doing household chores or working in factories).
General Intuition’s competitive advantage is clear: the billions of hours of game data accumulated by Medal are unique assets. It would take years for others to collect such data, which is also why OpenAI was willing to pay a substantial amount for Medal. The funds raised will be used to increase computing power to process this massive amount of game data and to open the API, allowing more developers to use their model for robot development. This is a crucial step for the company to transform from a laboratory project into a platform-scale enterprise.
4. A Different Approach: Learning from Human Behavior Rather than Rigid Formulas
While other companies have used gaming engines (like NVIDIA Isaac) to train robots, General Intuition focuses on the actual traces left by human gameplay rather than purely simulated data. For example, instead of teaching a robot about physics formulas (friction, center of gravity), they show the robot videos of humans jumping, running, and avoiding obstacles over 1000 hours. This approach fosters instinctive responses rather than rote memorization of rules. However, there is a gap between the physical laws in games and the real world (e.g., higher jump heights in games), and whether this gap can be bridged by data remains to be tested.
5. The True Test Lies Ahead with API Opening
General Intuition’s valuation of $2.3 billion is based on its approach of using game data for robot training, but the real test will come when they open their API to developers. Developers will then have the opportunity to use this model to create practical applications. For instance, could robots quickly adapt to new factory environments or learn to avoid obstacles in household settings? If successful products emerge, the company’s valuation will be validated; otherwise, it may be questioned.
The gaming industry has spent decades developing human spatial reasoning skills, and now these same data are being used to train robots. This cycle is intriguing, but whether it will truly succeed depends on the practical application of this technology. Overall, General Intuition’s approach is innovative, but its success will depend on how well it can be validated in real-world scenarios.
In conclusion, General Intuition’s strategy is clever, but it still needs to be proven through practical use. Its success could not only impact the robotics industry but also change the value of game data. It turns out that the traces we leave behind while playing games can become valuable tools for training robots.