虎嗅

The Fainthearted Game of AGI

原文:AGI的胆小者游戏

Title: AI Giants Calling for a Stop? Don’t Be Scared by “Human Safety” – It’s Actually a Respectable Way to Quit a Money-Draining Game

Hello everyone, I’m your financial journalist and economist. Recently, something quite interesting happened in the AI community: two of the leading large-scale AI models, OpenAI and Anthropic, suddenly started speaking in unison, claiming that AI could potentially destroy humanity and urging governments to strengthen regulation or even slow down the pace of AI development.

At first glance, it seems these giants are truly concerned about the welfare of all mankind, sacrificing their business interests for the sake of global safety. But we need to look beyond the surface and understand the real motivations behind their actions. As ordinary people, we don’t have to get bogged down in complex algorithms; instead, we should focus on the money and the bigger picture.

Today, I’ll break down this news into five key points in plain language, revealing the hidden motives behind this AI safety concern and what it means for us as individuals and for the technological competition between China and the United States.

---

1. Why the Sudden Change in Attitude? Because the Money Run Out and Profitability Has Declined

First, let’s figure out why OpenAI and Anthropic have suddenly decided to slow down. Is it really because they’re afraid that AI will rebel against humanity?

The core reason is that their expenses are outpacing their income, and there’s no end in sight:

  • Cash Flow Crisis: AI models are extremely costly. Companies like Google and Amazon have invested heavily in buying chips and building data centers to support AI. The capital market is worried that their profits aren’t enough to cover these expenses, and their free cash flow (the money they can freely use) might be depleted.
  • Delayed IPOs: Both companies were planning to go public this year, but those plans have been postponed. The reason is that they need to disclose financial data (such as annual recurring revenue, ARR) before listing. If the numbers don’t look good or if growth slows down, investors might flee. They’re in a “silence period” because the data might not be as attractive.
  • Price War Reducing Profits: Although the revenue from AI-generated content (tokens) is high, prices have dropped by more than 50%. To maintain revenue, they need to double usage, but doubling usage doesn’t necessarily mean doubling profits, as there are costs associated with servers, electricity, and maintenance. In other words, they need to sell more at lower prices just to break even or even lose money.
  • Lack of a Second Growth Driver: The most profitable use case for AI currently is coding. However, coding itself doesn’t generate revenue; it just makes programmers work faster. As a result, companies are replacing more programmers with AI, leading to job losses. Even if more code is written, it doesn’t create new revenue to support the AI infrastructure. It’s like building a super-efficient loom only to find that no one buys the fabric, or buyers are unwilling to pay a high price, leaving the loom owner in trouble.

In summary: The giants are claiming concern for safety because their business models aren’t sustainable. If this continues, investors might withdraw their funds, and their stock prices could plummet. By calling for a stop, they’re trying to create an excuse of “force majeure” – “It’s not that we can’t continue, but the government has ordered us to.”

---

2. A Clever “Political Excuse”: Turning Business Difficulties into a Moral High Ground

It would be embarrassing to admit directly, “We’re out of money and need to take a break.” But claiming it’s for human safety changes the narrative completely.

This is what’s known as a “respectable exit.” The author made a great analogy: it’s like the Cold War-era Intermediate-Range Nuclear Forces Treaty between the US and the Soviet Union. When the Soviet Union was struggling, it proposed to destroy its missiles, ostensibly for world peace, but in reality, it was a signal of weakness.

OpenAI and Anthropic’s strategy might be:

  • Forcing Government Action: They want the US government to intervene and introduce policies to restrict AI development under the guise of “national security” or “human survival.”
  • Shifting the Blame to Force Majeure: If the government imposes restrictions, they can say to investors, “See, it’s not that we don’t want to expand, but the government is stopping us. This is a political risk, not a business failure.”
  • Attacking Competitors: If Chinese AI models continue to advance rapidly, they can accuse China of “recklessly developing dangerous technology,” isolating China internationally and possibly leading to sanctions from the Western world.

In simple terms: It’s like two boxers who are tired and one of them shouts, “Referee! The other guy is hitting too hard and could cause injury! Please stop the match!” With the referee’s stoppage, they can catch their breath and even accuse the other boxer of being aggressive.

---

3. Trump’s “Winning Philosophy” vs. AI Giants’ “Brake Application”: A Political Game

Here’s a significant variable: Donald Trump. If the AI giants really wanted to slow down, they would have to consider Trump’s stance. Trump’s philosophy is “America First” and “the winner takes all.”

  • Performance Dependency: Trump has made “making America the leading AI power” one of his major achievements. If he suddenly claims that AI is too dangerous and needs to be slowed down, where would his achievements lie? How would he perform in the midterms?
  • Political Material: Trump needs enemies and victories. If AI development slows down, he loses a platform to showcase America’s technological dominance.
  • Interests at Stake: Trump has a good relationship with Elon Musk (CEO of Tesla and Nvidia, a major AI chip manufacturer). If AI giants stop buying Nvidia’s chips, Nvidia’s stock price could drop, undermining Trump’s narrative of American technological success.

Therefore, Trump is unlikely to agree to a slowdown. He would say, “Don’t talk about human safety; what I want is for America to win! AI must accelerate. Anyone who slows down is a traitor!”

This means that the AI giants’ attempt to use political pressure to justify their actions might meet resistance. Trump wants AI to accelerate, not to slow down.

---

4. China’s “Fearless Spirit”: The Drive to Avoid Being Left Behind

The article mentions a profound point: Chinese AI companies are not falling behind and are still catching up, with a gap of 3-6 months. Why are they so competitive?

The driving force is a sense of urgency stemming from historical trauma: For over a century, China has been bullied due to technological backwardness. When AI emerged, their first thought wasn’t whether it would destroy humanity, but rather, “If I have it and you don’t, I can protect you; if you have it and I don’t, you could use it against me.”

  • A Competitive Mindset: Since everyone sees AI as a key future technology, the first to stop could be eliminated. China is using price wars to squeeze out competitors, aiming to wear them down or at least prevent them from making easy profits.
  • The Result: Even if OpenAI and Anthropic call for a stop, Chinese companies are unlikely to follow. For China, stopping means losing; continuing means survival. This asymmetric competitive mindset will keep the AI arms race going.

---

5. Lessons for Us All: This is a “Game for the Fearful”; Don’t Be Duped

The author offers two pieces of advice for Chinese AI practitioners and us all:

  • Survival of the Fittest in a Narrow Path: The current AI competition is like two trucks heading towards each other on a single lane, both accelerating. The one that slows down first loses. OpenAI and Anthropic want to slow down, but China won’t. So, the game isn’t over; it might even intensify.
  • Be wary of Universal Values: When European and American countries use “human safety” or “global ethics” to persuade you to slow down, be cautious. It’s often not out of kindness but because they see their own weaknesses (high costs, unclear business models, political pressure, etc.) and want to use moral arguments to restrict your progress.
  • Historical Lessons: The Carthaginians surrendered their weapons, thinking it would bring peace, only to be completely destroyed by Rome. In technological competition, “safety” is often a tool used by the strong to control the weak.

Implications for Us:

  • Investment: The AI sector will experience greater volatility. The giants’ calls for a stop might cause short-term stock price drops, but in the long run, since China won’t slow down, competition will continue, and technological progress won’t slow. Don’t blindly believe in a “AI bubble burst” theory, nor should you chase high prices.
  • Career: The impact of AI on employment is real. Jobs in coding, content creation, and design will be affected. The focus should be on learning to use AI to become someone who can leverage it, not someone who gets replaced by it.
  • Mindset: Stay clear-headed. Don’t be scared by fear-mongering about AI destroying humanity, nor be blinded by AI’s supposed omnipotence. AI is a tool, and its fate depends on how it’s used and the political dynamics behind it.

In conclusion: OpenAI and Anthropic’s calls for a stop are not about fear of AI’s potential harm but about avoiding financial collapse. China won’t stop because it fears falling behind. This “game for the fearful” continues, and we need to understand the underlying commercial motives and be prepared for the technological changes ahead.