When AI Giants Start Shouting “Stop”: A Game of Life, Death, Money, and Power
Hello everyone, I’m your financial journalist. Today, we’re not talking about some dry technical report; instead, we’re covering a major drama that’s playing out intensively between Silicon Valley, Washington, and Wall Street.
If you’ve recently come across headlines like “AI could destroy humanity” or “AI companies need to slow down,” don’t rush to scroll past them. Behind these headlines are the world’s top tech giants, politicians, and capital moguls, all fighting over how AI should be developed.
In simple terms, while before it was about who could develop the fastest technology, now some are shouting “brake,” while others want to step on the accelerator, and some even want to change the game rules. At stake are humanity’s fate, trillions of dollars in market value, and the balance of power between nations.
Let me break down this complex situation in plain language and show you how this chess game is being played.
---
The Trigger: A Young Man’s Resignation Letter That Set Off a Wave of Panic
It all started on September 8th when Jacob Coxon, a 27-year-old software engineer from Anthropic (a company focused on AI safety), resigned.
You might think it’s common for programmers to change jobs, but Jacob’s resignation was particularly significant. He posted a statement that caused a huge stir: “AI could kill all of us within a decade.”
This statement exploded on social media, receiving 170 million views. Why such a big reaction? Jacob is no ordinary employee; he has worked at both OpenAI and Anthropic and is one of the people closest to the core of AI research. He accused these companies of engaging in an “irresponsible race” to be the first to develop advanced AI, risking the lives of all humanity.
Soon after, another former Anthropic employee echoed his concerns, saying, “We might not survive this.” The current AI systems are becoming much smarter than humans, but no one can guarantee they will always act according to human commands.
Even more alarming, a senior security expert from Anthropic admitted that the probability of AI wiping out humanity within the next decade is over 10%. That’s like playing Russian roulette with one out of six bullets being lethal—only 16.7% chance of survival. In the eyes of these experts, the risk of AI getting out of control is as high as facing death itself.
Key Point: This isn’t science fiction; these are the people in charge of the most powerful AI models admitting that we might no longer be able to control them.
---
Silicon Valley’s “Brake Pedals”: Giants Suddenly Want to Slow Down
Faced with this panic, Anthropic’s CEO, Dario Amodei, couldn’t sit back. On September 12th, he published a lengthy article with a straightforward title: **“We Must Put the Brakes on Advanced AI.”
Dario listed three main concerns:
1. Out of control: Humans might lose complete control over AI.
2. Abuse: AI could be used for large-scale cyberattacks or biological terrorism.
3. Economic Disruption: Rapid technological progress could disrupt jobs and the economy.
To address these concerns, he proposed a set of measures, similar to putting surveillance in place for AI systems:
- Independent Audits: Third-party organizations should regularly inspect AI labs to ensure compliance.
- Common Safety Standards: Leading companies should work with the government to establish unified safety regulations. Any violator would be forced to stop operations.
- Global Cooperation: This wouldn’t just be the U.S.; democratic countries should join in, and coordination with countries like China would be necessary to ensure everyone follows the same rules.
OpenAI’s CEO, Sam Altman, and Google’s Demis Hassabis also expressed support for these measures. However, there’s a significant contradiction: although everyone talks about safety, who will set the rules?
- Dario (Anthropic): Supports strong government involvement, believing the government has the authority to shut down unsafe AI systems.
- Hassabis (Google): Favors a self-regulatory body similar to the FINRA in the financial industry, with the government playing a supervisory role.
- Altman (OpenAI): Is more pragmatic, suggesting that if the government fails, the leading companies should take the lead in establishing the rules.
The underlying message is clear: whoever controls the rule-making power will have significant influence over the future of the AI industry. Early rule-makers could use their technological advantages to create standards that later competitors would have to comply with, effectively controlling market entry.
---
Washington’s “Accelerator Pedals”: Politics and Electionary Calculations
While Silicon Valley is calling for a slowdown, Washington (the U.S. government) is pushing for acceleration.
On September 13th, Trump publicly responded to the controversy, stating, “Whoever wins in the AI race will win everything.” In his view, AI is not about ethics but about outperforming China. He believes the U.S. is currently ahead and that slowing down due to safety concerns would mean ceding that advantage. He labeled those warning about AI risks as “very negative forces” spreading unfounded fears.
Why is Trump so eager?
- Midterm Elections: The 2026 midterm elections are less than two months away, and AI is a hot topic.
- Voter Sentiments: Some voters (supporters of MAGA) fear AI will take jobs and consume resources. Trump is also concerned that falling behind China would erode public confidence in America’s strength.
Thus, Trump is playing the card of “competition with China.” He believes that as long as China continues to advance in AI, the U.S. cannot stop. Even if AI poses risks, it’s better than losing to China.
This creates a dilemma:
- Silicon Valley: Worried about losing control and advocating for global regulation.
- Washington: Fearful of falling behind and pushing for faster development to maintain a competitive edge over China.
- The Result: Policy is uncertain, as both want to regulate AI but also don’t want to harm economic competitiveness.
---
Wall Street’s “Heartbeat”: Valuation Bubble and Capital Games
While Silicon Valley focuses on safety, Wall Street is all about money.
In the past few years, AI companies’ valuations have skyrocketed because of the belief that stronger AI models would attract more users and generate more revenue, leading to increased investment in data centers and GPUs. Now that CEOs are suggesting a slowdown, Wall Street reacted quickly:
- Stock Prices Plummeted: AI-related stocks dropped by 13%.
- Re-evaluation: If AI development can’t continue at its current pace, all previous financial projections need to be reassessed. Will data centers and GPUs still be needed in such large quantities?
Here’s a ironic twist:
- OpenAI: Decided to postpone its IPO. Sam Altman hinted that if safety issues need to be resolved, the company might delay its listing.
- Anthropic: However, it’s accelerating its efforts to go public on NASDAQ, with a target valuation of $2 trillion and Nvidia investing $10 billion.
Why would Dario, who advocates for a slowdown, want his company to go public faster? Because regulation can be a competitive advantage. If AI is strictly regulated in the future, small companies will struggle to comply and may be forced out. Only giants like Anthropic, OpenAI, and Google will have the resources and expertise to meet the new safety standards. This means that higher regulatory barriers will protect their market dominance.
---
The Echo of History: Can We Really Control It?
Jacob Coxon’s resignation highlighted a hidden anxiety: similar warnings were made ten years ago by Ethereum founder Vitalik Buterin, but they were dismissed as theoretical exercises. Now, these concerns are being voiced by CEOs and included in government agendas.
This reminds us of a letter Einstein wrote to President Roosevelt in 1939, warning that Germany might develop a nuclear bomb, and the U.S. needed to act. Einstein later regretted signing it, knowing the potential destruction of such a weapon.
The current situation is similar:
- We’re aware of the dangers of uncontrolled AI.
- We also know that competitors (like China) are working hard to develop similar technologies.
- Stopping might mean losing; not stopping could lead to destruction.
This is the so-called “security dilemma”: everyone talks about slowing down, but the real question is whether we’ll sacrifice progress for safety or risk everything for faster development.
In summary, this controversy is about more than just AI safety; it’s a struggle for power, capital, and geopolitical influence.
- For ordinary people, it means AI development might slow down but become more regulated (or more monopolized by giants).
- For investors, it means the high-growth narrative of the AI industry could be challenged, leading to greater volatility.
- For humanity as a whole, we’re at a crossroads: do we choose safety or risk everything for faster progress?
For now, no one has a definitive answer. But one thing is clear: AI has become too important to allow anyone to enter the game without proper oversight. In the coming months, we’ll see how regulatory rules will be implemented and how the giants will balance making money with protecting their interests.
This is not just about technology; it’s about the political and economic future of our lives. Stay tuned for more updates.