When AI Giants Call for a Stop: A Power Realignment about “Who Makes the Decisions”
Hello everyone, I’m your financial journalist. Today, we’re going to discuss a very insightful article from Havenlon Labs.
The main argument of this article is quite profound: For the past few years, the AI industry has been about who can develop the fastest technology. But now, even the CEOs of the leading AI companies, such as Dario Amodei from Anthropic, are publicly advocating for a slowdown.
This isn’t just because everyone is scared or for safety reasons alone. Behind this is a significant reallocation of power and responsibility. In the past, AI companies decided what to develop, how to develop it, and when to release it. They kept the profits for themselves and bore the consequences of any problems that arose. However, the impact of AI is now so significant that if something goes wrong, the entire society could be affected. Therefore, society is beginning to ask: If the consequences are shared by all, shouldn’t the decision-making power also be shared with the public and the government?
To make this easier to understand, I’ve broken down the long article into five key points and explained them in plain language.
---
Point 1: The Core Conflict: Private Decisions, Public Disasters
[Simple Explanation: In the past, if your house caught fire, it was your problem. Now, if your house catches fire and it burns down the entire street, you’re still only responsible for compensating for the damage to your own house.**
The article points out that the biggest issue in the AI industry is not a lack of technology but a mismatch between power and responsibility.
- Current Situation: Giants like OpenAI and Anthropic have enormous private power. They decide how powerful their models should be, when to release them, and how much autonomy to give to the AI systems. These decisions are made internally, with the outside world having no knowledge of them.
- Problem: AI has moved beyond being just a chatbot; it’s now involved in finance, healthcare, cybersecurity, and even the military. If AI makes a mistake or is exploited by hackers, the consequences could be widespread, leading to mass unemployment, financial collapse, or a cyber blackout.
- Conclusion: This is what’s known as “private decisions, public consequences.” While companies used to be responsible for their own successes and failures, their decisions now affect everyone. Since the outcomes are shared, it’s only fair for the government and the public to have a say in the decision-making process.
---
Point 2: Historical Perspectives: Scientists Once Had the Privilege to Define Danger
[Simple Explanation: In the past, only scientists knew how to build nuclear bombs, so they had the power to set the rules. Now, only AI companies know how to create these models, so they want to define the rules. But is this fair?**
The article cites two historical examples to explain why it’s so difficult to regulate AI now:
1. 1975’s Asilomar Conference on Genetic Engineering: When recombinant DNA technology emerged, it was very dangerous. The government and the public didn’t understand it, only the scientists did. Therefore, society had to listen to the scientists and let them decide which experiments could proceed and which could not. The key point is: Those who understood the risks first had the power to define them.
2. Nuclear Weapons After 1945: After the development of nuclear energy, the U.S. took control away from the military and handed it over to civilian agencies because neither scientists nor the military were suitable to wield such destructive power.
[Applied to AI: The situation today is similar to 1975, but more complex. The people with the most advanced technical knowledge are private companies that have received substantial funding and are tasked with national strategic goals.**
- Anthropic’s Proposal: They suggest inviting external teams (like METR) to conduct inspections to address the issue of only insiders knowing the details.
- Opponents’ Concerns: Critics like David Sacks question whether these external teams have conflicts of interest with the companies. If regulators rely on the companies they are supposed to regulate to know what to oversee, is the regulation really independent?
---
Point 3: Political Dynamics: The Triangle of Security, Monopoly, and National Competition
[Simple Explanation: Some want to slow down for safety reasons; others want to prevent China from catching up; still others want to stop monopolies by large companies. Everyone talks about safety, but their motives are different.**
The article analyzes the positions of three key figures and finds that they are not on the same page:
- Dario Amodei (Anthropic CEO) – “Systemic Risk Camp”:
- Argument: We must slow down.
- Reason: AI is developing too fast, and safety cannot keep up. If a major accident happens, no one will be responsible.
- Approach: Introduce external evaluations, coordinate within democratic countries first, and then discuss global coordination.
- Implication: I recognize the power of AI, but I’m concerned about its potential out-of-control nature. I also want the U.S. to remain ahead, so we need to compete while ensuring safety.
- David Sacks (Former White House AI Advisor) – “Market and Anti-Monopoly Camp”:
- Argument: If you want to slow down, do so on your own and don’t hold back others.
- Reason: Don’t use “safety” as an excuse to create new approval processes that actually raise barriers for the industry and protect large companies.
- Core View: If a product fails, let legal product liability mechanisms take effect; there’s no need for pre-approval by the government. He also questions the independence of external evaluation agencies.
- Implication: Don’t use regulation to create monopolies. Large companies claiming to need slower development may actually be trying to prevent new players from entering the market.
- Donald Trump (Former President/Political Figure) – “National Competition Camp”:
- Argument: Reduce regulation to maintain the U.S. lead over China.
- Reason: Strict regulation could cause the U.S. to fall behind in technology.
- Implication: My goal is not AI safety but U.S. dominance. If slowing down means losing to China, I’m against it.
[Summary: This is not a simple conflict between “safety advocates” and “acceleration supporters”; it’s about technological governance, market competition rules, and national security strategies all being intertwined.
---
Point 4: The Economic Impact: Slowing Down is a Small Cost for Big Companies, but a Death Sentence for Small Ones
[Simple Explanation: If all cars have to stop for brake checks, a wealthy truck can afford to wait for half a year without issue, but a small, cash-strapped van might go bankrupt. Is that fair?**
The article uses the 2008 financial crisis as an analogy to raise a profound economic question: “Too Consequential to Self-Govern.”
- Surface Fairness: If all AI companies are required to conduct additional safety tests for half a year, it seems fair.
- Actual Injustice:
- Giants (OpenAI, Anthropic): With billions in cash and well-established infrastructure, a six-month pause is no problem; they might even use this time to strengthen their competitive advantages.
- Startups: Without sufficient funding, their cash might only last for 12 months. A six-month halt could lead to bankruptcy.
- Compliance Costs: Setting up new safety systems could cost tens of millions of dollars. For giants, this is a cost; for new entrants, it’s a bar to entry.
- Deeper Logic: If AI companies are treated like regular products, Sacks is right: Let the market punish them if they fail to meet the standards. But if AI companies are as crucial as banks and cannot fail, then Amodei is right: The market mechanism is failing, and collective action is needed.
- Dilemma: If a company slows down for safety, competitors might take market share. Does the market really reward caution or punish it? This is a classic example of a collective action dilemma.
---
Point 5: The Ultimate Question: If Slowing Down Is Wrong, Who Is Responsible?
[Simple Explanation: We often worry about the risks of accelerating, but we rarely consider the risks of slowing down. If the brakes are applied incorrectly, who will compensate for the consequences?**
This is the most interesting and often overlooked point in the article: The Cost of Action vs. The Cost of Inaction.
- Risks of Acceleration: AI mistakes can lead to cyberattacks and unemployment; these are easy to quantify and report in the news.
- Risks of Slowing Down: AI could have earlier discovered cancer treatments or accelerated drug development. If overcaution delays this by five years, the people who die from missing out on these treatments will not be recorded in any statistics. Their suffering is invisible.
[Core View:**
1. Slowing down is not free: It buys time, but time itself does not guarantee safety. What if the models become stronger a year later?
2. What’s Needed is a “Framework,” Not Just Brakes: The article cites the aviation industry as an example. Aviation safety is not due to slow development but to independent accident investigation agencies (NTSB), clear certification standards, and redundant designs.
- The NTSB in aviation does not have regulatory power; its role is to uncover the truth.
- What the AI industry lacks is an independent verification mechanism and a clear chain of responsibility.
3. Mutual Responsibility: If the government or the public gains the power to “apply the brakes,” they must also be responsible for any mistakes made in doing so. If excessive regulation leads to technological backwardness or delays in life-saving technologies, the government must also bear the responsibility.
---
Journalist’s Conclusion: From a “Technology Race” to a “Regulatory Race”
This article concludes that the AI industry has crossed a critical point.
Previously, we asked: How much smarter can AI become? Now, we need to ask: Who has the authority to decide when this technology can be released and how it can enter society? And if the direction is wrong, who is responsible?
- Authority: It used to be in the hands of companies; now it needs to be shared with the government, the public, and third-party agencies.
- Responsibility: It used to be borne by companies; now it needs to be shared by society as a whole.
- Risk: It used to be about technical risks; now it’s about systemic risks.
- Benefit: It used to be about company profits; now, it must also consider social welfare.
In one sentence: AI hasn’t slowed down, but it has shifted from a “technology race” to a “regulatory race.” The outcome of this debate is not about who wins or loses, but about redefining the social contract regarding who has the power to accelerate or slow down and who is responsible if things go wrong.