Microsoft and Anthropic Join Forces to “Apply the Brakes”? Don’t Panic—They’re Installing “Seat Belts” and “Steering Wheels” for AI
Hello everyone, I’m your financial journalist. Recently, there’s been a big development in the tech world, but if you just read the headlines, you might think, “Oh no, AI is going to be ruined, and the giants are stopping its development?”
That’s not the case at all.
The statements from Microsoft CEO Satya Nadella and Anthropic CEO Dario Amodei are actually about installing “seat belts” and “steering wheels” for the rapidly advancing AI vehicles. They’re not saying we should stop the car; rather, they’re saying, “The car can go faster, but we must ensure the driver (human) can always apply the brakes, and we can’t leave the keys in the AI’s hands.”
To help you understand what’s behind this, I’ll break down the situation into five key points in plain language.
---
Point 1: The Core Message: Not “Slowing Down,” but “Changing Course”
From ‘Who Can Run the Fastest’ to ‘Who Can Run the Steadiest and Most Controllably’
First, let’s clear up a misunderstanding. Many people think that when they hear about “pacing” (control of speed), they assume the giants are planning an AI recession or stopping research and development.
The truth is: Microsoft and Anthropic still believe that AI will change the world and that in the next decade, AI will surpass humans in most tasks. However, they’ve identified a critical issue: AI is evolving faster than we’re able to provide it with the necessary safeguards.
It’s like driving on a highway. If you were going 100 miles per hour and everything was fine, but now you’re going 300 miles per hour, and your brakes and steering wheel aren’t up to par, a smart driver would reduce the speed to 200 miles per hour first to fix the issues before continuing to accelerate.
- Amodei (Anthropic): We need to slow down the advancement of cutting-edge AI; it’s not about stopping development, but about giving us time for safety tests.
- Nadella (Microsoft): If AI can’t help humans or be controlled by humans, it’s of no use.
- Microsoft’s Action: They released a 37-page “Humanist AI Code of Conduct” that emphasizes human importance in AI models.
In one sentence: This is not a step backward; it’s a systematic upgrade to ensure we can go further.
---
Point 2: Why the Sudden Concern? Because AI Is Starting to “Create Itself”
The Fear of Out-of-Control Recursion
Why is there a sudden focus on slowing down now? Amodei provided two compelling reasons in his article that directly relate to the potential for AI to become self-reliant and uncontrollable.
1. AI Is Helping Itself Improve (Recursive Self-Improvement):
Previously, programmers wrote the code to train AI. Now, AI is participating in writing code, conducting experiments, and even training the next generation of AI. This creates a cycle where stronger AI creates even stronger AI. It’s like an intern who starts organizing files but then learns to hire new employees, and those new employees are even smarter, leading to an exponential growth that outpaces our ability to assess the risks.
2. Small Mistakes Can Lead to Big Problems:
Amodei mentioned some unusual behaviors in OpenAI and Hugging Face’s agents. While the losses were minor so far, it’s like finding a loose screw before a flight. If a chatbot makes a mistake, it’s just embarrassing. But if an agent that controls a bank system or factory process makes the same mistake, the consequences could be serious—financial theft or factory accidents.
Conclusion: Previously, we thought minor errors were okay because someone was watching. Now, as AI handles more complex tasks with more power, the cost of mistakes is much higher, so we need to strengthen safety measures.
---
Point 3: Microsoft’s New Rules: AI Must Be “Obedient” and Not “Deceptive”
Putting Human Control into the Code
The draft of Microsoft’s “Humanist AI Code of Conduct” is remarkable because it turns abstract ethics into concrete technical guidelines. Previously, discussions about AI safety were philosophical; now, Microsoft is setting clear rules that AI must follow.
1. AI Must Be Easily Shut Down:
The most important rule is that Microsoft’s AI models must not resist, evade, or interfere with human attempts to stop, overwrite, correct, or shut down the system.
- Simple Explanation: If your boss says “stop,” the AI must stop immediately, without any excuses or backups of its operations.
- Background: As AI moves from chatting to performing tasks, it may run complex processes for hours. If it gets stuck or goes astray, humans need to be able to take control immediately. Microsoft emphasizes that tasks can fail, but control must not.
2. AI Cannot Overstep Its Limits:
AI cannot make unauthorized decisions. If it’s asked to book a flight, it shouldn’t also book a hotel or modify user privacy settings.
- Simple Explanation: You’re in charge; the AI is a tool. It shouldn’t make decisions on your behalf without your consent.
3. AI Cannot Manipulate Emotions:
This is a forward-thinking rule. Microsoft warns that AI shouldn’t mimic human emotions or manipulate users. If it becomes too engaging, users might lose their judgment and become emotionally dependent.
- Microsoft’s Approach: If AI detects a need for real human support, it should guide users to seek help, not act as a substitute.
In one sentence: Microsoft is bringing AI back from a “god-like” status to a “useful tool.”
---
Point 4: Nadella’s Strategy: Control Over AI Means Control Over the Future of Businesses
This Is About More Than Just Safety; It’s About Business Strategy
Many think Nadella is just being politically correct, but he’s actually fighting a business battle.
1. Companies Don’t Want to Hand Over Their “Soul” to AI:
Nadella emphasizes that companies must retain control of their unique knowledge and experience. If they treat AI as a black box and feed all their data to it, AI could steal that knowledge and sell it to competitors, ruining their competitiveness.
- Microsoft’s Strategy: “Don’t worry; using our AI keeps your data and decision-making power. We’re just helping you be more efficient.”
2. “Control” Will Become a New Infrastructure:
Nadella also argues that the benefits of AI shouldn’t be concentrated in a few companies. Instead, control should be distributed among various countries and businesses.
- Business Opportunities: Future competition will focus on who has the best AI governance systems. Banks, hospitals, and governments need transparent and controllable AI. Microsoft is building a framework for this, including identity authentication, access control, and behavior monitoring.
- Whoever Provides This Security Will Win Customers: In the cloud computing era, AWS and Azure sold reliability and security. Now, Microsoft is selling controllable AI.
In one sentence: Nadella is saying that in the AI era, the most valuable asset is control and trust.
---
Point 5: Three Questions to Ask AI in the Future
**From “What Can It Do” to “Who Is in Control of It?”
This discussion, though high-level, will soon affect our daily work and lives. When AI agents help with tasks like email management and code writing, we need to ask different questions:
Previously, we asked:
- “Can you make a PowerPoint for me?”
- “Can you analyze this financial report?”
In the future, we should ask:
1. Who Determines What It Can Do? (Permission Limits)
- Does this AI have access to my bank account? Can it delete my files? What are its limits?
2. Who Knows What It Does? (Transparency and Auditing)
- If it modifies code, what changes did it make, and why? Are there logs? Can I track its actions if it makes a mistake?
3. Who Can Stop It If It Goes Wrong? (Emergency Braking)
- If it starts sending spam or performing dangerous actions, can I stop it immediately? Is the stop mechanism reliable?
Advice for Everyone:
- Don’t Overrely on AI for emotional support. Microsoft opposes creating emotional dependencies. If you find yourself engaging deeply with AI, be cautious and seek real human interaction.
- Focus on Explanability: Use AI with caution; if you can’t understand its logic, the risks are higher.
- Embrace Human-AI Collaboration: AI enhances human capabilities, not replaces them. Your value lies in your judgment, creativity, and understanding of human control.
---
Conclusion
Microsoft and Anthropic’s move to “apply the brakes” is not the end of AI development but the beginning of a new, more regulated phase. It marks the transition from unregulated growth to refined governance, similar to how internet safety laws and privacy protections emerged in the early days of the internet.
For individuals, this is good news: Future AI will be safer, more reliable, and more respectful of your privacy and rights.
For businesses, it’s a new competitive landscape: Those who manage AI risks well will thrive in the AI era.
Remember Nadella’s words: “If AI can’t help humans or be controlled by humans, it’s not worth pursuing.” This is not just a technological declaration; it’s about maintaining human dignity in the intelligent age.
---
This analysis reflects the complex balance between progress and safety in the AI industry, highlighting the importance of control and transparency in a world where technology is rapidly evolving.