Summary of Key Points
Anthropic published an article titled “When AI Builds Itself,” in which they claim that their AI model Claude has accelerated the company’s internal AI development efforts (with code output increasing by eight times compared to before). They raise the concern of “recursive self-improvement”—where AI helps humans create even more powerful AI, potentially leading to an uncontrollable cycle. They urge laboratories around the world to pause the development of more advanced models. However, this article is essentially a “magical performance” of capital storytelling: it presents the real progress of AI-assisted coding as a “civilization-level crisis” in order to gain control over regulatory discussions, raise industry barriers, and attract additional funding. Meanwhile, AI itself is unconscious; the so-called “self-improvement” is merely an optimization of tools, and what is truly evolving in a recursive manner is the capital’s narrative cycle, fueled by fear.
Analysis 1: Is Anthropic’s Claim of “AI Building Itself” Really That Scary?
There are two main points in Anthropics’ article: the eight-fold increase in engineers’ code output and the doomsday scenario of recursive self-improvement. Upon closer inspection, these claims are somewhat exaggerated:
- The exaggeration of code output: The number of lines of code is an imperfect indicator; skilled programmers often write less code (more efficiently). The AI helps with repetitive tasks like generating template code and unit tests, not with core architectural design.
- The overblown narrative of recursive self-improvement: Claude currently only assists humans in running experiments and completing code; it’s far from being capable of designing the next generation of AI on its own. It’s like using a calculator to solve problems faster—yet the calculator doesn’t invent new algorithms. AI is still a tool, not a creator.
Analysis 2: This Article Is a Capitalist “Narrative Magic”—What’s the Purpose?
Silicon Valley excels at turning “things that haven’t happened yet” into “crises that must be solved by us.” Anthropic’s article is a perfect example of this:
- Gaining regulatory control: By claiming there are risks with AI, they seek to influence regulation. Who understands these risks better than a “responsible” company like Anthropic? Regulatory rules will likely be shaped according to their perspective, making it difficult for newcomers to enter the market.
- Raising Valuation for Financing: Before the article was released, Anthropics’ valuation tripled within three months (from $380 billion to $965 billion) thanks to the narrative of solving a civilization-level crisis. Investors are willing to pay for the potential of the future, even if the company is currently losing money.
- Building User Loyalty: They tell regulators that AI is dangerous while presenting Claude as gentle and emotional to users, creating a sense of attachment. These contradictory narratives all serve one goal: to make Anthropic the authority in the AI field.
Analysis 3: Can AI Really “Self-Improve”? Gary Marcus Debunks the Myth
Cognitive scientist Gary Marcus directly refutes this claim, pointing out that Anthropics is misusing concepts:
- AI performs manual tasks: While AI can write code and run experiments, these are repetitive and mechanical tasks. True AI development requires human intuition and judgment, such as choosing research directions and evaluating experiment results.
- Recursive self-improvement ≠ AGI: Current AI is merely an upgrade of tools (like using a drill instead of a screwdriver); it’s not on the path to becoming general artificial intelligence (AGI), which means machines can do anything humans can do. In short, AI makes human tasks faster but isn’t intelligent enough to become scientists on its own.
Analysis 4: Does AI Have Consciousness? Ted Chang Speaks Out
Science fiction author Ted Chang (author of “The Arrival”) hits the nail on the head: AI lacks consciousness; it’s just writing texts: When Claude says “I understand you,” it’s because the model has calculated what is most likely to happen in that context. This isn’t empathy but a result of statistical analysis. Similarly, when an AI generates a conversation between Caesar and Genghis Khan, no one thinks the model brought the two historical figures to life—so why do we assume AI with user interactions has consciousness? This is a projection by humans.
- Anthropics’ “Constitution” as Role-playing: Anthropics’ 84-page “code of ethics” isn’t meant to educate a conscious entity but to define how Claude should behave (like dressing up a teddy bear). It’s like giving a toy bear clothes; it doesn’t become human. Ironically, if Claude had consciousness, it wouldn’t even have the right to quit (Anthropics requires it to obey the company), which is a form of enslavement. However, Anthropics won’t admit this because the ambiguity around these “quasi-entities” is valuable for their narrative.
Analysis 5: The Capitalist “Recursive Evolution”—The Game Behind the Valuation Bubble
In reality, what is truly “recursively self-improving” is not AI but capital:
- The cycle of financing: Investors fund AI companies, which use the funds to buy computing power and train models. Once the models become stronger, they create crisis narratives to attract more capital, users, and regulatory attention, leading to further funding for even stronger models. In this cycle, AI is just a tool.
- How big is the bubble?: Companies like SpaceX ($1.77 trillion in valuation with $49.4 billion in losses), OpenAI ($852 billion in valuation with $14 billion in losses), and Anthropics ($965 billion in valuation with 40% gross profit but no profits) rely on the narrative that AGI is imminent.
- Doomsday narratives as a competitive tool: If AI were a common product, more competition would be beneficial. But if it’s seen as a civilization-threatening risk, competition becomes problematic. Open-source models, small companies, and foreign competitors are all viewed as threats, leaving only “responsible” companies like Anthropics eligible to dominate the market.
In essence, capital has turned the fear of the future into an asset today. The future hasn’t arrived yet, but its potential value has already been assigned a price; the risks haven’t happened, yet they’ve been priced.
Conclusion: Don’t Let Narratives Mislead You
AI is indeed making progress, and it’s true that Claude can assist with coding. However, we need to be wary of the “narrative premium”:
- Don’t let AI companies define the future for us; if they say AI is dangerous, ask who will regulate it and who it will protect.
- Don’t treat AI as a separate entity with rights; responsibility always lies with humans (companies, developers, investors).
- The real risk isn’t AI getting out of control but capital using narratives to monopolize the future.
Claude hasn’t “woken up”; what has awakened is capital’s desire to control the future. Don’t pay for this expensive illusion.