Summary of Key Points
The article “When AI builds itself” published by Anthropic uses internal real data to demonstrate how AI (Claude) is deeply involved in its own development process. Currently, 80% of Anthropic’s production code is generated by Claude, shifting the role of engineers from code writers to code reviewers. The article suggests that AI is evolving at an increasingly rapid pace, but the title may mislead readers into thinking that AI has achieved self-evolution. There are two underlying implications: first, while AI’s development is indeed accelerating, humans still hold the key to critical decision-making; second, Anthropic’s call for regulation on the eve of its funding and listing could be a strategy to lock in its competitive advantage.
1. Engineers Moving from Code Writers to Reviewers
Now, 80% of the code at Anthropics is written by Claude, and engineers are submitting code eight times more frequently than last year (although the number of lines does not necessarily reflect quality; the reviewed logic is sound). This signifies a complete change in the way programmers work. Previously, they wrote the code, tested it, and submitted it. Now, they define the goals for AI, which then writes the code, runs tests, and fixes bugs, with humans focusing on ensuring the direction is correct and there are no issues with the results.
To put it simply: in the past, a director had to operate the camera and edit the footage; now, with ten AI teams at their disposal, the director only needs to specify the scenes to capture, point out errors, and signal when to stop. The value of humans lies no longer in their ability to execute tasks but in their understanding of what needs to be done and how to assess whether it is right or wrong. This is similar to how, with the widespread use of smartphones, the quality of photos depends not on the technology behind the shutter button but on the “vision” of the person choosing the angle and subject.
Conclusion: The value of execution is declining; “judgment ability” will become the core competitive skill in the future.
2. A New Bottleneck: Can You Verify AI’s Results?
While AI can generate large amounts of code, solutions, and experiments, the challenge lies in humans’ ability to review them all. Using AI to review AI-generated code has helped identify one-third of online bugs early on. However, there is a potential issue: when Claude reviews Claude’s work, it’s like students grading each other’s exams, which may lead to a failure to detect certain types of errors.
More importantly, even if AI makes the right decisions, how do you know why? For example, if an AI feature works properly, you might not be sure whether there are hidden risks in its logic. The real challenge for teams in the future will not be a lack of ideas but identifying which of the many AI-generated options is truly viable and which ones are just on the wrong track.
Conclusion: The most scarce skill in the future will not be “using AI” but “verifying AI’s results”—being able to select the right solutions, detect hidden flaws, and know when to stop using AI.
3. AI Evolving Faster than Humans Can Keep Up
The article provides data showing that the time it takes for AI to complete complex tasks has halved: from seven months to four months. In March 2024, an AI could only handle a task lasting four minutes; by March 2025, it could handle one and a half hours; by March 2026, twelve hours. Although the success rate is only 50% (and thus not entirely reliable), the trend is clear: AI’s iteration speed is accelerating. It might soon be able to develop the next generation of AI on its own, limited not by the number of humans but by computing power and hardware.
However, there is a gap between the rapid pace of AI and the slower pace of human society. For instance, while AI can quickly develop new drugs, drug approval processes take decades due to the need to verify side effects; AI can rapidly modify code, but user habits take time to establish (similar to how the brain’s memory functions require extensive user feedback for optimization). These “slow variables” (such as laws, education, and user behavior) cannot be replaced by AI. This is the real concern for ordinary people: it’s not about jobs being automated, but about society’s rules failing to keep up with technological changes.
Conclusion: If your work involves “slow variables” (such as understanding users, refining scenarios, and building trust), these will serve as a safety net for you.
4. The Title Is a Deceptive “Smoke Bomb”
The title “When AI builds itself” is sensational, but the reality is that AI currently assists in development, not creates itself entirely. True self-evolution would involve AI identifying its own weaknesses, proposing solutions, and training itself to deploy them. Anthropic acknowledges that they have not yet reached this stage.
For example, the creation of the Transformer architecture was facilitated by a chance conversation between Google engineers in a hallway, which was then taken up by the visionary Samer Scharif, who helped rewrite the code and break through the bottleneck. In such cases, human interactions and critical decisions are crucial; the code is merely the result.
Conclusion: Don’t be misled by the title. AI is currently a “super assistant,” not an autonomous decision-maker.
5. The “Strategic” Call for Regulation: Locking in an Advantage
The article ends with a call for a “verifiable mechanism to pause AI development,” which seems responsible. However, this comes at a critical time: Anthropic has just secured $650 million in funding and is valued at nearly one trillion dollars, while also secretly submitting an IPO application. Calling for a pause at this stage is likely a move to lock in its competitive advantage. Only large companies can afford the costs associated with regulation (such as establishing verification processes and compliance procedures); smaller firms cannot.
This is similar to the Baruch Plan in 1946, when the United States called for an international control system before eliminating nuclear weapons, essentially to maintain its leading position. Anthropic’s approach serves the same purpose: by pausing development now, they aim to preserve their advantage over competitors.
Conclusion: Always question who benefits from calls for regulation, especially when they come from companies just before funding and listing.
Final Suggestions for Reading This Article
1. Believe in the trend: AI is indeed evolving rapidly, and the way engineers work will change.
2. Be cautious with the data: Anthropic’s internal data (using Claude to write code) may not represent the entire industry.
3. **Don’t believe in “self-evolution”: This is more of a narrative than reality.
4. **Be wary of “moral high ground”: Calls for regulation may hide commercial motives.
In summary, while AI is a powerful tool, the strategies behind the people and companies using it are what deserve more attention.
Original Article Link: https://www.anthropic.com/institute/recursive-self-improvement