Summary of Key Findings
This study reveals a reality that contradicts the narrative of AI replacing human programmers: while AI coding tools can significantly increase the efficiency with which programmers write code, these improvements diminish at each stage of the software development process from writing code to its final release and use by users. As a result, these tools hardly contribute to the creation of more useful software for end-users. The reason is that the relationship between AI and humans in software production is one of strong complementarity—human involvement remains indispensable in downstream tasks such as code review, integration, and distribution. Moreover, newly generated applications are either of poor quality or go unnoticed by users, leading to an abundance of “zombie applications” (applications with little or no user engagement).
1. AI确实 speeds up coding, but only at the writing stage
AI coding tools, such as auto-completion systems and intelligent assistants, show significant benefits in the most basic task of writing code:
- Auto-completion tools can increase the efficiency of programmers by approximately 40%.
- The use of synchronous intelligent assistants (like Claude Code), which allow real-time collaboration with programmers, can double this efficiency.
- Asynchronous intelligent assistants (such as GitHub Agents), which can complete tasks autonomously, further boost efficiency by a factor of 1.8.
The benefits are particularly noticeable for less active programmers, effectively giving them an advantage similar to using a “power boost.” For example, synchronous assistants increased the number of lines of code by 741% (almost eight times), while auto-completion increased it by 228%.
2. Efficiency decreases as software development progresses
Software production is a sequential process: writing code → combining code into files → submitting code for review → pulling requests for integration (PRs) → forming projects → releasing versions. The efficiency gains brought by AI are significantly reduced at each stage:
- Synchronous intelligent assistants increased the number of lines of code by 741%, but the number of PRs only increased by 65%, and this increase was further reduced to 20% in the release phase.
- Auto-completion tools had an even more dramatic impact: while they increased the number of lines of code by 228%, their contribution to the submission process was only 36%, and at the release stage, it was just 10%.
It’s like using a machine to quickly chop food ten times faster, but you still have to mix, cook, and serve it manually; the total cooking time does not decrease by ten times due to the human bottleneck in the downstream steps.
3. Why is efficiency reduced? AI and humans are a “golden partnership” that cannot replace each other
An economic model used in the study calculated that the substitutability of AI-generated code for human work is only 0.25 (the lower the substitutability, the stronger the complementarity). This indicates that AI and humans are like chopsticks and hands—each is essential and cannot replace the other:
- AI can write code quickly, but humans are needed to review it for bugs and ensure it meets requirements.
- Humans must evaluate the logic of the code when integrating it into projects.
- After release, humans are responsible for promoting the application to make it accessible to users.
Even if AI automates the entire coding process, without human involvement in the downstream steps, the overall output will not improve.
4. More applications are created, but users are not interested
AI has enabled programmers to develop more new applications, with noticeable increases in the number of apps on platforms like the Apple App Store and Chrome Web Store, as well as a moderate growth on Google Play. However, these new applications go virtually unnoticed by users:
- The total usage of newly released applications in all four app stores did not increase over three months.
- Most of these new applications are “zombie applications” with minimal user engagement.
There may be two reasons for this: either the quality of AI-generated applications is poor (e.g., they have limited functionality or many bugs), or the distribution process is inadequate.
5. Future directions for improvement
To truly leverage the efficiency of AI, two issues need to be addressed:
- Enhance AI capabilities: Make AI capable of not only writing code but also automatically reviewing it, integrating it, and even assisting with promotion and distribution (e.g., generating application descriptions and optimizing keywords for better discoverability).
- Wider adoption of tools: Ensure that more companies integrate AI tools throughout the entire software development process, not just the coding phase.
For now, humans remain the “chief directors” of software production, and AI serves as a valuable assistant but cannot completely take over.
In one sentence
AI coding tools are a useful aid, but the idea of them replacing programmers or creating a trillion-dollar market is still far from reality. Human involvement in the downstream stages of software development remains an irreplaceable bottleneck, and whether users will accept these tools ultimately determines their success.