Summary of Key Points
While AI has increased the speed of code writing by 10 times, the overall product delivery has only improved by 18%—this is because the development process is not solely about “writing code.” Coding accounts for only 20% of the entire production and development cycle; the remaining 80% is spent on clarifying requirements, reviewing, testing, and waiting. Once AI breaks through the bottleneck in coding, other issues within the process become apparent: vague requirements, dispersed context, chaotic processes, and inadequate management. To truly leverage the efficiency of AI, companies need to restructure their production and development systems. This includes standardizing requirement documents, managing the context effectively, building an AI-operated system, and clearly defining the responsibilities of human participants.
Detailed Analysis
1. The Illusion of Efficiency Boosted by AI: Why hasn’t Delivery Speeded Up?
You might think that AI can make the entire development process much faster, but in reality, only the coding phase has accelerated, while other steps remain slow. For example, Baidu’s team has seen a 10-fold increase in coding speed using AI, yet their two-week iteration cycle hasn’t shortened. This is because coding accounts for only 20% of the time; the other 80% is spent on unclear requirements, delayed testing, and lengthy reviews.
To illustrate this with simple math: Suppose the total delivery time was originally 100 hours, with coding taking 20 hours. If AI reduces coding time from 20 to 2 hours (a 10-fold improvement), the total time still becomes 82 hours, an increase of only 18%. The bottleneck in the process is not in coding but in the subsequent steps.
As Dropbox puts it, AI has merely “shifted the burden of delivery” further downstream—resulting in longer review queues, more congested testing processes, and more challenging operations. Problems that were previously hidden within the process are now brought to light by AI.
2. The Magnifier of Vague Requirements: AI Can Amplify Errors
In the past, product managers could be somewhat vague when writing requirement documents (PRDs), as developers would ask questions during face-to-face meetings; even if the backend interface documentation was poor, front-end developers could make do with it. However, with AI, things are different. If you ask for a membership system, AI will generate a preliminary version immediately, but if details like refund rules, concurrent logins, or coupon discounts are not specified, AI will make assumptions, leading to incorrect implementations.
Previously, vague requirements might waste half a day from one engineer’s time; now, AI can cause 20 agents to produce 20 flawed versions simultaneously, significantly increasing the cost of rework. Therefore, companies are adopting Structured Requirement Documents (SDDs), which break down requirements into six parts: goals, scope, constraints, decisions, tasks, and acceptance criteria, clearly defining what constitutes a successful outcome (e.g., “Can users retry after payment fails? How should the system handle insufficient permissions?”). SDDs are not new, but AI has forced companies to use them, as they help solidify vague requirements and hold upstream stakeholders accountable.
3. Old Issues Revealed by AI: The Need to Pay Off “Contextual Debts”
AI requires a complete context to function effectively—e.g., all code must be in one repository, the environment must be reproducible, and business processes must be documented. Many companies have historical issues such as scattered code across multiple repositories, inconsistent development and production environments, and business rules stored only in the memories of senior employees. Shopify’s example is illustrative: In 2024, they merged their dispersed code into a single repository (Monorepo) and unified their development, testing, and production environments using Nix. The reason? Without a consistent context, AI couldn’t work effectively with the fragmented code. They concluded that “the debt we incurred to make the system understandable for AI actually comes from previous shortcomings in human management.”
4. The AI-Driven Development Operating System: The Command Center for Managing Agents
It’s manageable for one engineer to use 3–5 agents simultaneously, but when the number exceeds 10, things get chaotic—which tasks are stuck? Which need approval? Which tests have failed?
An “AI-driven development operating system” is needed to manage this. OpenAI’s Symphony transformed the project management tool Linear into an agent-controlled console: users submit tasks, and the system automatically assigns agents to execute them, runs tests, and handles review comments. As a result, the number of merged PRs increased by 500% in some teams. Shopify’s River Agent is even more advanced; you can use it via Slack to view code, run tests, and submit PRs, with it helping to merge 3,536 PRs within 30 days.
Such systems serve three main purposes: they act as information channels (storing requirements, code, and rules), workflow containers (connecting tools, testing, and deployment), and control systems (managing permissions, approvals, and logs). They automate tasks that were previously done manually—e.g., rejecting PRs without acceptance criteria or automatically returning failed tests to agents.
5. The Shift from Execution to Judgment: The Value of Human Skills
Research by Anthropic highlights that AI handles execution, while humans are responsible for planning and making critical decisions. For example, business-savvy engineers can use AI to perform extensive tasks based on their expertise; however, decisions regarding release permissions, architectural choices, and high-risk operations still require human intervention.
OpenAI’s approach is to let AI automate low-risk tasks in a sandbox environment, while high-risk actions (like changing production data) must be approved by humans, with all operations being recorded. This is because AI, despite its speed, doesn’t make mistakes, but the consequences of its actions can be severe if not managed properly.
The current trend is that code creation has become cheaper, but the need for judgment, clear boundaries, and accountability remains unchanged. While AI can generate multiple solutions, humans are still responsible for selecting the best one, deciding whether to release it, and addressing any issues that arise.
What Can Ordinary Teams Do?
There’s no need to reinvent the entire system from scratch. Start with a small task (e.g., writing a login interface) and clearly define the requirements and acceptance criteria. Use AI to process it, and then identify and address issues such as outdated documents, mismatched environments, or insufficient testing. Gradually expand this approach.
To measure effectiveness, don’t focus on the amount of code written or the number of agents used; instead, consider how quickly tasks are delivered, how often rework is required, and how many defects occur. These are the true indicators of efficiency improvement.
AI is not a magic solution; it merely exposes problems that were previously hidden within development processes. Only by addressing these issues can the entire production line truly become more efficient.
(End of Article)
Note: All examples in this article are based on practices from companies like OpenAI, Shopify, and Baidu, reinterpreted in plain language without using technical jargon.