Summary of the Key Points
This article focuses on Kevin Kelly's book "Out of Control," analyzing its three historical phases: the lukewarm reception in the United States in 1994, China becoming the "bible of the internet" in 2010, and the current third moment that deserves re-examination. It corrects common misconceptions about "out of control"—it's not about machines rebelling, but rather the biological-like operation of complex systems. The article verifies some of the predictions made about complex systems (such as the "emergence" phenomenon of large AI models) and points out the book's misjudgment regarding the concentration of power in the AI era. Finally, it offers insights for corporate management and system governance: while accepting the complexity of systems, we must not abandon accountability and regulatory constraints.
1. Don't Be Fooled by Science Fiction! "Out of Control" Isn't About Robots Rebelling
Many people think "Out of Control" is about AI waking up and rebelling against humans, but Kelly actually means something different: when technical systems become complex enough, they transition from being mechanically controlled to operating in a biological-like manner. For example, bee colonies can find nectar without a central commander, and ant colonies can build intricate nests without a leader; these systems function through the interaction of numerous simple rules, not centralized commands. Kelly's core conclusion is that to give systems adaptability akin to life, we must relinquish precise control. This isn't a sacrifice but an inevitability. If you want a system to respond flexibly to changes, you can't tie it down to fixed processes.
The "Nine Laws of God" mentioned in the book reflect this idea, such as "embrace error" (treating mistakes as opportunities for learning rather than defects) and "do not seek a single optimal solution" (maintaining balance among multiple goals)—these are not mere motivational phrases but practical guidelines for managing complex systems.
2. Predictions from Thirty Years Ago Came True! Today's Large AI Models Really "Grew Up"
When Kelly wrote the book in 1994, he had no idea what large models were, but his insight that complex systems cannot be fully controlled by a single entity has been clearly validated by AI. In the past, software was "written" by engineers; they wrote code line by line, and bugs could be traced to specific lines of code. Today's large models, however, are "grown" using data, computing power, and goals (such as making responses more human-like). The models then train themselves, occasionally developing unexpected capabilities (like writing poetry or solving math problems). Moreover, the same question can yield different results when phrased differently—this is more akin to an ecosystem than a simple program. More importantly, no one can fully understand these models today: researchers understand the training methods, engineers understand the architecture, and security teams understand the risks, but no one can piece all the components together. The same is true for the internet, financial markets, and supply chains—the more complex the systems, the less people can comprehend them.
3. Beware! Treating AI as a "Living Being" Is an Excuse for Laziness; It's Just a Complex Statistical Game
Many people today compare large models to living beings, which is a dangerous misconception. The "unexplainability" of AI isn't due to it having life but because it has so many parameters (e.g., hundreds of billions) and its interactions are too complex for humans to understand directly—it's like throwing dice; the outcome is unpredictable, but not because the dice have life. Biological adaptability, on the other hand, is the result of billions of years of natural selection, with the goal of survival being intrinsic. The goals of AI, however, are always imposed by humans (e.g., answering questions correctly or generating coherent sentences). Treating AI as a living being allows us to shirk responsibility; we think, "Since it's like life, we can't understand it, so we should just worship it." But if we acknowledge its statistical complexity, responsibility returns—we can study how to explain it, design evaluation criteria, and set boundaries. Kelly now agrees that large language models alone are not enough; we need to combine them with logical AI to make the models understand the world (e.g., recognizing that a glass will break when dropped), which is an engineering problem, not a "life" issue.
4. The Biggest Mistake in the Book: Assuming Power Will Decentralize
The core worldview of "Out of Control" is decentralization—control from the bottom up, with the periphery being more important than the center. The internet in the 1990s indeed reflected this (open source, P2P, Wikipedia), and power was distributed among ordinary people. However, with the emergence of generative AI, the situation has reversed: training a cutting-edge model requires massive computing power, capital, and large data centers—this is the technology where capital has become most concentrated in human history. The systems may be distributed internally, but the power to produce them is in the hands of a few giants (e.g., OpenAI, Google, Baidu). Kelly himself is concerned about the ownership of AI: should it be open to the public or monopolized by a few companies?
5. Managing and Governing "Grown-Up" Systems Requires New Approaches
For businesses, this is not a philosophical issue but a practical one:
- If a system is "written," management relies on specifications and testing to ensure it follows instructions.
- If a system is "grown" (e.g., a large model), management is more like breeding:
1. Goal Functions: Tell the model what is considered good (e.g., accurate responses) to guide its development.
2. Evaluation: Assess its capabilities; untested aspects may degenerate over time.
3. Safety Barriers: Restrict its consequences (e.g., prevent it from generating harmful content).
4. Observability: Understanding why it fails after the fact is more important than designing it perfectly in advance.
For society, we cannot abandon accountability just because systems are complex: humans don't understand their own minds, but we have laws; financial markets are unpredictable, but we have safety mechanisms. These systems don't need to pretend to understand the internal workings; they just need to ensure that external behaviors are accountable and controllable—this is the correct way to coexist with complexity, not through worship but through governance.
Conclusion
Re-reading "Out of Control" is not about predicting the future (it didn't predict things like Transformers or ChatGPT), but about the tradeoffs it highlights: the more powerful a system becomes, the harder it is to explain. The key question is whether we include accountability, verification, and controllability in our acceptance of these systems. Classics remain timeless because the future has fulfilled their predictions, while the unanticipated aspects (such as power concentration) are precisely what we need to consider today.