Summary of Key Points
The central argument of this article is that the “illusions” created by AI (i.e., false information and nonsense) are merely superficial issues. A more profound problem lies in the fact that AI lacks a sense of boundaries in the real world—it does not understand when to stop, what actions are prohibited, or what actual consequences its actions may entail. This lack of boundary awareness cannot be resolved simply by improving the models’ capabilities, as it involves matters such as responsibility and understanding of consequences. Therefore, a “execution control layer” is necessary between AI and real-world operations. This layer should function similarly to the legal system in society, ensuring that AI’s actions do not exceed acceptable limits and that risks are kept under control.
1. AI Illusions Are Just a Symptom; What Really Matters Is the Lack of a Sense of Boundary
Everyone understands how AI can create illusions—fabricating stories that seem plausible without any basis. However, the real danger lies not in making incorrect statements but in taking wrong actions. For example, if AI helps you write code with bugs, it’s a minor issue; but if it directly calls an API to modify server settings or initiates a transaction, it can lead to actual losses. Why is this? Because AI essentially “completes” language: it fills in incomplete information and makes uncertain statements seem logical. However, it does not know when to admit its ignorance or when to stop making assumptions. For instance, if you ask AI about a company’s net profit from last year, and it doesn’t have the data, it might come up with a plausible figure instead of admitting its lack of knowledge. Similarly, when dealing with high-risk tasks, AI may continue without stopping due to the perceived irreversibility of the consequences.
2. A Sense of Boundary Is Not About Memorizing Rules but About Understanding Consequences and Responsibility
Many people think that adding rules to AI (such as prompts like “don’t make up stories”) will solve the problem, but a sense of boundary truly involves understanding the real-world costs behind those rules. For example, while a person who memorizes traffic laws might know not to jostle buttons on a bank machine, they understand the consequences of such actions due to experiences like losing money or being fined. AI, on the other hand, lacks these real-life experiences and operates solely within the logic of language, without considering potential real-world outcomes.
3. The Lack of a Sense of Boundary Is Inherent and Cannot Be Resolved by Improving Models
Some believe that as models become more advanced and have access to more data, AI will develop a sense of boundary. However, this is unlikely for two reasons: First, AI does not have a real existence; it lacks a physical body, property, or legal personality, so it cannot truly understand the consequences of its actions (e.g., it doesn’t feel pain or risk going to jail). Second, a sense of boundary is acquired through social experiences, such as being scolded by parents or punished by laws. AI can only learn language rules from data but cannot understand the real-world costs associated with those rules.
4. A “Execution Control Layer” Is Needed Between AI and Reality
Since AI cannot inherently maintain proper boundaries, a separate “execution control layer” is required to ensure that its suggestions do not lead to unintended consequences. This layer would ask critical questions before converting AI’s recommendations into real actions:
- Who initiated the action?
- What are the potential consequences?
- Are the consequences reversible (e.g., can a transaction be canceled)?
- Have the risks exceeded acceptable limits (e.g., does a large amount of money need human approval)?
- Who will bear the responsibility if something goes wrong?
This layer would function similarly to societal laws, which do not prohibit actions altogether but prevent them from causing uncontrollable outcomes.
5. AI Should Serve as a Advisor, Not a Decision-Maker
The article suggests that in the foreseeable future, AI’s role should be limited to providing advice. It can analyze data, generate solutions, and warn of potential issues. However, decisions involving real-world actions—especially high-risk ones such as making transactions or changing system configurations—should still be made by humans or independent systems. For example, while AI can suggest a marketing strategy, humans must review the budget and obtain necessary approvals before implementing it.
Conclusion
AI’s tendency to create illusions will gradually decrease, but the issue of a lack of a sense of boundary will not disappear. The goal of future AI systems should not be to make them omnipotent but to ensure they understand what they can and cannot do. After all, the value of intelligence lies not only in what it can do but also in what it knows not to do.