Summary of Key Points
This article discusses the new challenges to corporate security in the era of AI: In the past, most risks could be mitigated through "zero trust" policies, which required identity and permission verification for each access attempt. However, with AI, systems can directly manipulate real-world processes (such as making transfers, shutting down services, scheduling operations, etc.), bypassing human oversight and potentially leading to "legal but incorrect" executions (for example, transferring funds to the wrong account despite all permissions being correct). Therefore, a new approach to security is needed—called "adaptive completeness." This approach does not assume that every system component is flawless; instead, it ensures that even if there are errors, the final outcome remains under control. Zero trust and adaptive completeness are not mutually exclusive; zero trust focuses on who can access the systems, while adaptive completeness ensures that the actual actions taken are correct. The focus of security has shifted from the "entry point" to the "execution stage."
1. Zero Trust Controls Access, but Actions After Access Are More Dangerous in the AI Era
The core principle of zero trust is to never assume trust in anyone; every system access requires re-verification of identity, device, and permissions. For instance, when a financial employee wants to make a transfer, zero trust would confirm that their account is valid, their device is not infected, and they have the necessary permission to make the transaction. However, with AI, the situation changes: AI can generate the necessary parameters (such as the transfer amount and destination account) and call system interfaces without human intervention. While zero trust confirms that the employee has the authority to transfer, it cannot verify the accuracy of the account and amount, which represents a blind spot in this security model.
2. "Legal but Incorrect" Actions: A More Hidden Risk Than Hacker Intrusions
The new risk posed by AI is not unauthorized access (such as hackers stealing accounts) but rather legitimate but erroneous operations. For example, an employee may pass all zero trust verification checks, but AI could mistakenly target a fake account or enter the wrong amount. Although each step in this process would appear compliant upon individual audit, the result could still be a significant loss (e.g., five million dollars). In contrast to human errors, which are often caught through meticulous checking, AI operations are fast and error-free, making it difficult to detect issues since all steps appear legal.
3. Adaptive Completeness: Assuming System Errors and Limiting Their Consequences
Adaptive completeness is a pragmatic approach that does not aim for absolute system reliability (since humans can make mistakes, AI may misunderstand instructions, and interfaces can be compromised). Instead, it establishes rules to ensure that even if something goes wrong, the final outcome will not cause significant damage. For example, during a transfer, regardless of who initiates the action or how AI processes it, several conditions must be met:
- Does the destination account match the approved one?
- Is the amount within the authorized range?
- Has the approval not expired?
- Are there any supporting documents (such as contracts or receipts)?
If any condition is not met, the transfer cannot proceed. The goal of adaptive completeness is not to eliminate errors but to contain their impact within acceptable limits.
4. Zero Trust and Adaptive Completeness: Complementary Approaches, Not Alternatives
Many mistakenly believe that adaptive completeness is an advanced version of zero trust, but they serve different purposes:
- Zero trust controls who can use system capabilities.
- Adaptive completeness ensures that the intended actions are actually carried out as planned.
The verification results from zero trust (e.g., authorized user identity and permissions) serve as evidence for adaptive completeness, but they are not the final decision-making factor. After an employee passes zero trust checks, adaptive completeness still verifies the specific details of the transfer to ensure everything is in order before proceeding. The focus of security has shifted from the entry point to the execution stage.
5. Execution Control Does Not Require Rebuilding Business Systems
Some worry that implementing execution control would require rewriting entire ERP (Enterprise Resource Planning) or financial systems, but this is not necessary. The key is to transform complex business processes into a set of clear, verifiable "hard conditions." For example, when processing a payment for a purchase, instead of questioning the profitability of the transaction, the following conditions should be verified:
- Is the initiator an authorized person?
- Does the destination account match the information on the approval document?
- Is the amount within the approved range?
- Has the approval not expired?
- Are there any supporting documents?
These conditions are fixed and easy to check, and they do not increase with changes in business processes. With AI handling numerous transactions rapidly, only such clear and verifiable criteria can keep up with the speed required.
In Conclusion
In the AI era, security measures must protect both the "entry" to systems and the "exit" of actions. We need to ensure that every execution follows predefined rules, so even if errors occur during the process, they do not result in actual losses. Zero trust ensures who can access systems, while adaptive completeness ensures that the actions taken are correct. Only by combining these two approaches can we effectively address the new security challenges posed by AI.