Summary in Plain Language
Over the past two years, the entire industry has been focused on AI security, with everyone struggling with the question of “which model to choose with confidence”: whether to use proprietary or open-source models, or whether to develop models in-house or modify existing ones. They conducted numerous tests and checks to ensure the models were “trustworthy” before entrusting their core operations to them. However, Goldman Sachs has now adopted established security principles from traditional cybersecurity, such as “zero trust” and “defense in depth,” and proposed a counterintuitive approach: don’t assume any model is 100% reliable. Even if it’s an open-source model with an unknown origin, as long as a robust set of independent security layers is in place, it can be safely used in highly sensitive environments like banks. This shift represents a fundamental change from focusing on selecting reliable models to building reliable systems. It not only allows companies to fully leverage AI but also reduces the risk of being tied to a single model provider, marking a significant milestone as AI moves from a novelty to a critical infrastructure component.
---
Detailed Explanation
1. The traditional approach to AI security was like granting full access after a single verification
Previously, companies approached AI usage similarly to hiring new employees: background checks, written tests, security assessments, and supplier qualification reviews were conducted, and only after confirmation would access be granted. Over time, companies became less cautious and gradually granted more permissions, starting with accessing general data, then moving on to sensitive customer information, and eventually allowing AI to interact with payment systems and modify production configurations, completely removing manual review processes. This approach relied on the assumption that the model would never fail.
2. The core of AI-based zero trust: I don’t trust anything the model says; everything must be verified individually
Many mistakenly think zero trust means treating AI as a potential threat. In reality, it’s a mature security principle borrowed from traditional cybersecurity. Traditionally, internal network users were assumed to be trustworthy, but after cyberattacks, zero trust was implemented to ensure that even authorized employees had to re-authenticate for every access to core systems. This is even more necessary with AI, as it can behave normally even after being infected or manipulated. The requirement is simple: even if a model costs millions and has passed all security tests, it must still be verified for each specific task—how much money is being transferred, to whom, and whether the operation fits the business context?
3. Security responsibility is shifted away from the model to the system
Goldman Sachs’ “defense in depth” approach ensures that models don’t have ultimate control. They are merely tools that submit requests. To access core data or perform actions, they must go through additional security checks. If a model is compromised, the surrounding security layers can still protect the system. This approach prevents companies from being locked into a single model provider. For example, you can switch from a proprietary model to an open-source one without having to rebuild the entire security infrastructure.
4. The final piece of the puzzle: an independent authority that doesn’t favor any party
The current system still lacks an independent oversight mechanism. Many AI incidents occur despite all compliance checks. A future AI security architecture will include a completely independent “final judge” that evaluates the operation’s legitimacy, regardless of the model or user. This judge will block suspicious transactions, verify large transfers, and prevent actions that deviate from business logic.
5. “Not trusting the model” doesn’t mean discrediting AI; it means treating it as a critical component
This approach doesn’t suggest that AI is unreliable. It’s like equipping a plane with advanced safety systems or requiring double verification for large transactions. A mature system ensures that even if a component fails, the overall system remains functional. Similarly, AI should be treated as a critical component that must meet strict security standards, regardless of its reliability.
In summary, Goldman Sachs’ approach transforms AI security from a focus on models to a focus on building robust systems that can handle core business operations, freeing companies from dependency on specific models and enhancing their resilience to cyber threats.