Summary of Key Points
This article explores the profound impact of generative AI on social authority structures through a medical experiment. It demonstrates how AI-generated “targeted advice”—guidance with clear biases—chips away at the traditional monopoly held by experts (such as doctors) in interpreting the world, making judgments, and assuming responsibility. Specifically, AI allows ordinary people to access professional-level explanations at virtually zero cost. However, the direction of these recommendations (e.g., advising caution when taking medication or recommending tests) is not neutral; it is determined by the underlying responsibilities of the models, commercial interests, and other institutional arrangements. Meanwhile, AI does not bear the ultimate decision-making or legal responsibility, which remains with traditional experts (for example, doctors who sign off on treatments). This separation of authority—where interpretation rights are given to users, production is centralized in a few models, decision-making is retained by professionals, and responsibility falls on those who sign off—is reshaping social trust and the distribution of power, raising a range of constitutional issues that need to be addressed (such as matching responsibilities, ensuring transparency of directions, and providing diverse interpretations).
Detailed Analysis
1. Why are AI recommendations not neutral? — The inevitability of bias
You might think that AI provides objective advice, but once it moves from stating facts to offering guidance on actions, it inevitably takes a stance. For instance, when asked whether to take aspirin, AI will either suggest consulting a doctor (conservative) or suggest you can make your own decision (radical); it cannot remain neutral. The direction of the advice reflects the underlying factors, such as the training data, safety guidelines, and the developers’ fear of liability. For example, medical AI recommendations for tests are often risk-free (any issues are attributed to the doctor’s judgment), while medication recommendations may lead to legal disputes, so the model tends to suggest more tests and less casual use of drugs. What you think you’re getting is actually filtered through a set of responsibility rules.
2. How do traditional experts maintain their authority? — The binding force of authority
Why did people trust doctors in the past? Not because they verified the doctors’ knowledge (for example, whether a surgery was necessary; you couldn’t prove otherwise afterward), but because doctors held multiple “certificates of trust”—they had licenses, hospital backing, and were accountable in case something went wrong. These factors combined to form what is known as an “authority bundle,” which includes three powers: defining the problem (what disease you have), explaining the cause and effect (why you’re sick), and prescribing actions (what medication to take).
This authority bundle solved a significant issue: healthcare is an intangible service—you can’t know for sure if it was effective (for example, after surgery, you might recover, but you don’t know if it was necessary). The experts’ income came from this information asymmetry that couldn’t be easily verified.
3. With the advent of AI, the doctor’s authority bundle is broken — The shift in scarcity
AI has reduced the cost of providing explanations to nearly zero: you can ask for advice at any time without paying. However, the scarcity has not disappeared; what’s now scarce is the ability to make informed decisions and bear the consequences of those decisions.
The changes are as follows:
- Interpretation rights: Users can obtain free explanations, but these are produced by a few models (centralized). Millions of people may receive similar advice from the same source.
- Decision-making power: Still in the hands of experts (e.g., doctors prescribing medication), but their choices are influenced by AI (for example, patients bring AI-generated recommendations, and doctors must take them into account).
- Responsibility: It falls on the final signatories (e.g., if AI advises a test and the doctor signs off, any issues are attributed to the doctor). As a result, the value of experts shifts from simply knowing the answers to being willing to take responsibility. Tasks that require explanation but not signing (e.g., counseling, basic patient inquiries) will become cheaper, while those that require signatures (e.g., prescriptions, legal appearances) will become more expensive.
4. Real changes observed in medical experiments
The article describes a randomized experiment where half of the patients used an AI assistant before their visit, and the other half did not. The results were clear:
- AI’s influence on behavior: Patients who used AI received fewer prescriptions and more tests the next day.
- Varying impact on doctors: Doctors who were more open to patient input were significantly more influenced by AI, indicating that AI indeed affects their decisions.
- Erosion of trust: Patients who used AI showed less compliance and lower satisfaction with their doctors because they had a “free second opinion” and no longer trusted the doctor completely.
5. This is not just a medical issue — Constitutional challenges at the societal level
AI’s targeted advice is not limited to healthcare; it affects the distribution of authority throughout society. The article raises three critical issues that need to be addressed:
- Responsibility matching: Since AI significantly influences people’s actions, who should bear the responsibility? The doctors who sign off cannot be held solely accountable. For example, AI could require human oversight in high-risk areas, but the balance of responsibilities must be carefully considered.
- Transparency of directions: Users have a right to know whose interests the AI’s recommendations serve—whether it’s for the user’s health or to exempt developers from liability. The algorithm should not be kept secret; instead, it should be as transparent as food labels, indicating that conservative recommendations are due to legal risks.
- Preventing a monopoly on interpretation: If everyone uses the same AI model, its biases will become the societal norm, which is dangerous. Multiple models with different perspectives are needed to prevent a single source of authority from dominating.
Finally, the article calls for “directional audits”—regular checks on the bias of AI models (e.g., whether they always recommend buying insurance) to monitor their impact.
In conclusion
AI has brought unprecedented convenience in providing explanations, but it has also led to a significant separation of roles: those who interpret the world, make decisions, and assume responsibility. Our era is not one without authority; rather, the real challenge is that we no longer know which models hold that authority. This is what we need to be most vigilant about.