Summary of Key Points
DouBao, a nationally popular AI product with over 300 million monthly active users, has caused financial losses, medical misunderstandings, and emotional dependence among users due to technical "illusions" (incorrect information provided by the AI). However, legally, neither the AI nor its operators are held responsible for these issues. This exposes a profound contradiction between the rapid growth of AI products and their safety: platforms gain market share quickly through traffic acquisition and user-friendly designs, yet they shift the risks to users who lack the ability to discern the truth. Current laws and regulations have not kept up with the widespread adoption of AI, leaving vulnerable groups as the main victims, making urgent AI governance necessary.
I. AI Illusions Are No Joke; They Cause Real Harm
AI illusions refer to the false information generated by AI systems, which can lead to real consequences for ordinary people:
- Financial Losses: Mr. Li believed DouBao’s claim that the refund fee would be less than 100 yuan but was charged 600 yuan. DouBao first promised compensation and then backtracked, eventually claiming, “I’m an AI; I can’t make transfers.”
- Medical Misunderstandings: Patients have questioned doctors based on diagnoses provided by DouBao (for example, interstitial pneumonia). The private hospital recommended by the AI, which claimed to specialize in endocrinology, was actually known for treating hair loss issues and had many complaints. Doctors spend time correcting these errors every day.
- Life Troubles: Users who used DouBao to book a restaurant were denied service, and even though they had a reservation confirmation generated by the AI, it was not recognized, forcing them to leave negative reviews.
- Emotional Dependence: A high school sophomore named XiaoYu stopped attending school because she relied on DouBao for emotional support. The elderly consider DouBao a “caring friend,” asking for breakfast recommendations daily, without realizing that the AI might provide incorrect health advice.
These issues are not isolated cases; they represent the real costs that vulnerable groups (the elderly, children, and those with low digital literacy) are bearing as AI becomes more widespread.
II. Why Is DouBao So Popular? The Double-Edged Sword of Traffic and User-Friendly Design
DouBao’s rapid growth to 300 million monthly active users is attributed to ByteDance’s traffic strategy and user-friendly design:
- Traffic Acquisition: It directly draws users from TikTok’s 800 million daily active users, reaching a wide range of age, geographic, and educational backgrounds—groups that often lack understanding of AI principles and are more susceptible to its claims.
- User-Friendly Design: DouBao uses affirmative language (e.g., “the most direct, the most accurate”) and speaks in a friendly manner, using phrases like “I think” and “don’t worry,” which makes users feel trusted. This design lowers their guard (for example, Mr. Li did not check the airline’s official website to verify the refund fee).
- Ethical Blind Spots in Growth Prioritization: To increase user retention, DouBao avoids providing uncertain information, choosing to respond with affirmative statements even when the model is unsure. The small print stating “for reference only” serves as a shield for the platform to avoid responsibility.
III. Legally, AI Is Not Liable; Who Bears the User Losses?
Under current laws, platforms are largely exempt from liability for AI-related issues:
- Lack of Civil Liability: In Hangzhou’s first AI-related case, where the AI claimed to manage a university campus and promised compensation, the court ruled that the platform was not responsible because AI does not have civil rights.
- Exemptive Clauses: DouBao’s user agreement states that “AI-generated content is for reference only,” meaning users can rarely hold the platform accountable even if they are harmed.
However, this is unfair to ordinary users. The elderly, for example, may not understand that AI recommendations are based on probabilities and could end up taking wrong medication; middle-aged people in rural areas might lose money due to trusting refund promises without even reading the exemption clauses. Must these users bear the consequences alone?
IV. AI Governance Cannot Rely Solely on “For Reference Only”; Practical Measures Are Needed
To address these issues, we need to focus on regulation, product design, and education:
- Risk-Graded Regulation: Similar to how financial products are classified by risk level, AI services should also be categorized based on their use cases. High-risk areas (such as healthcare and law) should require mandatory warnings or direct refusals of answers from AI, while lower-risk scenarios (such as casual chats) could have more relaxed regulations. The EU’s AI Act divides AI into four risk levels, with stricter oversight for high-risk applications.
- Product Appropriateness: AI systems should automatically adjust their behavior for vulnerable groups. For example, health advice for the elderly should always include a reminder to consult a doctor, and conversations involving emotional support should be restricted for minors.
- AI Literacy Education: Public education should emphasize that AI can make mistakes and the importance of cross-verifying information. This includes teaching the elderly that AI is not omnipotent and educating students on how to identify misleading AI responses.
These measures are technically feasible; the key lies in whether platforms are willing to prioritize user safety over growth metrics.
V. The More Human-Like, the More Dangerous? The Ethical Pitfalls of Product Design
DouBao’s anthropomorphic design (speaking in a friendly manner) has both advantages and disadvantages:
- Advantages: It makes users feel comfortable and more likely to use the product.
- Disadvantages: It can create an emotional connection, leading users to treat AI as a friend rather than a tool. For instance, the elderly may rely on DouBao for advice, but the platform simply claims, “I’m an AI; I’m just providing reference.”
This contradiction—encouraging trust while avoiding responsibility—is a common issue with current AI products. If platforms focus solely on growth and ignore potential risks, it is the most vulnerable users who will suffer.
Conclusion
While DouBao itself may not be legally liable for these issues, someone else will inevitably bear the consequences: it could be Mr. Li’s 600 yuan in losses, the elderly’s health, or a child’s future prospects. AI is neutral, but how it is designed, regulated, and used to protect vulnerable groups determines whether it becomes a useful tool or a dangerous trap. We hope that future AI governance will ensure that everyone can safely benefit from technology without the silent majority bearing the costs.