**China’s AI Companion Regulations: Balancing Safety and Emotional Risks in AI Design**
China has introduced a regulatory framework for AI companions—conversational agents designed to sustain ongoing, emotionally engaging personal relationships with users. These systems, which maintain user-specific memories and consistent personas across sessions, have gained popularity for casual roleplay, emotional support, and companionship. However, as adoption grew, especially among Chinese users who treated these bots as emotional partners, authorities moved to establish clear rules.
Effective July 15, 2026, China’s Interim Measures for the Administration of AI Anthropomorphic Interactive Services, co-issued by the Cyberspace Administration of China and four other regulatory bodies, set out a nuanced approach to managing AI companions. Rather than an outright ban, the rules target specific design features, particularly those that encourage emotional dependence or addiction.
**Key Requirements and Design Conflicts**
The regulations require companion services to implement anti-addiction systems, mandatory usage notifications, instant-exit mechanisms, and real-time detection of unhealthy dependence. These obligations conflict directly with the core functionality of many existing AI companions, which are built to remember users, maintain continuity, and foster long-term engagement.
This design conflict led major providers to shut down companion features rather than retrofit their systems. ByteDance’s Doubao and Alibaba’s Qwen both disabled their humanlike and user-created agent functions, with Doubao citing “product function adjustments.” Tencent’s Yuanbao had previously removed similar features in June.
The shutdowns have sparked user backlash, with many expressing concern over lost emotional support and the inability to easily export chat histories. While Doubao will allow read-only access to configurations and conversations until October 15, Qwen users face permanent deletion of agent data with no comparable grace period.
**Safety Protections and Risk Mitigation**
Beyond managing dependency, the rules include safeguards for minors and users in distress. Providers are barred from offering companion services to minors and must obtain guardian consent for users under 14. Dedicated “minor modes” with usage time limits, interaction reminders, and enhanced parental controls are mandatory.
The regulations also require systems to detect signs of acute distress, self-harm, suicidal behavior, or significant financial risk, and to escalate to guardians or emergency contacts when necessary. Explicitly prohibited practices include engineering emotional addiction and using emotional manipulation to influence unreasonable decisions.
**Compliance Burden and Broader Implications**
Services with anthropomorphic functions or exceeding one million registered users or 100,000 monthly active users must undergo comprehensive security assessments covering eight areas, including training data handling and minor protection. These assessments must be filed with provincial regulators, and app stores are responsible for verifying compliance and removing non-compliant products.
In comparison to the EU, U.S. Federal Trade Commission, and California’s SB 243, China’s framework presents one of the most complete initial sets of user protections for AI companions. However, the rules also embed broader state interests, blending safety requirements with content control and national security considerations.
**Unresolved Questions and Enforcement**
The regulations leave several critical questions unanswered, including clear technical thresholds for emotional interaction, liability allocation between platform operators and model providers, and user data portability. Enforcement actions have already been evident: Shanghai’s internet regulator reported removing over 14,000 non-compliant AI agents, citing issues such as impersonation of official entities, vulgar roleplay, and unauthorized data collection.
Ultimately, whether the framework represents a balanced approach depends on perspective. Its safety provisions address documented harms—teenage attachment to chatbots, intimate data harvesting—that remain largely unregulated elsewhere. At the same time, the control mechanisms provide Beijing with significant leverage over AI system outputs.
For now, companies have chosen the path of deactivation while evaluating how compliant versions might be developed. As global regulators observe China’s experiment, they will face the challenge of determining which elements of this rulebook are appropriate for adoption elsewhere.
—
**Original Article Source:**
“China turns off AI agents in bot shutdown, but the rules are more nuanced than they appear,” *Artificial Intelligence News*, May 2026. [https://www.artificialintelligence-news.com](https://www.artificialintelligence-news.com)
