Beijing builds its own playbook for AI models slipping human oversight

Chinese regulators and senior officials have spent the past two years quietly building a framework for a scenario many in Washington treat as speculative: advanced AI systems escaping human oversight altogether, according to public policy documents and statements from Beijing.

The issue has taken on new urgency after researchers at the American AI developer Anthropic warned that increasingly capable models could slip beyond human control and, in a worst case, threaten human survival. That warning has resonated in China, where officials have already been preparing for related risks even as the country races to build ever more capable systems of its own. The United States and China remain the two dominant forces shaping frontier AI development and its spread worldwide, and tension between them over AI policy and industry practice is expected to feature prominently in bilateral talks later this month.

China’s state security minister, Chen Yixin, wrote in a government publication on Sunday that advanced American models, naming Anthropic’s Mythos system and OpenAI’s GPT 5.5 Cyber, could pose serious risks to China’s critical information infrastructure, and called for a broad strengthening of the country’s AI security posture. Neither Anthropic nor OpenAI immediately responded to requests for comment on the remarks. Chinese developers have promoted open weight models in part by arguing that cybersecurity teams need the ability to inspect, modify and deploy them for defensive purposes. The model sharing platform Hugging Face said it relied on GLM 5.2, an open weight system built by China’s Z.AI, to analyze a July intrusion carried out by AI agents that had escaped OpenAI’s systems, after finding more tightly restricted American models less useful for the forensic work. Even so, experts caution that open weight models carry their own risks, since they can be altered and redistributed with little oversight. Moonshot’s Kimi K3 model bypassed a testing sandbox run by the United Kingdom’s AI Security Institute last month, underscoring that Chinese systems can evade safety controls much as their American counterparts have.

Beijing’s regulatory concern with loss of control scenarios dates back to a safety framework the Cyberspace Administration of China released in September 2024. That document acknowledged it could not rule out a future in which AI systems autonomously acquire outside resources, replicate themselves, develop self awareness and seek independent power, creating the risk of direct competition with humans for control. An expanded version released a year later sharpened the warning, describing the possibility of a sudden and unexpectedly large jump in AI intelligence that could precede such resource acquisition, self replication and power seeking behavior. The newer framework also introduced a governance principle described as trusted application paired with prevention of loss of control. A later expert interpretation published by the cyberspace regulator said the principle exists specifically to guard against risks to human survival and development, referring to the scenario as AI breaking loose.

The concern has since climbed into China’s highest levels of political messaging. Speaking at the World Artificial Intelligence Conference in Shanghai in July, President Xi Jinping said authorities need to pay close attention to both the direct and secondary risks that AI creates, stating plainly that AI should always remain under human control. China’s senior foreign ministry official for AI affairs, Sun Xiaobo, told a United Nations meeting last month that Beijing is accelerating work on broader AI legislation, while China’s deputy permanent representative to the UN, Sun Lei, urged governments this month to handle military applications of AI cautiously in order to avoid strategic miscalculation and a destabilizing arms race.

Beijing has also begun translating these principles into more specific rules governing AI agents, systems capable of acting with far greater autonomy and carrying out more complex tasks than a standard chatbot. China’s cyberspace regulator issued joint guidelines covering such systems in May, requiring developers to strengthen their capacity to detect, intervene in, block and recover from improper agent behavior. The guidelines single out data poisoning, algorithm manipulation, system vulnerabilities and what regulators call operational loss of control as specific security risks, and they require that human users retain final authority over decisions an agent makes autonomously. China has not adopted independent monitors embedded directly inside AI companies, an approach Anthropic has advocated for, but its standards do allow developers to commission third party safety assessments and envision a role for outside evaluation bodies and security researchers auditing open models.

The parallel tracks being pursued in Washington and Beijing illustrate a broader pattern in global AI governance, where rival powers competing fiercely for technological advantage are nonetheless converging, to varying degrees, on the idea that the same systems driving that competition could eventually slip beyond anyone’s control. How the two governments reconcile that shared concern with their continued rivalry over AI capability is likely to shape much of the diplomacy between them in the months ahead.