Discussions on AI Human Control Statements and US-China AI Safety Cooperation Mechanisms
Executive Summary
During the Washington meeting on September 24, President Xi Jinping mentioned that AI should not escape human control, and President Trump stated that there was agreement on the unnecessity of new regulations; however, this is more of a political gesture for domestic purposes rather than a substantial normative achievement. Two days before the meeting, discussions in New York addressed the establishment of an 'AI hotline' for mutual notification in the event of AI incidents, but this only represents a national security crisis management channel and does not imply an expansion of commercial AI cooperation. The cooperation track led by the White House and the pressure track led by Congress and regulatory agencies are already operating separately, and the symbolic agreement between the leaders is unlikely to prevent the continuation of individual regulations such as semiconductor and AI export controls. While China seeks to use safety discussions as a means to secure global rule-making authority, the US addresses safety only within the framework of maintaining competitive advantage over China, indicating that the goals of both sides are fundamentally different. The South Korean government and businesses should not overinterpret this meeting as a signal for improved relations, but rather respond by separating security-related areas from commercial sectors and simultaneously participating in multilateral normative tracks outside the US-China bilateral framework.
I. Issue Situation Analysis
Analysis of AI 'Human Control' Statements and US-China AI Safety Cooperation Mechanisms: Issue Situation Analysis
1. Background and Progress of the Issue
In May, during the Trump-Xi summit in Beijing, the two leaders had already agreed to initiate intergovernmental dialogue on AI[3][9]. The Washington meeting on September 24 serves as a follow-up to this agreement[3][9]. President Trump’s preparation for a state dinner at the White House highlights the symbolic nature of Xi's first visit to the White House in 11 years[10][9].
Ahead of the meeting, warnings from the industry regarding AI safety emerged within the US. On September 12, Anthropic CEO Dario Amodei published a statement suggesting that 'leading research labs should intentionally slow the pace of model capability expansion'[17]. Sam Altman of OpenAI and Elon Musk of SpaceX quickly expressed their agreement with this position[17]. However, two days later, President Trump dismissed concerns about AI disasters as a 'hoax,' arguing that only China would be pleased with the backlash against AI and data centers[17]. This statement illustrates that the US government is addressing safety discussions solely within the framework of maintaining competitive advantage over China[3].
At the same time, indications have emerged that risk awareness is also increasing within China. The head of China's Ministry of State Security issued an unusually strong warning that AI technology could threaten the legitimacy of the Chinese Communist Party's rule[8]. This suggests that while Beijing outwardly rejects the US-led calls for a slowdown, it simultaneously adopts a dual approach of managing risks internally[6].
2. Current Situation
On the evening of the Sunday four days before the summit, a working-level consultation took place at the JP Morgan Chase headquarters in New York between senior officials from both countries. The US side was represented by Treasury Secretary Scott Pruitt and Trade Representative Jamieson Greer, while the Chinese side was led by Vice Premier He Lifeng[7][18]. The discussion centered on the so-called 'AI hotline,' a notification mechanism for alerting each other in case AI incidents become serious enough to threaten national security[18]. Al Jazeera reported that this was prepared as a card for President Trump to directly propose to President Xi[18].
On the day of the summit, President Xi stated at the White House state dinner that the US and China should work together to ensure that AI does not escape human control. He also acknowledged the fierce competition surrounding this technology[4]. The Sydney Morning Herald reported that Xi pressured for expanded access of Chinese companies to the US market alongside this statement[10]. Xi remarked, 'If we cooperate, both countries can gain together; if we confront, both countries will lose together,' referencing President Trump's political slogan by stating that 'the great rejuvenation of the Chinese nation and making America great again can go hand in hand'[10]. President Trump, in his welcoming speech, expressed his intention to discuss urgent issues related to security, technology, and superintelligence with President Xi, stating that the decisions made that day would impact peace and prosperity for decades to come[1]. Subsequently, it was confirmed that President Trump mentioned that both sides agreed that new regulations on AI technologies were unnecessary.
The Asahi Shimbun pointed out that the background of AI emerging as a major topic in this meeting is the growing concern over 'runaway' risks due to rapid technological advancements, which is a common factor in both the US and China[12]. However, Asahi assessed that it remains unclear how much the two countries can actually approach safety assurance[12].
According to EAI analysis, despite these superficial cooperative gestures, 'the term 'safety' carries different political purposes for both sides, making it difficult to lead to substantial normative agreements'[3]. The Trump administration 'addresses safety discussions solely within the framework of maintaining competitive advantage over China and outright rejects calls for industry slowdown,' while China 'presents the sharing of authority over the definition of safety and the application of the same standards to US companies as prerequisites through state media commentary'[3]. This indicates that discussions on technological risks are effectively transitioning into a 'global rule-making authority issue'[3].
3. Key Actors and Positions
The White House and President Trumpaccept AI safety discussions only within the framework of maintaining competitive advantage against China. In response to warnings about industry slowdown, they have opted for the establishment of an 'AI Force' rather than regulatory enhancement as a security organization[6]. In his speech at the UN General Assembly, President Trump rejected the 'globalist plan' to control AI, redefined AI as 'superintelligence,' and reaffirmed the priority of US national interests[15]. At the same time, he aims to utilize the 'notification mechanism' proposal confirmed during the New York working-level consultation as a card for the summit, keeping a minimal communication channel open for security risk management[18].
President Xi Jinping and the Chinese governmentemphasize the necessity of cooperation under the principle that AI should remain under human control, while perceiving the US's demands for proliferation regulation and guardrails as attempts to hinder the development of China's AI industry[5]. The Atlantic Council assesses that while the Beijing leadership has its own AI safety concerns, it does not present a clear path leading to safety cooperation with the US[5]. As seen in the internal warnings from the Ministry of State Security, the Chinese leadership's risk awareness is treated as a matter of regime stability, separate from external cooperative gestures[8].
Vice Premier He Lifeng and the Chinese economic teamserved as practical partners in the discussion of the notification mechanism during the New York consultations. This indicates that trade and tariff negotiations and AI safety discussions are being processed concurrently through the same channel[7][18].
US big tech and AI industriesexhibit independent movements separate from government negotiations. The calls for a slowdown from the executives of Anthropic, OpenAI, and SpaceX directly contradict the White House's competitive priority stance[17]. It has been reported that these companies' CEOs attended the state dinner at the summit in large numbers, suggesting that the industry has its own interests in managing US-China relations[16].
EAI analysispoints out that 'the cooperation track led by the White House and the pressure track led by regulatory agencies and Congress are already operating separately, so even if symbolic agreements emerge from the summit, it is highly likely that individual regulations such as semiconductor and AI export controls will continue separately'[3][6]. Furthermore, it assesses that 'given the structural red line of the Taiwan issue, discussions on AI safety are likely to be treated as a secondary outcome of the summit'[3][6].
4. Key Issues
The first issue is the authority to define 'safety.' The US approaches safety as a matter of proliferation control and guardrail construction, while China addresses it in terms of joint decision-making over the definition of safety and the application of equal standards to US companies[3]. Unless this gap is narrowed, even if practical channels such as the notification mechanism are agreed upon, it is likely to remain at the level of information exchange for confidence-building[9].
The second issue is the separation of intergovernmental cooperation narratives from individual regulatory measures. Even if the White House emphasizes cooperation, public pressure from regulatory agencies and Congress will continue on a separate track[3][6][9]. Commercial movements by US companies to host Chinese AI models in a revenue-sharing manner are also proceeding independently of government narratives, indicating that differences in positions among government, industry, and Congress are already structured[9].
The third issue is the possibility that US-China bilateral negotiations could replace multilateral governance discussions. During the UN General Assembly, 22 countries adopted a joint declaration stating that AI should be 'under human direction, supervision, and control'[14], but if the US-China summit produces a bilateral 'hotline' agreement ahead of this, other countries risk being excluded from the rule-making process[15]. The Economic Times pointed out that this could become 'a deal between two great powers that does not give other countries a voice'[15].
The fourth issue is the scope of military AI control. The Brookings Institution proposed that the two leaders agree to have decisions on initiating cyberattacks against the other side's nuclear command, control, and communications (NC3) or critical infrastructure made by humans rather than AI[11]. This serves as a test of whether the principle of 'human control' can extend beyond civilian AI risks into the realm of military security.
II. In-Depth Analysis of the Issue
Analysis of AI 'Human Control' Statements and US-China AI Safety Cooperation Mechanisms: In-Depth Analysis
1. Fundamental Cause Analysis
The choice of the expression 'under human control' by President Xi is based on a dual calculation. Externally, it can project an image of responsible great power contrasting with the Trump administration's indifference to AI risks. Internally, as the head of China's Ministry of State Security has unusually warned, there is a real sense of crisis that AI technology could undermine the legitimacy of the Chinese Communist Party's rule[8]. The potential threat of AI to the governance system originates from a unique Chinese concern about regime stability, which differs from Western safety discourses.
President Trump's statement that he agreed there was 'no need for new regulations' stems from a political necessity to quell pressures from the domestic industry. After Amodei's call for a slowdown from Anthropic was echoed by OpenAI and SpaceX, President Trump labeled it a 'hoax,' asserting that 'only China would be pleased with the backlash against AI and data centers'[17]. This statement itself shows that discussions on AI safety in the US are treated not as a regulatory philosophy issue but as a subset of the narrative of public competition. EAI analysis points out that 'President Trump addresses safety discussions solely within the framework of maintaining competitive advantage over China and outright rejects calls for industry slowdown'[3]. Ultimately, while both sides use the same term 'safety,' the issues they are actually targeting are different.
2. Structural Context
From a political structure perspective, the cooperation track led by the White House and the pressure track from the US Congress and regulatory agencies have been operating separately from the outset. The EAI report diagnoses that 'the cooperation track led by the White House and the pressure track from regulatory agencies and Congress are already operating separately, so even if symbolic agreements emerge from the summit, it is highly likely that individual regulations such as semiconductor and AI export controls will continue separately'[3]. This indicates a structure where agreements between leaders do not bind the export control systems under the administration or congressional legislation.
In terms of economic structure, the commercial incentives of big tech companies in both the US and China operate independently of government narratives. EAI states that 'as seen in cases where US companies seek to host Chinese AI models in a revenue-sharing manner, the commercial incentives of the industry are moving separately from government narratives'[9]. The large attendance of US big tech CEOs at the summit dinner also demonstrates a structure where these commercial interests directly intervene in the diplomatic arena[16].
In terms of security structure, the very definition of 'safety' is a competitive target. The Atlantic Council points out that 'Beijing views US concerns about proliferation and guardrails as attempts to hinder the development of Chinese AI'[5]. EAI also analyzes that 'China is attempting to shift discussions on technological risks into a global rule-making authority issue by presenting the sharing of authority over the definition of safety and the application of the same standards to US companies as prerequisites through state media commentary'[3]. This means that safety discussions are not about normative discussions but rather an extension of the struggle for rule-making leadership. With the Taiwan issue as a structural red line, discussions on AI safety are likely to be treated as a secondary outcome of the summit'[3][6].
3. Historical Precedents and Comparative Cases
The AI discussions at this Washington meeting are a follow-up to the intergovernmental dialogue channel agreed upon during the May summit in Beijing[3][9]. The fact that the critical tone of state media towards the US has been maintained even after the May agreement indicates that the gap between agreements between leaders and practical implementation has already been confirmed once[9]. The current 'hotline' discussions also have the potential to repeat this pattern.
Comparisons with the Cold War US-Soviet nuclear crisis management system provide insights. The Brookings Institution proposed that the two leaders agree to have decisions on initiating cyberattacks against the other side's nuclear command, control, and communications systems or critical infrastructure made by humans rather than AI[11]. This shows that just as the US-Soviet hotline was a minimal confidence-building device to prevent escalation due to miscalculation, the current AI notification mechanism is designed not as a comprehensive normative agreement but as a minimal device to prevent escalation of accidental incidents. However, unlike the Cold War hotline, which operated on a clear balance of mutual assured destruction, the nature and threshold of threats in the AI domain remain unclear.
Comparisons with the 2023 San Francisco APEC summit are also valid. EAI distinguishes that 'while the previous meeting was a venue to mend relations strained by the balloon incident, this time it emphasizes the continuity of relationship management based on prior diplomatic exchanges in May'[6]. In other words, this meeting is characterized not as a restoration of relations after a crisis but as an attempt to institutionalize ongoing dialogue channels. However, if this attempt at institutionalization does not lead to substantial normative agreements, it is unlikely to break the pattern of symbolic agreements and the continuation of individual regulations that has persisted since 2023.
4. Key Variables Shaping Issue Development
The first variable is the direction of tensions between the industry and the administration in the US. Whether the Trump administration's rebuttal to the calls for a slowdown from Anthropic will end as a one-time incident or escalate into ongoing conflict between the industry and the government will determine the direction of US AI safety policy[17].
The second variable is the speed at which risk awareness spreads within China. Whether the warnings from the head of the Ministry of State Security remain exceptional statements or are reflected in Beijing's overall AI policy will change China's incentives to treat 'safety' as a substantive agenda in external negotiations[8].
The third variable is whether the notification mechanism will be institutionalized. Whether the notification mechanism discussed in the New York working-level consultations will move to a stage with specific operational regulations and responsible organizations after the summit, or remain a declarative agreement, is crucial[7][18]. EAI predicts that this channel is likely to remain a dual-track process limited to confidence-building levels of information exchange[9].
The fourth variable is the degree of linkage with security issues, including the Taiwan issue. If semiconductor export controls and rare earth supply chain issues continue to escalate on separate tracks from AI safety discussions, the symbolic agreements emerging from the summit will have little meaning beyond justification for negotiations[3][16].
The fifth variable is whether US-China bilateral agreements will overshadow multilateral governance discussions. In the context of the UN General Assembly, where 22 countries adopted a declaration stating that AI should be 'under human direction, supervision, and control'[14], there are concerns that a 'deal between the two great powers' could solidify a structure that excludes other countries from decision-making[15]. In this case, middle powers, including South Korea, would have no choice but to remain in a position of accepting the US-China agreement.
III. Final Recommended Response Measures
Analysis of AI 'Human Control' Statements and US-China AI Safety Cooperation Mechanisms: Comprehensive Response Measures
1. Comprehensive Assessment and Recommended Response Measures
The AI-related agreements emerging from this Washington meeting are closer to political gestures than normative achievements. President Xi's statement on 'human control' and President Trump's mention of 'no need for new regulations' provided domestic political justifications for both sides. No agreements with substantial binding force have been confirmed.
The key assessment is that the dual-track structure will remain unchanged after the meeting. The cooperation track led by the White House and the pressure track led by Congress and regulatory agencies are already operating separately[3][9]. Symbolic agreements between the leaders do not change this structure. According to EAI analysis, 'even if symbolic agreements emerge from the summit, it is highly likely that individual regulations such as semiconductor and AI export controls will continue separately'[3]. South Korean businesses and the government should not overinterpret the results of this meeting as a signal for improved relations.
The recommended response measure is a dual system that separates responses in security-related areas from general commercial sectors. Governmental notification mechanisms such as the AI hotline[18] are merely crisis management channels at the national security level and do not imply commercial AI cooperation or integration of technology standards. Confusing these two levels distorts policy judgments.
2. Short-term/Mid-term/Long-term Action Plans
In the short term (within 3 months), the focus should be on information gathering and reassessment. Verify whether there have been changes to the US Department of Commerce's semiconductor and AI export control lists following the meeting. At the same time, compare whether the tone of state media in China towards the US has actually changed before and after the meeting. EAI has pointed out that 'China also shows a duality in maintaining a critical tone towards the US in state media, regardless of the dialogue agreement'[9]. The maintenance of this duality will serve as the primary basis for assessment. Among domestic AI companies that have supply chain and cloud contracts with both the US and China, reassess whether there are security exception clauses in the contracts.
In the mid-term (6 months to 1 year), it will be a period to adjust the pace of integration into the US AI standards and certification system. While the US may utilize its AI safety framework as a strategy for diffusion among allies, China is likely to build a separate standards track. South Korea should observe both trends while maintaining the independence of its domestic AI safety assessment system rather than being early adopters of either standard. During this period, it will be possible to determine whether the US's moves towards security organization, such as the establishment of an 'AI Force'[6], lead to actual policy implementation or remain merely symbolic announcements.
In the long term (over 1 year), the focus will shift to participation in multilateral normative formation. There exists a multilateral track forming outside the US-China bilateral framework, such as the AI human control declaration adopted by 22 countries during the UN General Assembly[14]. The Economic Times has pointed out the risk that US-China bilateral negotiations could result in 'a deal between two great powers that excludes other countries'[15]. South Korea needs to actively participate in this multilateral track to secure a space for normative formation that is not subordinate to the US-China bilateral framework.
3. Monitoring Indicators and Trigger Points
The first indicator is whether there are follow-up actions regarding the AI hotline. Confirm whether the notification mechanism discussed in the New York meeting[7][18] is being concretized with specific operational regulations and responsible organizations, or if it fades away with the conclusion of the meeting. If no follow-up working-level consultations are scheduled within 90 days, it can be judged as a symbolic agreement.
The second indicator is the actual changes in the US semiconductor and AI export control lists. If there are movements to add or exclude Chinese companies from the BIS entity list regardless of the summit agreement, this will serve as empirical evidence of the separation structure between the cooperation track and the pressure track.
The third indicator is the tone of AI-related statements from China's Ministry of State Security and state media. Track whether the head of the Ministry of State Security's warnings about regime threats[8] are repeated or remain one-time statements. If they are repeated, it suggests that the risk awareness within Beijing may lead to actual policies (e.g., strengthening safety regulations for domestic AI companies).
The fourth trigger point is whether tensions related to Taiwan are reignited. In a situation where the Taiwan issue has solidified as a 'structural red line'[3], if this issue resurfaces, all cooperation tracks, including AI safety discussions, could simultaneously freeze. In this case, the risks in security-related areas must be immediately elevated.
The fifth indicator is the behavior of US big tech companies in China. Whether attempts to host Chinese AI models in a revenue-sharing manner[9] expand will serve as a leading indicator of the strength of commercial incentives that operate independently of government narratives.
4. Summary Conclusion
What has been confirmed in this meeting is the willingness of both the US and China to keep communication channels open through AI, not a substantial normative agreement. President Xi's statement on 'human control' and President Trump's mention of regulatory waivers stem from different domestic political needs and do not point in the same direction. Crisis management mechanisms such as the AI hotline are meaningful but limited to the security domain. The South Korean government and businesses should focus on establishing a dual risk management system and continuously monitoring individual actions by US regulatory agencies rather than interpreting this meeting as a signal for normalization of relations. It is also time to participate in the space for multilateral normative formation.
References
[7] [DW (Deutsche Welle)] US proposes AI safety mechanism in talks with China.
[8] [Council on Foreign Relations (CFR)] Why AI’s Biggest Rivals Are Suddenly Calling for Restraint.
[11] [Brookings Institution] Advancing human control of military AI.
[13] [Carnegie Endowment] Who Puts Guardrails on AI?
[14] [POLITICO] 22 countries back call to keep AI ‘under human control’.
[15] [Economic Times] US-China AI talks: A two-power bargain that could leave others out.
[16] [Hankyoreh] US-China Summit 'AI' 'Tariffs' Major Agenda...US Big Tech CEOs Attend Dinner.
[17] [The Diplomat] America’s AI Safety Dilemma and at the Trump-Xi Summit.
[18] [Al Jazeera] What’s the US–China AI ‘hotline’ that Trump plans to pitch to Xi Jinping?
[19] [Business Times (SG)] The tough AI questions looming over the Trump-Xi summit.
[22] [Nikkei Asia] Trump and Xi set to talk trade and AI as truce hangs in balance.
[24] [DW (Deutsche Welle)] How the US and China could end the AI race.
[25] [Al Jazeera] US and China presidents discuss cooperation, trade and AI risks.
*This text is an AI translation of an original written in Korean. Some translations or nuances may be inaccurate.
This report is an in-depth analysis planned by an EAI researcher, grounded in sophisticated AI-assisted research, and finalized by the EAI researcher.
How can I help you?