Intensifying Debates on Superintelligence Regulation and International Governance: The UK Case and South Korea's Strategy

Executive Summary

Despite bipartisan pressure from a majority in the UK Parliament, the Cabinet Office rejected the introduction of a “kill switch.” This was not a political retreat but a practical judgment that physically controlling decentralized commercial AI models is unfeasible. The UN High Commissioner for Human Rights has defined AI as an existential threat and urged the development of international norms, but the structural constraint of U.S.-China tech competition continues to suppress the pace of regulation. Over the next 12-18 months, a prolonged regulatory vacuum is the most likely scenario, and the early convergence of international norms should be seen as a low-probability outcome. South Korea needs a dual strategy: prioritizing the reform of its domestic institutions while also preparing a foothold for participation in multilateral discussions, even in the unlikely event of early regulatory convergence.

I. Situational Analysis

Intensifying Debates on Superintelligence Regulation and International Governance: A Situational Analysis

1. Background and Developments

The flashpoint for this issue was the UK Parliament in September 2026. Over 70 members of the House of Commons and the House of Lords sent a letter to Andy Burnham, the Secretary of State for Culture, Media and Sport, urging him to support a ban on the development of artificial superintelligence (ASI) [1]. The list of signatories is notable. It includes 15 former cabinet ministers, former Cabinet Secretary Sir Robin Butler, and senior MPs from across the political spectrum, including the Labour Party's John McDonnell [1]. This can be read not as an issue raised by a specific faction, but as a bipartisan concern across Westminster.

The immediate trigger was a warning from an AI industry insider. A post by a team leader at Anthropic, stating they “sincerely believe AI could kill everyone,” spurred UK parliamentarians into action [5]. Based on this, Labour MP Darren Jones sent an open letter to the UN and the OECD, calling for intervention in the “unsafe development of superintelligence” [5]. Wired described this period as “AI's summer of chaos,” explaining the context behind the flurry of responsive legislation from UK politicians over the summer [5]. Labour MP Alex Sobel also introduced a related bill in the House of Commons during the same period [5].

This trend is not unique to the United Kingdom. In South Korea, the EAI Working Paper series has already noted that “the rapid development of artificial intelligence is triggering revolutionary changes across all domains, including military, security, politics, diplomacy, economy, and society” [7]. The structural factor of U.S.-China tech competition suppressing the pace of safety regulation has also been pointed out in domestic editorials. The Hankyoreh pointed out that “despite a chorus of similar concerns from industry and experts, there is no regulatory movement to slow down development because the U.S. and China are unwilling to back down in the AI development race for future hegemony” [8]. The UK Parliament's call for a ban on superintelligence can be seen as a political attempt to break through this structural stalemate.

2. Current Situation

The UK government's response was a rejection. The Cabinet Office did not accept the demand to introduce a legal “kill switch” to forcibly shut down dangerous AI in an emergency. The BBC reported that the Cabinet Office stated, “The UK cannot simply switch AI off” [13]. The Daily Sabah reported the same fact, specifying that the Cabinet Office is the department overseeing AI safety [11]. In other words, the very department in charge of safety policy has drawn a line against hardline regulation, citing technical and administrative feasibility. This creates a direct clash between the political demands of a parliamentary majority and the practical judgment of the executive branch.

During the same period, warnings were also issued at the UN level. UN High Commissioner for Human Rights Volker Türk warned in a speech to the Human Rights Council that AI could pose an “existential threat to humanity” [15][16][17]. He asked, “The need for AI governance is widely acknowledged, but where is the action?” and stated that delays only benefit large tech companies and their owners [15]. Ahead of the start of his second term, Türk demanded “iron-clad guarantees” to ensure AI safety [16][17]. This statement, coinciding with the debate in the UK Parliament over a superintelligence ban, created a situation where national regulatory attempts and calls for UN-level norm-setting emerged simultaneously.

The NPT analogy presented in the Hankyoreh editorial is functioning as a symbolic reference point in this context [8]. The logic is to apply the precedent of the Non-Proliferation Treaty, which curbed the spread of nuclear weapons through a framework that included the major powers, to the establishment of norms for AI. However, an institutional path for this analogy to lead to an actual treaty is currently absent. The fact that SIPRI experts participated in the first informal UN consultation on military AI in Geneva in June [6] shows that multilateral discussions are in their initial stages. At the same time, this also means it is an early exploratory phase, still far from the formation of binding norms.

3. Key Actors and Positions

The Bipartisan Group in the UK ParliamentThis group is the primary actor pressuring the executive branch. The letter, signed by former ministers and senior MPs from both the government and opposition parties, carries significant political weight [1]. However, their demand for the UK to lead an international movement is not a goal achievable by the UK alone. Considering the size of the UK's AI industry and its international negotiating power, this is more of a call for symbolic leadership.

The UK Government (Cabinet Office)The government's position considers both regulatory effectiveness and industrial competitiveness. Its rejection of the kill switch with the logic that AI “cannot simply be switched off” [13] can be interpreted as the result of a combination of factors: the practical judgment that it is difficult to centrally control technically decentralized and replicated AI systems, and industrial policy considerations aimed at not undermining the international competitiveness of the UK's own AI industry.

The Office of the UN High Commissioner for Human Rights (OHCHR)The OHCHR plays the role of providing the moral justification for norm formation. Türk's statements are closer to exercising political pressure than to a binding measure [15][16][17]. The discussion on military AI, co-hosted by the UN Institute for Disarmament Research (UNIDIR), the UN Office for Disarmament Affairs (UNODA), and SIPRI [6], shows that the human rights and military security frames are proceeding in parallel within the UN.

Insiders at Frontier AI Companies like AnthropicParadoxically, these insiders are the biggest catalysts for the regulatory debate. Their public warnings about the dangers of their own technology [5] are, in fact, stimulating legislative attempts by politicians. This can also be read as a strategy by companies to secure a voice in the regulatory design process by preemptively leading the safety discourse.

The United States and ChinaAlthough not visible on the surface of this debate, they form the structural backdrop. As the Hankyoreh pointed out, the hegemonic competition between the two countries is the root source of pressure for deregulation [8]. A briefing from the Brookings Institution assesses that the gap between the U.S. and China has actually widened through a series of AI summits, including at the G7 and the UN [12]. In such a structure, the international norm-setting demanded by the UK or the UN is likely to have only a limited effect without the substantive participation of both the U.S. and China.

4. Key Issues

The first issue is the substance of the risk. There is no consensus on whether the 10% probability figure for human extinction [1] is a verified scientific estimate or a discursive rhetorical device. This uncertainty is the fundamental reason for the divide between the hardline response from politicians and the cautious stance of the executive branch.

The second issue is the feasibility of regulatory tools. The UK government has directly pointed out the gap between creating a clear control point like a kill switch and the technical characteristics of decentralized AI systems [13]. This problem is not limited to the UK; it is a structural constraint that will apply equally to all national-level regulatory attempts.

The third issue is the path to international norm-setting. The NPT analogy [8] presupposes the inclusion of major powers. However, in the current U.S.-China competitive dynamic [8][12], the incentive for both countries to voluntarily slow their pace of development is weak. The moral pressure from the OHCHR [15][16][17] and the discussions on military AI by expert bodies like SIPRI [6] are seeds for norm formation, but a political path to a binding treaty is not yet visible.

The fourth issue is the politics of the discourse. It is difficult to distinguish whether the warnings from corporate insiders are genuine risk assessments or strategic positioning to secure a voice in the regulatory design process. This is a crucial point that middle powers, including South Korea, must address when participating in international discussions. Failure to discern the political nature of the risk discourse could lead to ceding the initiative in regulatory design to frontier companies and major powers.

II. In-Depth Analysis

Intensifying Debates on Superintelligence Regulation and International Governance: An In-Depth Analysis

1. Analysis of Root Causes

The root cause of this situation is the time lag between technological risk assessment and political response. The public warning from an Anthropic insider triggered not a debate on the reality of the risk, but the speed of the political reaction to the risk narrative [5]. Wired directly conveys MP Darren Jones's logic that the government must intervene “whether it's marketing hyperbole or a genuine harbinger of extinction” [5]. In effect, a structure has been created that demands political action before the veracity of the risk can even be confirmed. This is a classic pattern where avoiding political responsibility takes precedence over verifying the risk.

Internal divisions within companies are another root cause. The resignation of Anthropic researcher Jacob Coxson and Evan Hubinger's statement about a 10% extinction probability reveal a lack of consensus on safety even within a frontier AI developer. The fact that a company that has championed safety as a top priority has itself exposed internal alarm bells to the public demonstrates that there is no objective risk standard for regulators to reference in the first place.

This is coupled with structural incentives in the industry. The fact that the U.S.-China AI hegemonic competition is a fundamental condition constraining the pace of safety regulation has already been pointed out in domestic editorials. The assessment is that “there is no regulatory movement to slow down development because the U.S. and China are unwilling to back down in the AI development race for future hegemony” [8]. The UK Parliament's call for a ban on superintelligence was a unilateral attempt to go against this international competitive dynamic, and was therefore destined to face a debate on its effectiveness from the outset.

2. Structural Context

Political Structure: The Legislative-Executive Divide

The bipartisan coalition within the UK Parliament is politically unusual. The letter, signed by 15 former ministers, former Cabinet Secretary Sir Robin Butler, and senior figures from Labour's left-wing like John McDonnell, as well as from the Conservative, Liberal Democrat, and Scottish National parties, represents pressure that transcends party lines [1]. However, the Cabinet Office, which is actually responsible for AI safety policy, did not accept it. The BBC reports the Cabinet Office stated, “The UK cannot simply switch AI off” [13]. This reflects a structure where the symbolic political actions of the legislature clash with the technical and executive realities assessed by the administration. Given that the very concept of a kill switch was proposed without accounting for the physical impossibility of controlling decentralized commercial AI models, the government's rejection is less a political retreat and more an acknowledgment of practical limitations.

This divide also represents a universal dilemma facing AI regulation. The Brookings Institution points out that the very structure forcing middle powers to choose between the U.S. and China is itself an economic and geopolitical risk [9]. The UK finds itself needing to strike a balance between attempting to lead on norms and protecting its own industrial competitiveness amidst the two great powers. The rejection of the kill switch can be interpreted as a decision that tilts this balance toward industry.

Security Structure: Links to the Military Domain

Discussions at the UN level are not limited to the human rights domain. Experts from the Stockholm International Peace Research Institute (SIPRI) participated in the first informal UN consultation on military AI in Geneva in June [6]. The fact that this consultation was co-hosted by the UN Institute for Disarmament Research (UNIDIR), the UN Office for Disarmament Affairs (UNODA), and the Office of the High Commissioner for Human Rights (OHCHR) shows that High Commissioner Türk's recent warning is not confined to his Human Rights Council speech but is linked to disarmament and security tracks [6]. In other words, the debate on superintelligence risk is being addressed not just as a matter of civilian chatbot safety, but in the same context as the regulatory vacuum for military AI applications like lethal autonomous weapons systems (LAWS). The EAI Working Paper series has also separately addressed the “Dynamics and Challenges of Governance Competition over Lethal Autonomous Weapons Systems,” suggesting that the norm discussions on the civilian and military tracks effectively stem from the same root [7].

Economic Structure: The Beneficiaries of Regulatory Delay

High Commissioner Türk directly pointed out that “delay only benefits the giant tech companies, their owners, and their enablers” [15]. This statement reframes the AI safety regulation debate not as a simple technical risk assessment, but as an issue of profit distribution. A structural incentive exists whereby the longer the regulatory vacuum persists, the more entrenched the market dominance of leading companies becomes. This explains why frontier companies prefer self-regulation and are passive about introducing legally binding international norms. A Council on Foreign Relations (CFR) survey of 350 experts also found consensus that “no one is prepared to control coming AI capabilities” [2]. This means there is agreement on the lack of preparation, but no agreement on who should fill the gap and how.

3. Comparison with Historical Precedents and Similar Cases

The axis of comparison presented by the Hankyoreh editorial is the Nuclear Non-Proliferation Treaty (NPT). The editorial argues, “There is no choice but to introduce international norms that encompass all major powers, like the NPT which prevented the crisis of unchecked nuclear proliferation” [8]. However, this comparison has a fundamental asymmetry. Whereas nuclear weapons development was restricted to states and the acquisition of materials itself was a high barrier to entry, frontier AI is developed by private companies and consists of relatively accessible elements like computational resources and data. The physical inspections to determine “who possesses what,” which are a premise of an NPT-style verification regime, are technically difficult to apply to AI. This connects to the UK government's logic for rejecting the kill switch. Nuclear weapons can be physically sealed and dismantled, but AI models that are decentralized and distributed even as open source cannot be turned off with a single switch.

Another axis of comparison is the international climate change regime. The Paris Agreement's adoption of voluntary commitments (Nationally Determined Contributions, or NDCs) instead of binding reduction obligations was the result of sovereign states resisting binding norms that would harm their industrial competitiveness. A similar pattern can be detected in the AI governance debate. The UK's rejection of the kill switch and the U.S. and China's avoidance of a regulatory race are both choices that prioritize national industrial and security interests over international norms. However, unlike with climate change, there is not even a scientific consensus on the timing and scale of AI risks. Therefore, the current situation should be seen as an early stage where even a minimal commitment system on the level of the Paris Agreement has yet to be formed.

The pattern of consecutive G7 and UN high-level discussions, as summarized by the Brookings Institution, shows characteristics similar to the early phase of Cold War arms control negotiations [12]. The dynamic of the U.S. and China each rallying their respective blocs to engage in a competition over norms could potentially replicate the pattern of pre-détente U.S.-Soviet arms control talks, which were deadlocked for a long period amidst mutual distrust. However, whereas Cold War arms control was centered on bilateral negotiations, the AI norm debate involves the UK Parliament, the UN High Commissioner for Human Rights, the G7, and individual companies all raising the issue simultaneously on different tracks, meaning the negotiating parties themselves are fragmented.

4. Key Variables Shaping Future Developments

The first variable is the intensity of the U.S.-China competition. The Brookings Institution assesses that the series of AI summits held in the summer of 2026 actually highlighted the gap and disagreements between the U.S. and China [12]. As long as both countries prioritize securing technological superiority over competing on norms, discussions on a superintelligence ban led by the UK or the UN are unlikely to lead to a binding multilateral regime.

The second variable is the direction of the conflict between the legislature and the executive in the UK. The key factors are the fate of the bill introduced in the House of Commons by MP Alex Sobel and what practical regulatory alternatives the Cabinet Office will propose in place of a kill switch [5][13]. If the government repeatedly rejects proposals without offering alternatives, bipartisan parliamentary pressure is likely to intensify again.

The third variable is whether the UN track becomes institutionalized. The watershed will be whether High Commissioner Türk's warning remains rhetoric at the Human Rights Council level or leads to a substantive multilateral consultative body, like the informal military AI consultation involving SIPRI [6][15]. If the consultations co-hosted by UNIDIR, UNODA, and OHCHR become regularized, this could serve as an initial institutional foothold for discussions on introducing an NPT-style international regime.

The fourth variable is whether internal divisions within companies will spread. If the warnings and researcher resignations originating from Anthropic spread to other frontier companies, the credibility of the industry's self-regulation logic could be rapidly undermined. In that case, the possibility that political pressure demanding government intervention could spread beyond the UK to the United States and the EU cannot be ruled out.

From South Korea's perspective, the combination of these four variables will determine the window of opportunity for a middle-power response strategy. As the Brookings Institution points out, middle powers, including South Korea, face a structural vulnerability of being forced to choose between the U.S. and China [9]. In a context where the discourse on superintelligence risk is consumed more as political rhetoric than as actual technical verification, it is more advantageous for South Korea to participate in the early stages of norm formation and engage in discussions on designing verification standards, rather than passively accepting norms set by major powers later on.