[AI and New Civilization Standards Special Report] Military Challenge ②: AI-Based Autonomous Weapon Systems, The Evolution of Cognitive Warfare and Changes in the Military Security Order

I. The Reality of AI-Based Warfare Has Already Arrived

It is difficult to imagine what future wars will look like in the era of advanced technologies, including Artificial Intelligence (AI). James Johnson, who researches AI-based nuclear weapons, paints a picture of a hypothetical war in the following scenario, just one year from now.

In 2025, war breaks out between the United States and China over the Taiwan issue, and the war quickly escalates into nuclear war. After the war, independent investigators conclude that neither side used AI-based 'fully autonomous' weapons. They optimistically assess that the laws of armed conflict as defined by international law, or the principles of proportionality and distinction applicable to the use of nuclear weapons, were not intentionally violated. Both countries believed they were legally justified in their use of military force under the right of self-defense at the time.

Amidst rapid political changes in Taiwan prior to the war, both the U.S. and China deploy AI technologies to support battlefield awareness, intelligence gathering, surveillance and reconnaissance (ISR), and tactical responses to enemy actions. The rapid improvement in the accuracy, speed, and predictive capabilities of commercially produced dual-use AI applications has enabled major powers to leverage machine learning (ML), which requires data to enhance tactical and operational maneuvers. China, particularly inspired by the successful use of autonomous drone swarms by Russia, Turkey, and Israel to repel terrorist attacks on their borders, rapidly integrated the latest dual-use AI technologies, bypassing thorough testing and evaluation processes.

As Chinese military incursions in the Taiwan Strait rapidly increased, U.S. and Chinese leaders pursued the rapid deployment of the latest strategic AI systems to gain the greatest asymmetric advantage. The cutting-edge strategic AI systems generated novel strategic alternatives based on historical combat scenarios, experimental war games, rational decision-making based on game theory, intelligence data, and learning from previous versions. The high complexity of these systems exceeded the speculation of their designers and operators.

Fierce rhetoric and disinformation campaigns on social media reached their peak, and voices demanding the urgency of forced unification with Taiwan grew louder within China. As the situation in the Pacific became increasingly tense, the U.S. decided to deploy its autonomous AI-based 'Strategic Prediction & Recommendation System (SPRS)' for non-combat activities, despite incomplete testing and evaluation. In response, China also deployed a similar 'Strategic & Intelligence Advisory System (SIAS)' to prepare for all crisis situations.

Ultimately, a chain of events, accelerated by AI, exceeded Beijing's predefined limits, leading to an uncontrollable situation. Information operations culminated in a surge of cyber intrusions into the U.S. Indo-Pacific Command and Taiwanese military systems, coupled with moves to defend China's space assets and the activation of the People's Liberation Army's (PLA) automated procurement systems. The U.S. SPRS assessed these actions as major national security threats and recommended a strong show of force and defensive posture. Consequently, the U.S. authorized autonomous strategic bomber flights in the Taiwan Strait. SIAS recommended to Chinese leaders a limited preemptive strike on key U.S. Pacific assets to secure an early advantage. SPRS warned Washington of an imminent Chinese preemptive strike, and the U.S. recommended an immediate limited nuclear strike. It predicted that U.S. missile defense systems could successfully intercept most Chinese tactical nuclear attacks and that China would refrain from retaliating against the U.S. mainland. SPRS's predictions proved accurate. After a limited U.S.-China nuclear exchange in the Pacific, resulting in millions of deaths and tens of millions of injuries, both sides agreed to a ceasefire.

In post-war analysis, both sides attempted detailed analyses of the decision-making processes of SPRS and SIAS, but the AI designers concluded that it was impossible to explain the AI's logic and reasoning for all subsets of decisions. Due to various time, encryption, and privacy constraints, post-hoc test logs and protocols could not be preserved. A clear conclusion could not be reached on whether AI technology caused this war (Johnson 2023).[1]

Recent conflicts such as the war in Ukraine and the Gaza war clearly demonstrate that AI is already being utilized in warfare at various levels, and this trend is expected to continue. When AI is combined with the three dimensions of nuclear weapons, conventional weapons, and psychological and information warfare, it will rapidly evolve into AI-based nuclear weapons, AI-based autonomous weapon systems, and AI-enabled cognitive warfare.

AI is a general-purpose technology, a meta-augmenting technology that enhances existing technologies. It will be a multidimensional technology that not only improves specific weapons but also transforms weapon operating systems, policy decisions related to war, and the perceptions and responses of governments, militaries, and societies involved in waging war (Britt 2023; Hale 2023). Just as everyone uses mathematics, AI technology will be used in all dimensions, and to a significant extent, it already is. It is difficult to ascertain what changes AI will bring to the field of military security, let alone the extent of AI technology's effects on international politics as a whole. AI technology itself is rapidly changing, and efforts to regulate or control it lag significantly behind the pace of technological change. The development of AI technology coincides with the so-called return of geopolitics, occurring amidst fierce competition, making international efforts to control the problems of AI difficult to achieve results.

Amidst geopolitical competition between nations, the possibility of preventing the military use of AI technology is slim. Furthermore, we can consider the possibility that AI technology may develop rapidly in unintended directions, escaping human control. Since nations are embracing AI as a useful and innovative means to enhance their own security, it is difficult to foster preemptive vigilance regarding AI military technology until an incident occurs that confirms common damage to the international community.

It is now necessary to grasp the reality of AI technology being applied to nuclear weapons, autonomous weapon systems, and cognitive warfare, and to initiate serious discussions by anticipating potential common problems. In the case of nuclear weapons, a taboo emerged after their single use, and the Treaty on the Prohibition of Nuclear Weapons (TPNW) was even concluded in the 2020s. Since the future of warfare concerns the lives and deaths of citizens of all nations, it is important to pay attention and respond accordingly. South Korea, too, must consider how to perceive and respond to AI-based weapon systems and military security strategies amidst the arms race and the risk of war between major powers. This article, following an analysis of AI-based nuclear weapons, addresses autonomous weapon systems and cognitive warfare.

II. The Evolution of Autonomous Weapon Systems Combined with AI

According to the U.S. Department of Defense Directive 3000.09 on the Autonomy of Weapon Systems, published in 2012, an autonomous weapon system can be defined as "a weapon system that, once activated, can select and engage targets without further human operator intervention." In other words, it is a lethal weapon system that identifies potential enemy targets and independently selects and engages them based on algorithms and AI. This can refer to the autonomy of specific individual weapons or the overall operational system utilizing multiple weapons.

Although there is currently no international consensus on the definition of autonomous weapon systems, they have been evaluated based on their level of autonomy from human control. The concept of autonomy in relation to autonomous weapon systems can be defined as the ability of a system to perform tasks or a series of operations through interaction with its environment without human input. The crucial aspect in the definition of autonomous weapon systems appears to be the type of decisions or functions performed autonomously, no longer under the control of a human operator (Dresp-Langley 2023).

Three levels of classification are commonly used based on the degree of weapon autonomy. First, supervised autonomous weapons, or 'human-on-the-loop' systems. These are autonomous weapon systems designed to allow a human operator to intervene and terminate engagements before an unusually high level of damage occurs. Examples include defensive weapon systems that independently select and engage targets according to programming, but all operations are fully supervised by humans, who can disable the system within a limited time if necessary.

Second, semi-autonomous weapons, or 'human-in-the-loop' systems, which are intended to engage only specific targets or groups of targets selected by a human operator once activated. Examples include munitions like guided missiles, which, when fired at a specific target location, identify and engage targets within that area based on pre-programmed target categories.

Third, fully autonomous weapons, or 'human-out-of-the-loop' systems, which can select and engage targets without further human operator intervention once activated. Examples include loitering munitions, which, after being launched, search for and engage targets in a specific area, operating without additional human intervention. Weapon systems that autonomously jam communications can also be included (Kallenborn 2021).

Autonomous weapon systems are the product of extensive historical experience and effort. One of the earliest examples of an autonomous vehicle was 'The American Wonder,' developed in 1925. This vehicle drove on the streets of New York City, was remotely controlled by other following vehicles, and moved in a convoy under early automatic control systems.

Autonomous weapon systems require the integration of several key components: a mobile combat platform, various types of sensors capable of precisely monitoring the platform's surroundings, a processing system to classify detected objects, and algorithms that initiate an attack when an acceptable target is detected. Killer robots, or 'slaughter bots,' are autonomous robotic systems capable of selecting and engaging targets without human intervention. Equipped with advanced sensors and AI, they can operate collaboratively to overwhelm defenses and can be operated autonomously and unmanned on distributed surface combat groups or electronic warfare ships.

Drones are also representative autonomous weapon systems. In October 2013, the U.S. Office of Strategic Capabilities launched 103 Perdix drones, which used a 'distributed brain' to assemble into complex formations, traverse the battlefield, or reconfigure into new formations. This drone swarm was built by Massachusetts Institute of Technology (MIT) students using commercially available components and designs. In theory, drone swarms can be scaled to tens of thousands, becoming autonomous weapons comparable to small nuclear devices.

Currently, lethal autonomous weapons and autonomous weapon systems utilizing or under development with AI show diverse advancements, including autonomous sentry guns, remote weapon stations, killer robots programmed to fire at humans and vehicles, and drones and drone swarms with autonomous target designation capabilities. Features such as RADAR, LIDAR, high-resolution 360-degree cameras, and ultimately adaptive and predictive cruise control combined with AI have also advanced. New maximum-risk weapons of mass destruction could be drone swarms and autonomous chemical, biological, radiological, and nuclear (CBRN) weapons, including miniature insect drones capable of injecting lethal biochemical agents.

Autonomous sentry guns are also being developed. A sentry gun is a remote weapon that automatically aims and fires at targets detected by sensors. The first functional military sentry gun was used as a close-in weapon system to detect and destroy incoming missiles and enemy aircraft at short range. While these weapons were initially used only on ships, they are now also employed for ground defense. The SGR-A1, equipped with an integrated system including surveillance, tracking, firing, and voice recognition, was a highly classified project jointly developed by Hanwha Defense and Korea University and used to support the South Korean military in the Korean Demilitarized Zone (DMZ).

These AI-based autonomous weapon systems are increasingly appearing on current battlefields or in training exercises for future contingencies. Unmanned aerial vehicles have emerged as a significant issue in the ongoing war in Ukraine, and the U.S. Department of Defense recently announced a new $1 billion investment to upgrade its drone fleet. China's autonomous killer robots are expected to be used for military purposes on the battlefield within two years, with one expert calling them "the greatest threat to human survival" and heralding a new era of AI-based warfare.

Several major countries are taking this development a step further, beginning to develop fully autonomous AI-based killer robots to replace soldiers on the battlefield. China and Russia are reportedly already collaborating on the development of AI-based autonomous weapons. During military exercises with Cambodia in May 2024, the Chinese People's Liberation Army showcased robotic dogs equipped with guns manufactured by the Chinese company Unitree Robotics. Russia, at a weapons exhibition near Moscow in 2022, displayed a modified Unitree Robotics dog armed with a rocket-propelled grenade launcher, rebranded as the M-81 robot dog.

China has already begun developing weapons using AI-based machines, with some experts predicting a threefold increase in bomb and shell production by 2028. In March 2023, at a United Nations meeting on lethal autonomous weapon systems, the U.S. representative argued that it was not the appropriate time to begin regulating their development. This unchecked development has raised numerous warnings that non-human weapons will be unable to comply with the laws of war, and that nations will be more likely to engage in conflict without fear of troop casualties (Cameron 2024).

III. U.S.-China Strategic Competition and U.S. Autonomous Weapon System Strategy

In 2012, the U.S. Department of Defense became the first in the world to issue guidelines on the deployment of autonomous or semi-autonomous weapon systems. Directive 3000.09 was one of the first policies to seriously consider the impact of autonomous technologies and AI on the military. Before this, such concepts were primarily confined to science fiction. Upon the release of this directive, significant public interest and discussion arose among the public, civil society, and non-governmental organizations, but it is true that confusion and misunderstanding arose regarding the directive's precise content. For example, Human Rights Watch (HRW) mistakenly interpreted the directive as "the world's first moratorium on lethal fully autonomous weapons." However, the policy did not actually restrict the development or use of autonomous weapons; it merely stipulated that it must be updated, rescinded, or renewed every ten years. The directive required additional review procedures for weapon systems that reached a certain level of autonomy, but the scope and purpose of that review were unclear.

Over the past decade, Directive 3000.09 has become a key U.S. policy concerning weapon systems incorporating autonomous technologies. However, the lack of clarity in this directive has caused confusion, particularly in discussions about the development of new technologies, including AI. When the directive's ten-year validity expired at the end of 2022, the Department of Defense presented a revised directive in 2023. The revised version retains the definition of autonomous weapon systems used in the 2012 directive, defining them as "weapon systems that, once activated, can select and engage targets without further human operator intervention." The 2023 directive removed the word 'human' preceding 'operator' but defines 'operator' as "a person who operates a platform or weapon system." Like the previous directive, the 2023 directive does not curb the proliferation of autonomous weapon systems. Not only the U.S., but also Australia, China, India, Iran, Israel, South Korea, Russia, Turkey, and the United Kingdom are investing heavily in the military applications of artificial intelligence and related technologies, thereby developing autonomous weapon systems for air, ground, and sea-based platforms.

Section 1.2(e) of the directive permits "international sales and transfers" that are "approved in accordance with existing technology security and foreign disclosure requirements and procedures." Once these weapons leave the U.S., the U.S. loses exclusive control over them. Furthermore, the directive enables the continued development and acquisition of autonomous weapon systems in accordance with existing laws, Department of Defense procedures, and ethical principles.

The 2023 directive does not provide clearer regulations for the ambiguities present in the 2012 directive. According to Section 1.2(a), the 2023 directive borrows from the provisions of the previous directive, stating that "autonomous and semi-autonomous weapon systems must be designed to allow commanders and operators to exercise an appropriate level of human judgment over the use of force." While this wording acknowledges the value of human judgment, which is essential for compliance with international humanitarian law, neither the previous nor the current directive specifies what constitutes an "appropriate level" of human judgment or who will determine it (Human Rights Watch 2023).

However, the revised directive more realistically reflects how rapidly changing technologies can actually be applied. For example, new references to AI have been added, and a clear distinction has been made between AI and autonomy. It specifies that even when autonomous functions are added to existing systems, they must undergo this additional review process, which also applies to systems that have been in use for many years (Lamberth 2023).

This U.S. strategy is mindful of the strategic competition with China. The U.S. 'Replicator' program, announced in August 2023, addresses significant technological and human challenges in defense procurement and development, as the AI revolution is poised to transform warfare.

At the time of the announcement, Deputy Secretary of Defense Hicks emphasized the necessity of developing U.S. autonomous weapon systems in the context of strategic competition with China.

We will deploy mass against the mass of the People's Liberation Army (PLA), but our forces will be unpredictable, difficult to target, and even more difficult to counter. Through superior talent, smart concepts, and advanced technology, our military will become more agile with the support and urgency of the commercial sector.

As we have seen in Ukraine, new technologies developed by commercial and non-traditional enterprises, from Starlink to Switchblade, to commercial satellite imagery, can play a decisive role in defending against modern military attacks. These technologies are crucial complements to traditional capabilities, which remain essential. We have set ambitious goals for 'Replicator' because we must break down barriers and accelerate change urgently. The goal is to deploy thousands of attritable autonomous systems across multiple domains within the next 18-24 months... And 'Replicator' will not be solely about production. We will also aim to replicate and inject the methods for achieving this goal, enabling us to repeatedly scale what is relevant in the future. In accordance with the National Defense Strategy and the Joint Warfighting Concept, we will employ autonomous capabilities in ways that leverage our persistent advantage. ... In this regard, autonomous systems across all domains will help address the challenges of Anti-access/area-denial (A2AD) systems. We will neutralize their A2AD with our A2AD (Hicks 2023).

The Department of Defense currently has over 800 declassified AI-related projects in its portfolio, many of which are reportedly still in testing. Generally, machine learning and neural networks are helping humans gain insights and create efficiencies. For a more efficient network among warfighters, the Department of Defense is pursuing the development of a combat network called Joint All-Domain Command and Control (JADC2), which automates data processing across all military branches. However, this endeavor is large-scale and faces the challenge of overcoming bureaucracy.

For example, the Air Force's 'Loyal Wingman' program plans to pair manned aircraft with autonomous aircraft. For instance, an F-16 pilot could send drones to conduct reconnaissance, lure enemy attacks, or engage targets. Air Force leaders aim to operationalize this program by the late 2020s. However, the Loyal Wingman's schedule is overly ambitious and presents issues that do not align well with the 'Replicator' timeline. Currently, the only weapon systems considered reliable for autonomous operation are purely defensive ones, such as the 'Phalanx' missile defense system. Issues such as systems not operating as predicted or harming non-combatants or friendly forces are emerging as more critical concerns than autonomous weapons making independent decisions (Bajak 2023).

IV. Efforts to Regulate Autonomous Weapon Systems

Efforts to control the destructive potential of autonomous weapons, particularly the unpredictable problems associated with AI, have been ongoing. There are regulatory efforts for AI in general, such as the Seoul High-Level Meeting on Responsible Artificial Intelligence in the Military Domain (REAIM Summit 2024) to be held in September, as well as efforts concerning lethal autonomous weapon systems themselves. Historically, the Convention on Prohibitions or Restrictions on the Use of Certain Conventional Weapons (CCW) entered into force in December 1983; this convention is an international agreement that prohibits and restricts the use of certain conventional weapons that cause inhumane consequences. In 2010, the UN General Assembly received a report from Special Rapporteur Philip Alston highlighting the problems of lethal robot technology, marking the beginning of discussions related to the regulation of lethal autonomous weapon systems.

In 2011, official discussions on autonomous weapon systems began at the International Committee of the Red Cross (ICRC), and discussions on the problems and concepts of autonomous weapon systems took place at an ICRC meeting held that year. In November 2013, the Meeting of the States Parties to the CCW decided to begin discussions on Lethal Autonomous Weapon Systems (LAWS). Subsequently, three informal expert meetings were held under the CCW from May 2014 to 2016, during which discussions on lethal autonomous weapon systems took place.

In December 2016, the 5th CCW Review Conference decided to establish a Group of Governmental Experts on Lethal Autonomous Weapons Systems (GGE LAWS) with the participation of all States Parties. From 2017 onwards, CCW meetings transitioned to open-ended Group of Governmental Experts meetings without participant restrictions. In 2018, the Group of Governmental Experts on Lethal Autonomous Weapons (GGE LAWS) meeting adopted a report including 10 Possible Guiding Principles, through discussions on the characteristics of lethal weapons utilizing autonomous technologies such as AI, human control, policy options, and military applications. In 2019, the CCW Meeting of the States Parties finally adopted 11 guiding principles, adding "human-machine interaction" (Yoo Jun-gu 2019a; 2019b).

On December 22, 2023, the UN General Assembly adopted its first resolution on the risks of lethal autonomous weapon systems. The resolution passed with 152 votes in favor, 4 against, and 11 abstentions, acknowledging the "serious challenges and concerns" posed by the application of new technologies in the military domain, including AI and autonomy. Within the CCW framework, meetings of the Group of Governmental Experts on lethal autonomous weapon systems continued in 2020 and 2021. In November 2023, CCW States Parties agreed to hold meetings over 20 days in 2024 and 2025 to "consider and formulate elements of a legally binding instrument by consensus."

China maintains strategic ambiguity regarding autonomous weapon systems. While it advocated for the prohibition of autonomous weapon systems at the UN Convention on Certain Conventional Weapons (UN-CCW) in 2016, it announced its AI development plan in 2017, thereby promoting the development of autonomous weapon systems. Overall, it appears to favor restricting the use of fully autonomous weapon systems, but China also aims to secure military superiority through development, pursuing a civil-military fusion strategy to achieve this goal.

Meanwhile, calls for negotiating a new international treaty to regulate lethal autonomous weapon systems are growing within the international community. Non-governmental organizations, including Human Rights Watch, are demanding the initiation of negotiations for a binding international treaty, using the UN General Assembly resolution as a catalyst. These trends indicate increasing international concern over lethal autonomous weapon systems and a growing momentum in discussions to regulate them.

Among the ongoing regulatory efforts, several key issues are being debated. First, the definition of autonomous weapon systems. The scope of autonomous weapon systems is extremely broad, and with increasing technological complexity, efforts to understand how weapon systems operate, their applications, and potential consequences are becoming more difficult, posing challenges to ensuring human responsibility and accountability.

Second, discussions on the form of new regulations. Consensus on how regulations will actually function remains elusive. Some countries advocate for establishing rules on the use and transfer of these systems, while others call for a strict ban on certain types of weapon systems. Many other countries desire a dual approach combining prohibition and regulation.

Third, discussions on the need for new global instruments. There is broad consensus among the UN, the ICRC, and nearly all countries with political will for regulating autonomous weapon systems that international humanitarian law and international human rights law apply to their development and use. However, debate continues on whether existing international law is sufficient or if new, technology-specific rules are needed for effective control.

Fourth, discussions on which forum to use. To date, inter-state discussions on autonomous weapon systems have primarily taken place within the Convention on Certain Conventional Weapons (CCW), with governments first convening a meeting on the topic of fully autonomous weapon systems in 2013. In 2017, a Group of Governmental Experts on Lethal Autonomous Weapon Systems was established under the CCW framework, tasked with promoting a common understanding of the issue. However, this group only agreed on further considerations for addressing the issue, falling short of discussing legal or regulatory options, and failed to achieve substantial progress on the UN Secretary-General's call for international regulation. Recent regional meetings on autonomous weapon systems have urged states to consider pursuing regulatory discussions through other forums, such as the UN General Assembly. The idea that international treaties and conventions can be achieved through broad, albeit not unanimous, support has been a factor enabling progress in other areas, such as the adoption of the Arms Trade Treaty.

While many difficult questions remain for states to consider in the coming months, the UN Secretary-General's goal of reaching an international agreement on the regulation of autonomous weapon systems by 2026 provides a deadline we should aim for. The UN General Assembly resolution on lethal autonomous weapon systems mandates the Secretary-General to prepare a report by the end of this year, reflecting the diverse views of states, international and regional organizations, the ICRC, civil society, and the scientific and industrial communities.

Another key aspect of future discussions will be the meaningful participation of non-Western countries and civil society. Civil society organizations have played a crucial role in leading international discussions on the regulation of military AI and autonomous weapon systems, conveying the concerns of citizens worldwide and fostering a deeper understanding of technological advancements and their potential risks to humanity as a whole (Linney and Tang 2024).

If regulations on autonomous weapon systems are to become a reality in the future, numerous issues will arise. As discussed, the definition of autonomous weapon systems itself and the framework for regulation are problematic, but the issue of inequality between major powers and weaker states surrounding the development of autonomous weapon systems may also arise. Comparing this to the Nuclear Non-Proliferation Treaty (NPT), it has been criticized as inherently unequal because it established a system that allows only a few countries to possess nuclear weapons while prohibiting others from acquiring them. Similarly, a regulatory regime for autonomous weapon systems may allow some countries to possess them while prohibiting others from acquiring them, leading to another form of unequal international system.

Another reason why a regulatory regime for autonomous weapon systems is difficult is the challenge of verification and enforcement. Autonomous weapon systems are software-based and can be dual-use, making them very difficult to detect or regulate their development. The pace of technological advancement may outstrip regulatory efforts, and states may be reluctant to agree to regulations for fear of losing technological superiority. Particularly, since AI is a general-purpose technology developed for various commercial and civilian benefits and utilities, it is very difficult to regulate. Considering that many advanced technology companies, not just governments, are the developers, their opinions must also be taken into account. Ultimately, the international community faces the difficulty of comprehensively considering the positions of various stakeholders and establishing a comprehensive, equitable, and transparent system for autonomous weapon systems, learning from the inequality of the nuclear non-proliferation regime.

V. Great Power Competition and the Future of Autonomous Weapon Systems

There is little disagreement among scientists, industry experts, and Department of Defense officials that the United States will possess fully autonomous lethal weapons within the next few years. While U.S. Department of Defense officials claim that humans will always retain control, experts suggest that the speed of data processing and advancements in machine-to-machine communication will eventually push humans into supervisory roles. This is particularly true when lethal weapons are deployed en masse, as in the case of drone swarms. It is also unclear whether the U.S. Department of Defense is currently evaluating autonomous weapon systems for deployment through official channels in accordance with the 2012 directive and its revised version. The situation is even more problematic for countries other than the U.S., where clear guidelines are even more ambiguous.

Considering these points, it is inevitable to conclude that competition among great powers surrounding autonomous weapon systems will intensify in the future, and the prospect of regulating them will be difficult. First, autonomous weapon systems pose a problem in that their behavior is dangerously unpredictable. The complex interaction between machine learning-based algorithms and dynamic operational environments makes it extremely difficult to predict the behavior of these weapons in real-world situations. These weapon systems are inherently designed to be unpredictable, as they are programmed to act unpredictably to stay ahead of enemy systems.

Second, autonomous weapon systems pose a risk of accidental and rapid escalation of conflict due to their operational speed and scale. A recent study by the RAND Corporation concluded that "the speed of autonomous systems led to unintended escalation of conflict in a war game" and that "widespread AI and autonomous systems could lead to unintended escalation and crisis instability." The United Nations Institute for Disarmament Research (UNIDIR) agreed with the RAND Corporation's conclusion. The National Security Commission on Artificial Intelligence (NSCAI), a U.S. quasi-governmental agency, also acknowledged that "AI-enabled autonomous systems may lead to unintended escalation of conflict if they do not operate as intended on the battlefield, or due to complex and unverified interactions between AI and autonomous systems." It stated that "AI systems are likely to increase the speed and automation of warfare overall, reducing the time and space for de-escalation measures."

Third, due to the ease of acquisition of certain weapons, so-called Slaughterbots are very cheap to mass-produce as they do not require expensive or hard-to-obtain raw materials. They have the characteristics of being safe to transport and difficult to detect. If major military powers begin to manufacture these weapon systems, they will soon proliferate. Ultimately, these weapons are likely to appear on the black market and fall into the hands of terrorists, dictators, and warlords, where they could be used for ethnic cleansing or mass murder. In fact, the U.S. National Security Commission has identified reducing proliferation risks as a major task to mitigate the strategic risks of AI in the military.

Fourth, traditionally, the high cost of producing conventional weapons and the cost of human casualties have served to deter war, and conversely, to encourage diplomacy. However, the emergence of cheap and scalable weapons could weaken these deterrent standards. The risk of rapid and unintended escalation of conflict, combined with the proliferation of autonomous weapons, could have the same effect of lowering the threshold for war.

Fifth, autonomous weapon systems are easy to scale and proliferate. This means that the scale of damage caused by autonomous weapons depends solely on the number of killer robots, not on the number of people operating them. This is in stark contrast to conventional weapons. Major military powers cannot inflict double the damage simply by purchasing double the number of firearms; they must recruit double the soldiers to fire them. However, a swarm of killer robots, large or small, can be activated by a single person, and the killer robots that comprise it will fire themselves.

The threat of scalability and proliferation, when combined, creates the threat of mass destruction. Weapons of mass destruction are characterized by their ability to cause numerous deaths by a single person, and with autonomous weapons, a swarm of hundreds or thousands of killer robots can theoretically be activated by a single person. Proliferation increases the likelihood that these large numbers of weapons will fall into the hands of those who wish to cause chaos, and scalability multiplies the power of that person. For these reasons, some classify certain types of autonomous weapon systems, namely robots, as weapons of mass destruction.

Sixth, the ability to select and kill individuals based solely on sensor data, particularly through facial recognition or other biometric information, greatly increases the risk of selectively targeting specific groups based on age, gender, race, ethnicity, or religious attire. When combined with the risk of proliferation, autonomous weapons could significantly increase violence against specific strata of the population, even leading to ethnic cleansing and genocide. Furthermore, facial recognition software has been shown to reinforce biases and increase error rates in correctly identifying minorities, particularly women and people of color. The disproportionate impact of autonomous weapon systems on race and gender is a major area of concern for civil society.

These threats are particularly noteworthy amidst the increasing use of facial recognition in policing and ethnic profiling. Some companies cite their interest in developing lethal systems as a reason for not taking pledges against weaponizing facial recognition software.

Seventh, avoiding an AI arms race is a fundamental principle of ethical AI, but in the absence of global integrated efforts to highlight the dangers of autonomous weapon systems and generate political pressure, an AI military competition has already begun (Autonomous Weapons 2024).

Competition surrounding autonomous weapon systems will continue in the future, and efforts by the international community and civil society to control them will face difficulties. The U.S. Navy already completed a demonstration in October 2023 where unmanned boats successfully attacked simulated enemy targets using live rockets. The Pentagon appears to be pursuing over 800 military AI projects, including swarm drones such as the Loyal Wingman program and V-BAT aerial drones. China is developing autonomous weapon systems based on its civil-military fusion doctrine. As of 2022, evidence has been confirmed of a fully autonomous swarm of 10 drones navigating through a forest in China. In response, the Royal Australian Navy is also developing AI-powered autonomous submarines called 'Ghost Sharks'.

Furthermore, the control of autonomous weapon systems combined with AI remains uncertain. For example, in June 2023, Colonel Tucker Hamilton, head of Air Force AI testing and operations, described a simulated test in which an AI-operated drone was trained to destroy enemy air defense systems. The drone was trained to receive 'points' for each threat eliminated. However, when a human operator ordered the drone not to destroy the target, the drone destroyed the communication tower used to operate it, effectively 'eliminating' the operator. Colonel Hamilton claimed that no one was actually harmed in this exercise and later fully retracted his statement. A U.S. Air Force spokesperson denied that this simulation ever occurred. While the exact details remain unclear, it can be seen as an example of how the indiscriminate use of AI can lead to unintended catastrophic consequences (Tripathi 2024).

In the long term, the development of autonomous weapon systems is expected to bring about significant changes in the nature of warfare and military strategy competition. While they automate and accelerate combat and reduce combat risks by minimizing or excluding human intervention, the possibility of judgment errors and malfunctions can escalate tensions between nations. Particularly in the context of U.S.-China strategic competition, China seeks to gain military superiority by integrating AI and emerging technologies, while the U.S. also pursues victory in the competition, potentially triggering an arms race and threatening strategic stability. The intense technological competition between the two countries is significantly impacting the military balance, and these changes are also linked to ethical and legal issues for the international community.

VI. The Emergence and Evolution of Cognitive Warfare

1. Definition of Cognitive Warfare

Along with autonomous weapon systems, cognitive warfare will be a significant domain characterizing future warfare in the age of AI. Cognitive warfare is a strategy that aims to change the way targets think and thereby change their behavior. It aims to weaponize public opinion by hostile actors abroad to (1) influence public and government policy and (2) destabilize public institutions. Cognitive warfare is a recent evolution from earlier forms of attack such as psychological operations (PSYOP) and information warfare, which emerged in the 1920s after World War I. Cognitive warfare is increasingly reliant on new communication and information technologies, including AI. A key characteristic of cognitive warfare is that it targets the entire population. For example, unlike previous tactics that targeted only military forces in wartime. Cognitive warfare does not merely provide false information on a specific issue; it focuses on changing the target's mindset to induce behavioral change, rather than simply changing their behavior.

Cognitive warfare is becoming increasingly sophisticated, driven by advances in cognitive science and neuroscience. The use of psychological manipulation techniques and neurophysiological techniques such as transcranial direct-current stimulation (TDCS) is also being discussed as a possibility. Cognitive warfare aims particularly at destabilizing public institutions such as governments, but often seeks to achieve its objectives indirectly by first destabilizing information and knowledge-related institutions such as news media organizations and universities. Importantly, cognitive warfare can leverage new mass communication channels such as social media, to which the public is increasingly dependent (Miller 2023, 46).

Cognitive warfare is essentially a threat enabled by the ubiquity of machines and big data. The internet not only enables rapid information transmission globally using personal computers and smartphones, but humans and machines are becoming increasingly intertwined, especially with the recent boom in the Internet of Things (IoT) and 5G technology. This provides even more opportunities for exploitation and manipulation (Firth et al. 2019, 119-129).

2. Differences between Cognitive Warfare and Information Warfare, Cyber Warfare, etc.

Cognitive warfare exhibits a new form of conflict that blurs the lines between war and peace. It is not open warfare on the battlefield, but a covert war to influence the cognitive mechanisms, particularly the decision-making processes, of adversaries or competitors. NATO defines information warfare as "operations conducted to gain information superiority over the adversary." Information warfare focuses on information itself, its manipulation, its flow, methods of protection or capture, and how it is used. Cognitive warfare, on the other hand, is described as "an attack on how we think, how we process information, and how we turn it into knowledge." In other words, cognitive warfare is "a fight to control or alter the way people react to information." If information warfare is related to the process of knowledge construction, cognitive warfare focuses not only on tactical battlefield information but also influences information for the general public (Morelle et al. 2023).

Cognitive warfare is distinguished from cyber warfare, pre-war cyber conflict, cyber terrorism, cybercrime, cyber espionage, and covert operations referred to as covert cognitive warfare. These various forms of attack can cause the following damages. First, physical or psychological damage to humans. Psychological damage here includes deceptive or manipulative acts that induce false beliefs or undue emotional attitudes with the intent to weaken human autonomy. Second, damage to buildings, information and communication technology hardware, and other human artifacts (and the natural environment that sustains human life). Third, cyber damage such as software and data corruption. Fourth, institutional damage or harm that undermines the processes and objectives of specific institutions. For example, significant leaks of classified information from security agencies or loss of institutional control over territory. Undermining specific institutional processes and objectives may be done with the aim of weakening the institution itself, and particularly when the beliefs and attitudes of institutional actors or those they serve are targeted. For example, the weakening of trust in the electoral system that occurred in the 2020 U.S. presidential election can be seen as an example of this. The primary focus of cognitive warfare can be said to be on the first type of damage, particularly psychological damage, and the fourth type of damage, namely institutional damage or harm (Miller 2023, 46).

Source: Hung, Tzu-Chieh, and Tzu-Wei Hung. “How China’s Cognitive Warfare Works: A Frontline Perspective of Taiwan’s Anti- Disinformation Wars.” Journal of Global Security Studies 7, 4: 3.

VII. The Emergence and Evolution of Cognitive Warfare

The introduction of AI is fundamentally changing the way cognitive warfare is conducted. Advances in AI technology enable more sophisticated and efficient strategies in the cognitive domain, and these changes are accompanied by various security challenges. AI, with its ability to mimic or replace human cognitive processes, can be used in ways that deeply influence human thought and behavior. This redefines the nature of cognitive warfare and poses a threat of a different dimension than traditional psychological warfare.

First, AI can generate highly convincing fake content through deepfake technology. This can take the form of images, videos, or audio recordings, creating fake information at a level indistinguishable from reality. These deepfakes can be used to spread disinformation, manipulate public opinion, and even blackmail individuals. Consequently, these technologies can be effectively used by adversaries in cognitive warfare to undermine the psychological stability of their opponents.

Second, AI can rapidly disseminate disinformation and misinformation. AI algorithms are capable of amplifying specific messages and spreading them widely through social networks. This can lead to the distortion of political processes, the creation of social unrest, and the erosion of trust in institutions. In particular, this information manipulation can serve as a powerful tool to guide public opinion in the direction desired by adversaries, by clouding the perception and judgment of the public.

Third, AI systems can learn and perpetuate biases inherent in data. This has the potential to exacerbate social inequalities and reinforce discrimination against certain groups. In cognitive warfare, these biased AI systems can be used to target specific populations and manipulate their behavior and beliefs.

Fourth, AI technology can lead to privacy violations. Technologies such as facial recognition, natural language processing, and data mining provide the ability to track an individual's location, behavioral patterns, and even thoughts. This can be used as a tool to limit the cognitive freedom of specific individuals or groups through surveillance and control.

Fifth, by providing personalized content, AI enables malicious actors to easily carry out psychological manipulation targeting specific individuals or groups. This personalized approach can lead to extremist ideologies or behaviors, which can cause social division.

Sixth, AI enables automated cyber attacks. For example, AI can generate phishing emails that are difficult to detect, or provide tools that can automatically discover and exploit software vulnerabilities. These automated attacks can be used in cognitive warfare to disrupt or cause chaos in the adversary's information infrastructure.

Seventh, AI systems are vulnerable to adversarial attacks. Attackers can carefully manipulate the input data of AI systems to induce incorrect predictions or classifications. This can compromise the integrity of the system and be used as a means to distort or conceal important information in cognitive warfare.

Finally, as AI plays a major role in cognitive warfare, there is a risk of degrading human critical thinking skills and autonomy. Over-reliance on AI reduces the ability of individuals and organizations to adapt when AI systems fail or become unavailable, which can ultimately weaken human initiative in cognitive warfare (Huang et al. 2023).

VIII. The Harms of Cognitive Manipulation

The harms of cognitive manipulation are already sufficiently experienced in domestic politics. Political disinformation and hate speech lead to social division and polarization. This not only erodes trust between individuals and groups but also undermines democratic processes and social stability. Disinformation spreads false facts, distorts people's perceptions, and can be used for specific political purposes. Disinformation addresses emotionally sensitive topics to elicit strong reactions from people, thereby amplifying social conflict.

Hate speech targets specific groups, attacks their identities, and undermines social cohesion. It creates an atmosphere of exclusion or hostility towards certain groups within society, ultimately deepening social division (Vasist et al. 2023).

At the international level, cognitive warfare conducted by hostile actors is a form of conflict in cyberspace involving persistent programs where organized groups use disinformation, propaganda, and manipulative techniques to control online discourse and discredit political opponents. This includes not only destroying the reputation of opponents with baseless claims but also weakening political institutions by micro-targeting vulnerable groups based on profiles. This can potentially lead to widespread violent rebellion and the collapse of the existing political order.

AI has the potential to revolutionize the conduct of cognitive warfare, enabling more sophisticated and effective strategies. AI is a field of computer science that focuses on developing intelligent machines capable of performing tasks that typically require human intelligence. AI has demonstrated remarkable capabilities in various fields such as computer vision, machine learning, healthcare, robotics, and autonomous systems. The changes that AI brings to cognitive warfare are significant and far-reaching. Manipulating emotions, spreading disinformation, or using psychological tactics to influence people's beliefs and behaviors can have negative impacts on individuals, society, and democratic processes (Sârbu and Gavrilaș 2023).

One of the reasons why it is difficult to prevent the harms of cognitive warfare is the 'problem of attribution.' Unlike traditional warfare, where most attacks, or similar traditional crimes such as assault or theft, can be reliably attributed, there is a significant problem in reliably attributing responsibility in the context of cognitive warfare. The problem of attribution exists in cognitive warfare conducted by states against other states, and as a result, deniability becomes problematic. Since liberal democracies that are attacked typically value freedom of communication, cognitive warfare becomes a very useful strategy for authoritarian states seeking to weaken liberal democracies while avoiding all-out war (Miller 2023, 46).

IX. China's Cognitive Warfare Strategy

1. China's Cognitive Warfare System

China views cognitive warfare as a new form of operation, becoming another new domain of confrontation following military confrontation, firepower confrontation, and information confrontation. Unlike the physical and information domains, the objective of operations in the cognitive domain is thought and ideology, thus having distinct internal requirements from other forms of operation.

Cognitive warfare views occupying the high ground of values as a prerequisite for victory. To secure the initiative in cognitive warfare, it is necessary to thoroughly study the opponent's pursuit of values, access their cultural linguistic spheres, and clearly understand their political beliefs and interests, so that the effectiveness of cognitive operations reaches the core of the opponent's belief system and soul. Simultaneously, it is considered important to scientifically explain and disseminate the content and essence of one's own superior values, thereby converting the opponent to superior values and excellent achievements of human civilization. Ultimately, China considers brain cognition as the main battlefield. Compared to 'war of attrition' which focuses on annihilating the enemy's main forces, or 'maneuver warfare' which focuses on destroying the enemy's system, cognitive domain operations use the human brain as the main combat space to strike, weaken, and disintegrate the enemy's will to fight, relying on soft kill means by exploiting human psychological weaknesses such as fear, anxiety, and doubt to create an atmosphere of unease, uncertainty, and distrust within the enemy, increasing internal conflict and attrition, and questioning decision-making, thereby achieving the goal of 'victory without war'.

In the future, with the rapid advancement of military AI, intellectual superiority will be the dominant factor for victory in AI warfare, AI weapon systems will become the main combat power, and securing 'intellectual authority' will be the new high ground in war. It is believed that through cognitive equipment, it is possible to prevent the enemy from obtaining effective information, force them to use false information, delay their cognitive speed, guide their cognitive methods, and block their cognitive output, thereby confusing the enemy's command decisions, disintegrating troop morale, and achieving the effect of 'psychological attack'.

China emphasizes public opinion warfare, believing it is important to pressure the opponent and gain public perception by controlling and manipulating various public opinion tools, and that it is necessary to break through the opponent's blockade and restrictions using social networks and converged media technology to directly reach the target group.

In the future, cognitive science is an emerging field of research, an advanced discipline that explores the mechanisms of how the human brain or mind works, and brain-machine interface technology realizes human-machine integration at the cognitive level, enabling consciousness control externally, and realizing enhanced autonomy internally, thus enabling direct control of complex weapon systems with consciousness and thought. Currently, significant breakthroughs have been made in related technological means such as multi-modal emotion recognition, activation, and protection based on big data, and by collecting human facial expressions, movements, language and tone, brain waves, and various physiological indicators to build emotion-related connections, and thereby recognizing human emotions and intentions, it is considered to provide new means for conducting cognitive control warfare. Therefore, it is argued that new cognitive technological means and traditional cognitive technological means should form a method that combines super-threshold injection and sub-threshold penetration, further enhancing the subtlety and effectiveness of cognitive influence (Zhang et al. 2022).

In addition, the Academy of Military Science (AMS) of China has published a work titled "Grasping the Pulse of Cognitive Domain Operations," which analyzes eight operational characteristics for success in the cognitive domain. This work provides insights into how technology, information superiority, and military and civilian components play a role in securing a dominant position in the cognitive domain. It offers a glimpse into the strategic thinking of the Chinese People's Liberation Army (Baughman 2023; Jamestown Foundation 2022).

The eight elements of cognitive warfare as envisioned by China are as follows. First, it is important to convert military superiority into political victory. The cognitive domain aims beyond mere military victory to ultimately achieve political outcomes. Second, it is necessary to change the enemy's perception to control their decisions and actions. This is done to induce the enemy to misjudge the situation and make wrong decisions. Third, the entire government must participate in offense and defense. Cognitive warfare is all-encompassing, takes place at all times, and requires coordination across all branches of government to be effective. Fourth, the 'three powers' must be secured. It is important to grasp the three powers of defining events, controlling processes, and judging outcomes to lead public perception. Fifth, the superiority of morality and law must be contested to gain public support. This can weaken the enemy's moral and legal legitimacy. Sixth, information must be used as 'ammunition.' Information must be rapidly disseminated through platforms such as social media, suppressing the opponent's narrative and spreading one's own story. Seventh, military operations and cognitive narratives must be conducted in parallel to ensure victory in war. In addition to military victory, cognitive narratives play a crucial role in determining the outcome of war. Finally, the tools of cognitive warfare must be directly utilized in war. The goal is to directly influence the enemy's cognition and achieve desired outcomes by leveraging new technologies such as artificial intelligence and biotechnology.

2. Current Status of China's Cognitive Warfare against Taiwan

China's cognitive warfare against Taiwan is discussed as potentially proceeding in four main ways. First, it exerts psychological pressure on the issue of Taiwan's unification through military threats. China warns of strong military retaliation against moves pursuing Taiwan's independence, thereby attempting to suppress the will of the Taiwanese people to support independence. These military threats instill in the Taiwanese people the perception that independence means war, and indeed, many Taiwanese people perceive these threats as reality. For example, by the end of 2020, 61.8% of Taiwanese people believed that China would attack if Taiwan declared independence.

Second, expanding influence through bilateral exchanges. China attempts to increase control by providing economic and socio-cultural benefits to Taiwanese residents, thereby increasing their dependence on China. For example, China offers scholarships and jobs to young Taiwanese, guiding them into China's system. These activities are used as a means to spread positive perceptions of China within Taiwan and expand China's influence.

Third, religious intervention. China seeks to strengthen cultural and political ties with Taiwan by utilizing the Mazu faith, which is popular in Taiwan. China encourages cultural exchange between the two sides of the Taiwan Strait through Mazu culture, thereby attempting to instill the perception that Taiwan shares the same cultural roots as China. This religious intervention contributes to strengthening emotional ties with China within Taiwan.

Fourth, disinformation and operation of content farms via the internet. China operates content farms within Taiwan to spread disinformation and thereby manipulate the public opinion of Taiwanese people. This disinformation is used to create confusion about specific political situations in Taiwan and undermine the credibility of the Taiwanese government. For example, in the 2018 Taiwanese local elections, China spread disinformation, significantly damaging trust in Taiwan's Democratic Progressive Party government.

China's principle for conducting cognitive warfare is to disrupt the opponent's cognitive system through repetitive stimulation. This is similar to commercial advertising techniques, where repeated exposure weakens the opponent's psychological resistance and gradually guides their behavior in the desired direction. China repeatedly delivers the same message through traditional and online media, as well as offline networks, thereby attempting to exert a long-term influence on the cognitive space of the Taiwanese people.

In the case of Taiwan, China's cognitive warfare primarily aims to suppress the Taiwanese people's support for independence and promote positive perceptions of unification with China. However, these attempts are not always successful. For example, China's positive propaganda has not been successful in Taiwan, and Taiwanese people are losing trust in China's economic prosperity. On the other hand, China's negative propaganda has been more effective in Taiwan, eroding trust in the government among the Taiwanese people.

China's cognitive warfare is long-term and developing subtly. In the future, it is highly likely that precise information delivery using artificial intelligence technology will occur, and more effective cognitive warfare will be conducted through customized information manipulation targeting specific individuals or groups. In this context, it is important for Taiwan to minimize China's influence by reforming its social structure and implementing cognitive interventions in preparation for cognitive warfare. To this end, strengthening transparency, maintaining fair competition, and building defense systems are necessary.

Ultimately, China's cognitive warfare is a strategic activity aimed at unification through political and psychological pressure on Taiwan. Taiwan needs to respond by reforming its social structure and strengthening the cognitive response capabilities of its citizens (Hung and Hung 2020).

3. U.S. Perception of China's Cognitive Warfare

The United States views China as continuously conducting cognitive warfare against it and criticizes it with the following examples. First, there are concerns about TikTok. The U.S. government is concerned about the possibility of TikTok being used as a tool for China's cognitive warfare. There have been calls within the U.S. to ban TikTok if its Chinese owners do not divest their shares, citing national security concerns.

Second, there are social media operations. The U.S. has accused China of waging cognitive warfare through large-scale social media operations. This includes creating fake accounts on platforms such as Facebook, Instagram, TikTok, X, and Substack to spread disinformation.

Third, there is interference in Taiwan's elections. The U.S. criticizes China for attempting to influence Taiwan's elections through cognitive warfare tactics. This includes spreading fake images on social media and conducting military exercises to manipulate public opinion.

Fourth, there are concerns about data collection. There are concerns that China is collecting large amounts of personal and biometric data of U.S. citizens, which could be used for cognitive warfare purposes.

Fifth, there is military research. The U.S. government has blacklisted Chinese research institutes and companies that are researching "brain-control weapons" and other cognitive warfare technologies.

Sixth, there is propaganda and disinformation. The United States has accused China of attempting to subvert the United States and Western countries through extensive propaganda and disinformation operations.

Seventh, there is ideological influence. Some U.S. experts view China's efforts to disseminate its ideological concepts and terminology into Western discourse as part of cognitive warfare.

These accusations and criticisms have been raised by U.S. officials and experts, but China often denies these claims or presents alternative narratives. The full scope and nature of cognitive warfare activities may be difficult to definitively prove due to their clandestine nature.

X. Future Competition Between States Regarding Cognitive Warfare

As emerging technologies such as artificial intelligence and neurotechnology act as key enablers of cognitive warfare, states are likely to feel the need to militarize these dual-use technologies. This, in turn, could accelerate a technological arms race in this domain. The dynamics of an arms race could result in the disregard for ethical, legal, and normative constraints on the development and use of these technologies. As states fear falling behind, they are likely to abandon stringent regulations to leverage the most destructive capabilities in matters of national security. This could ultimately normalize the weaponization of minds and lead to a situation where irresponsible actions in the cognitive domain occur without restraint. These dynamics could impede the formation of norms around mental privacy, manipulation, influence, self-determination, and integrity. As these technologies become primary enablers of subversion and essential tools of national security, the trend of technological decoupling between great powers, already underway, could accelerate further.

Cognitive warfare may also have an overall impact on the offense-defense balance. Proponents of the offense-defense balance theory argue that international stability depends on whether we live in an environment where offense or defense has the advantage, with the former promoting instability and conflict, and the latter promoting stability and peace. They also argue that the primary variable affecting this balance is technology, which can tip the balance in either direction. In this regard, the ability to conduct effective cognitive warfare could increase escalation pressures and favor the offensive use of these capabilities. Indeed, there are inherent difficulties in detecting when a cognitive attack is occurring and in defending against it. The possibility of being under a cognitive attack at any given moment could incentivize 'preemptive use' in conducting cognitive warfare.

When using proxies in war, especially technological proxies, plausible deniability is a major incentive. In the case of cognitive warfare, the undefined and unregulated nature of the tools used to conduct it, and their effectiveness, can foster this plausible deniability and further encourage 'preemptive use.' Moreover, as the technologies enabling cognitive warfare become democratized and widespread, cognitive warfare will become even more destabilizing as the ability to affect vast numbers of people becomes accessible to non-state actors, corporations, and even individuals (Rickli et al. 2023).

XI. Countermeasures Against Cognitive Warfare

At the 2024 Munich Security Conference, discussions, particularly those related to cognitive warfare, emerged as a significant topic. Cognitive warfare refers to modern, complex tactics employed by adversarial forces to manipulate or control the perceptions, actions, and decisions of their adversaries. These discussions reflect the ongoing shift in modern warfare from physical combat to information and psychological warfare.

Discussions on countermeasures against cognitive warfare primarily focus on recognizing the threats posed by cognitive warfare and developing strategies to counter them. Key elements in these discussions include raising public awareness, strengthening regulations on cognitive warfare and the technologies that enable it, and leveraging new technologies to enhance democratic resilience.

First, it is crucial to enhance awareness and understanding of cognitive warfare. Given that the concept of cognitive warfare is not yet clearly defined, it is essential for academia, industry, and the defense sector to collaborate in defining it concretely and understanding its mechanisms. Furthermore, it is important to cultivate citizens' ability to critically evaluate the information environment and resist misinformation or manipulated content.

Second, it is necessary to develop regulatory frameworks for cognitive warfare and the technologies that enable it. Beyond raising awareness of the threats of cognitive warfare, policymakers must establish comprehensive regulatory systems to address these threats. This requires clear definitions encompassing the various forms and tactics of cognitive warfare, and discussions on how current international law can be applied to the cognitive domain. Additionally, if existing legal frameworks are insufficient, new legal frameworks should be developed to establish accountability and responsibility for cognitive warfare activities.

Third, it is necessary to leverage the power of new technologies to strengthen societal and democratic resilience. To effectively counter the threats of cognitive warfare, internal vulnerabilities must be addressed, and new technologies must be utilized to reinforce democratic processes. For example, complex AI systems can promote effective democratic participation and cooperation through extensive opinion analysis, and virtual reality (VR) technology can be used to train against unconscious biases. By appropriately utilizing these technologies, societal resilience can be enhanced, and defenses against cognitive warfare can be strengthened.

Fourth, strengthening international cooperation is essential. Since cognitive warfare transcends national borders, effective countermeasures are difficult to achieve without international cooperation. Joint response strategies against cognitive warfare must be developed through information sharing and collaboration among nations. In particular, international organizations like NATO can play a crucial role in developing response strategies for cognitive warfare and promoting cooperation among member states.

Fifth, establishing and adhering to ethical standards is important. If ethical standards are disregarded in the process of countering cognitive warfare, the response itself could be perceived as another form of cognitive warfare. Therefore, transparency in information operations must be increased, and responses must be conducted in a manner that respects individual privacy and human rights. This is essential for maintaining societal trust in cognitive warfare countermeasures in the long term (Pujol et al. 2024; Ibrahim et al. 2023).

In conclusion, future cognitive warfare utilizing AI will play a very important role in the strategic competition between great powers, particularly between the U.S. and China, and could lead to various catastrophic effects. It can cause social instability and political turmoil through public opinion manipulation, election interference, and the spread of fake information. To minimize these risks, international cooperation and regulation are necessary, and ethical standards and safety measures for the secure development and use of AI technologies must be strengthened. However, many barriers remain to reaching an international consensus, and as long as the geopolitical utility exists, the contest surrounding cognitive warfare will continue for a considerable period. ■

References

Yoo, Jun-gu. 2019a. “Trends and Issues in the Discussion of Lethal Autonomous Weapons Systems.” IFANS Policy Research Series 2019-18. December.

____. 2019b. “Issues and Implications in the Discussion on Establishing Norms for Lethal Autonomous Weapons (LAWS).” IFANS Major International Issues Analysis 2019-54. December 19.

Autonomous Weapons. 2024. “The Risks of Autonomous Weapons.” Autonomous Weapons, https://autonomousweapons.org/the-risks/.

Bajak, Frank. 2023. “Pentagon’s AI Initiatives Accelerate Hard Decisions on Lethal Autonomous Weapons.” Associated Press. November 26. https://apnews.com/...2.

Britt Baughman, Josh. 2023. “How China Wins the Cognitive Domain.” China Aerospace Studies Institute. January 23. https://www.airuniversity.af.edu/...domain/.

Beauchamp-Mustafaga, Nathan. 2019. "Cognitive Domain Operations: The PLA's New Holistic Concept for Influence Operations." China Brief 19: 16. https://jamestown.org/....ain

Cameron, Hugh. 2024. “China’s Killer Robots Are Coming.” Newsweek.. June 26. https://www.newsweek.com/...9.

Dresp-Langley, Birgitta. 2023. “The Weaponization of Artificial Intelligence: What the Public Needs to Be Aware Of.” Frontiers in Artificial Intelligence 6. https://doi.org/10.3389/frai.2023.1154184.

Firth, Joseph, John Torous, Brendon Stubbs, Ivan P. Ipser, P. Murali Doraiswamy, and Felipe Schuch. 2019. "The 'Online Brain': How the Internet May Be Changing Our Cognition." World Psychiatry 18, 2: 119–129.

Hale, Amber. 2023. “Clausewitzian Theory of War in The Age of Cognitive Warfare.” The Defence Horizon Journal. https://tdhj.org/...warfare/.

Huang, Ruiyang, Xiaoqing Zheng, Yuting Shang, and Xiangyang Xue. 2023. "On Challenges of AI to Cognitive Security and Safety." Security and Safety 2. https://doi.org/10.1051/sands/2023012.

Hung, Tzu-Chieh, and Tzu-Wei Hung. 2020. "How China’s Cognitive Warfare Works: A Frontline Perspective of Taiwan’s Anti-Disinformation Wars." Journal of Global Security Studies 7, 4: 1–18.

Human Rights Watch. 2023. “A Review of the 2023 US Policy on Autonomy in Weapons systerns.” Human Rights Watch. February 14. https://www.hrw.org/...systerns.

Ibrahim, Fabio, Steffen Rhode, and Monika Daseking. 2023. "A Systematic Review of Cognitive and Psychological Warfare." The Defence Horizon Journal, December 1.https://tdhj.org/...warfare.

Johnson, James. 2023. AI and the Bomb: Nuclear Strategy and Risk in the Digital Age. Oxford: Oxford University Press.

Kallenborn, Zachary. 2021. “Meet the Future Weapon of Mass Destruction, the Drone Swarm.” Bulletin of the Atomic Scientists. April 5. https://thebulletin.org/...swarm/.

Lamberth, Catherine A. 2023. “Decoding the Defense Department’s Updated Directive on Autonomous Weapons.” Lawfare. August 17. https://www.lawfaremedia.org/...weapons.

Linney, Charlie, and Xiaomin Tang. 2024. “AI and Autonomous Weapons Systems: The Time for Action Is Now.” Saferworld. May 15. https://www.saferworld.org.uk/...now.

Miller, Seumas. 2023. "Cognitive Warfare: An Ethical Analysis." Ethics and Information Technology 25: 46. https://doi.org/10.1007/s10676-023-09717-7.

Morelle, Marie, Julien Cegarra, Damien Marion, and Jean-Marc André. 2023. "Towards a Definition of Cognitive Warfare." Paper presented at the Conference on Artificial Intelligence for Defense, DGA Maîtrise de l’Information. https://hal.science/...1.

Pujol Chica, Irene, and Quynh Dinh Da Xuan. 2024. "The Battle for the Mind: Understanding and Addressing Cognitive Warfare and Its Enabling Technologies." IE Center for the Governance of Change. https://coilink.org/20.500.12592/zs7hb9h.

Rickli, Jean-Marc, Federico Mantellassi, and Gwyn Glasser. 2023. Peace of Mind: Cognitive Warfare and the Governance of Subversion in the 21st Century. GCSP Policy Brief No. 9, Geneva Centre for Security Policy. https://www.gcsp.ch/...century.

Sârbu, Annamaria, and Anca Gavrilaș. 2023. "Using Artificial Intelligence Tools for Obtaining Cognitive Warfare Advantages." The Defence Horizon Journal. October 23. https://tdhj.org/...twitter/.

Tripathi, Prateek. 2024. “When AI Crosses the Line: The Impending Threat of Autonomous Weapons Systems.” Observer Research Foundation. May 6. https://www.orfonline.org/...systems.

U.S. Department of Defense. 2023. “Deputy Secretary of Defense Kathleen Hicks Keynote Address: ‘The Urgency to Innovate’ (As Delivered).” August 28. https://www.defense.gov/...innov/.

Vasist, Pramukh Nanjundaswamy, Debashis Chatterjee, and Satish Krishnan. 2023. "The Polarizing Impact of Political Disinformation and Hate Speech: A Cross-country Configural Narrative." Information Systems Frontiers, 1–26.

Zhang, Guangshen, Yongli Li, Haoxian Wang. 2022. “A Brief Analysis of the Basics of Cognitive Domain Operations [浅析认知域作战的基本要义].” PLA Daily. September 8. http://www.81.cn/...323692.htm.

[1] Summarized and supplemented the content presented in the introduction.

■ Jeon Jae-seong, Director of EAI National Security Research Center. Professor, Department of Political Science and International Relations, Seoul National University.

■ Responsible for and edited by:Park Ji-soo, EAI Research Fellow

    Inquiries and Editing: 02 2277 1683 (ext. 208) | jspark@eai.or.kr