[Visible Commentary] The Era of AI War 2.0 Triggered by DeepSeek: Our Response Strategy

YouTube Link : https://www.youtube.com/watch?v=XEw7Yypb9uY

Video Script

Welcome. This is Sungwoo from Naver Cloud. I believe artificial intelligence has already become a significant agenda of our time, along with deep tech. Since last year, we have seen rapid changes from this perspective, and I would like to discuss these aspects. I have had many discussions with Professor Sangbae Kim of Seoul National University and others. I intend to share my thoughts while discussing the impact of artificial intelligence on international relations. Before diving in, I think it is important to highlight the major paradigm shift occurring in AI technology. I believe the professors and experts present are likely using generative AI such as GPT. Models like GPT-4 or Anthropic Claude 3.5 Sonnet and Claude 2.0, in particular, are diligent in their memorization-based training. This involves significantly increasing the number of parameters, or model size, to over 10 trillion tokens, which can be thought of as words. They learn extensively by memorizing vast amounts of internet data, often through tasks like masking and predicting words. This is called pre-training, and after fine-tuning with data created by experts, they become tools like GPT that you are using. They are knowledge-based, understand language very well, grasp context, and possess basic reasoning abilities. We have defined this as the first generation of knowledge-based AI.

The emergence of second-generation, reasoning-based AI.

In Korea, we have Naver's HyperCLOVA X and the EXAONE series released by LG AI Research. However, with the release of EXAONE last September, we entered a new phase, a new era, which I refer to as the second generation. I call it reasoning-based AI, not knowledge-based AI, because it goes beyond simply writing well and possessing extensive knowledge; it is an AI capable of independent logical reasoning. It can independently unravel logic.

In Korea, we have Naver's HyperCLOVA X and the EXAONE series released by LG AI Research. However, with the release of EXAONE last September, we entered a new phase, a new era, which I refer to as the second generation. I call it reasoning-based AI, not knowledge-based AI, because it goes beyond simply writing well and possessing extensive knowledge; it is an AI capable of independent logical reasoning. It can independently unravel logic.

They possess a vast number of GPUs. As early as 2020, they had over 10,000 A100 GPUs, and according to various reports, their total GPU holdings, including H100, H800, A100, and H20, are estimated to be between 50,000 and 60,000. This has led the U.S. government to investigate Nvidia, questioning whether export controls are being effectively enforced. Deepseek did not emerge suddenly. Their V1 was first released in January last year, followed by V2 in May and V3 in December, with remarkable and rapid technological advancements. Using their powerful knowledge-based AI, Deepseek V3, and the aforementioned reinforcement learning and extensive logical flow data (COT data), they announced Deepseek R1.

Consequently, their chatbot service, similar to ChatGPT, was deployed on Android and iOS apps worldwide, even reaching the number one spot in the U.S. App Store. Some attribute this to engineering prowess, but it is also the result of significant GPU investment and the contributions of 100 to 200 top-tier AI engineers who have introduced various novel techniques. Over the past year, they have developed numerous algorithms that, while not entirely unprecedented, build upon existing methods and represent significant improvements. These algorithms are designed to reduce overall costs. When V1 was released, the impression was that it was simply another open-source AI company from China, nothing more, nothing less. At the time, it offered no advantages over Alibaba Cloud's Qwen model. However, starting with the V2 model, innovative

China's AI Technology Development and DeepSeek

aspects began to emerge. Key techniques used in Deepseek V3 include the 'Multi-Head Architecture.' In simple terms, unlike traditional transformers, this approach does not utilize the entire model for every input. Instead, it analyzes the query and directs it to specialized expert modules within the model, using only a relevant portion for computation. For instance, with V2, which has 26 billion parameters, if a query is related to law, it utilizes only the parameters relevant to law, approximately 21 billion, for calculation. Training with this method significantly reduces training costs compared to a single model with the same 26 billion parameters. However, operational costs increase. This is because, for each incoming query, calculations are performed using only a subset of 21 billion parameters. Compared to models that exclusively use these 21 billion parameters, operational costs can be more than six times higher. This is because the entire large model must reside in the hardware's memory.

Therefore, extensive engineering optimizations are employed to mitigate these operational costs. The basic architecture of V3 is largely similar, with the model size approximately tripled. It also utilizes 'Mixture of Experts' models. Additionally, new training techniques are incorporated, and the amount of training data is significantly increased to 15 trillion tokens. The global impact of Deepseek V3 was truly demonstrated by Table 1. That is where the revelation came from.

It can be produced for 8 billion won. While this may seem intentional, it is somewhat misleading. Let's look at the table. It is labeled 'Training Cost' and breaks down into 'Pre-training,' 'Context Extension,' and 'Post-training.' These can be understood as the training phases. It specifies the number of hours each H800 GPU is used for each phase. In total, it indicates approximately 2.788 million GPU hours were used. This implies the use of a supercomputer where 248 H800 GPUs are interconnected in a powerful network. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the development cost.

DeepSeek V3 and MoE Architecture

It is highly unlikely that the training was successful on the first attempt. Furthermore, the costs associated with the numerous trials and errors to find the successful recipe, labor costs, and data creation costs are not included. This is even mentioned in the paper. Considering all the GPUs and other resources, the estimated investment cost could range from hundreds of billions to 1-2 trillion won. They then developed R1, a model with exceptional reasoning capabilities, based on the V3 model. Both in the development of the V3 model and the R1 model, it appears highly probable that data was extracted from OpenAI's models and used for training. Although they deny this, several pieces of evidence have emerged. I am unsure if I have included the relevant slides, but let's proceed.

For example, if you input a query and then instruct it to do something unusual, it might respond with something like, 'The policy you have requested violates OpenAI's privacy policy.' This clearly indicates that the data was obtained and used. Regardless, the technology's impact stems from its development. The fact that it was created in China, not the US or China, is significant. Many AI companies, including ourselves, have been curious about how to create an AI capable of sophisticated reasoning, similar to OpenAI's GPT-4. While we had some estimations, conducting experiments to precisely replicate this was not feasible due to the immense cost of each new attempt. However, they disclosed details in their technical report, such as 'SFT-GPT,' implying that only this specific method is necessary and other approaches are not. They revealed approximately 80% of the detailed methodology.

This allowed numerous companies globally, not just in the US and China but also in France, Korea, Japan, and elsewhere, to replicate the technology. This has had a profound effect. The significant cost reduction was also achieved because, unlike OpenAI's GPT-4 development, which required human experts to create extensive COT data—a time-consuming and costly process—Chinese developers could generate data using R1, drastically reducing costs. Consequently, with a relatively modest investment, not the astronomical sums seen elsewhere, it became possible to develop AI with top-tier reasoning capabilities, comparable to what the US has achieved. This is not unique to Deepseek; similar capabilities are emerging from other companies.

Technology Disclosure and Cost Reduction Effects

Companies like Minimax, Moonshot, Deel, and Alibaba Cloud, among others, have begun releasing AI models with comparable performance. This trend is not limited to China; similar advancements are appearing in the US as well. OpenAI's GPT-4 is no longer an exclusive asset. The race has begun for second-generation models. Therefore, the claim that it can be developed for 8 billion won requires careful consideration. While it might be partially true, it is a misleading simplification. The table indicates 'Training Cost,' encompassing 'Pre-training,' 'Context Extension,' and 'Post-training.' It details the GPU hours required, summing up to approximately 2.788 million hours using H800 GPUs. This implies the use of a supercomputer network of 248 H800 GPUs. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the overall development cost.

The actual development cost, including the numerous trial-and-error attempts to achieve this success, labor, and data creation, is not factored in. The paper itself acknowledges this. Considering all the GPUs and other resources, the estimated investment could range from hundreds of billions to 1-2 trillion won. Furthermore, they developed R1, a model with exceptional reasoning capabilities, based on V3. In the development of both V3 and R1, it appears highly probable that data was extracted from OpenAI's models. Despite their denials, several pieces of evidence suggest this. I am unsure if I have included the relevant slides, but let's proceed. For instance, if you input a query and then instruct it to do something unusual, it might respond with something like, 'The policy you have requested violates OpenAI's privacy policy.' This clearly indicates that the data was obtained and used. The impact of this technology lies in its creation. The fact that it was developed in China is significant. Many AI companies have been curious about how to create an AI capable of sophisticated reasoning, similar to OpenAI's GPT-4. While estimations existed, conducting experiments was not feasible due to the immense cost of each new attempt. However, they disclosed details in their technical report, such as 'SFT-GPT,' implying that only this specific method is necessary and other approaches are not. They revealed approximately 80% of the detailed methodology. This allowed numerous companies globally to replicate the technology. The significant cost reduction was also achieved because, unlike OpenAI's GPT-4 development, which required human experts to create extensive COT data—a time-consuming and costly process—Chinese developers could generate data using R1, drastically reducing costs. Consequently, with a relatively modest investment, it became possible to develop AI with top-tier reasoning capabilities, comparable to what the US has achieved. This is not unique to Deepseek; similar capabilities are emerging from other companies.

Intensifying AI Competition and China's Strategy

Companies like Minimax, Moonshot, Deel, and Alibaba Cloud, among others, have begun releasing AI models with comparable performance. This trend is not limited to China; similar advancements are appearing in the US as well. OpenAI's GPT-4 is no longer an exclusive asset. The race has begun for second-generation models. Therefore, the claim that it can be developed for 8 billion won requires careful consideration. While it might be partially true, it is a misleading simplification. The table indicates 'Training Cost,' encompassing 'Pre-training,' 'Context Extension,' and 'Post-training.' It details the GPU hours required, summing up to approximately 2.788 million hours using H800 GPUs. This implies the use of a supercomputer network of 248 H800 GPUs. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the overall development cost.

The actual development cost, including the numerous trial-and-error attempts to achieve this success, labor, and data creation, is not factored in. The paper itself acknowledges this. Considering all the GPUs and other resources, the estimated investment could range from hundreds of billions to 1-2 trillion won. Furthermore, they developed R1, a model with exceptional reasoning capabilities, based on V3. In the development of both V3 and R1, it appears highly probable that data was extracted from OpenAI's models. Despite their denials, several pieces of evidence suggest this. I am unsure if I have included the relevant slides, but let's proceed. For instance, if you input a query and then instruct it to do something unusual, it might respond with something like, 'The policy you have requested violates OpenAI's privacy policy.' This clearly indicates that the data was obtained and used. The impact of this technology lies in its creation. The fact that it was developed in China is significant. Many AI companies have been curious about how to create an AI capable of sophisticated reasoning, similar to OpenAI's GPT-4. While estimations existed, conducting experiments was not feasible due to the immense cost of each new attempt. However, they disclosed details in their technical report, such as 'SFT-GPT,' implying that only this specific method is necessary and other approaches are not. They revealed approximately 80% of the detailed methodology. This allowed numerous companies globally to replicate the technology. The significant cost reduction was also achieved because, unlike OpenAI's GPT-4 development, which required human experts to create extensive COT data—a time-consuming and costly process—Chinese developers could generate data using R1, drastically reducing costs. Consequently, with a relatively modest investment, it became possible to develop AI with top-tier reasoning capabilities, comparable to what the US has achieved. This is not unique to Deepseek; similar capabilities are emerging from other companies.

Companies like Minimax, Moonshot, Deel, and Alibaba Cloud, among others, have begun releasing AI models with comparable performance. This trend is not limited to China; similar advancements are appearing in the US as well. OpenAI's GPT-4 is no longer an exclusive asset. The race has begun for second-generation models. Therefore, the claim that it can be developed for 8 billion won requires careful consideration. While it might be partially true, it is a misleading simplification. The table indicates 'Training Cost,' encompassing 'Pre-training,' 'Context Extension,' and 'Post-training.' It details the GPU hours required, summing up to approximately 2.788 million hours using H800 GPUs. This implies the use of a supercomputer network of 248 H800 GPUs. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the overall development cost.

AI Technology Hegemony Competition and National Stances

The actual development cost, including the numerous trial-and-error attempts to achieve this success, labor, and data creation, is not factored in. The paper itself acknowledges this. Considering all the GPUs and other resources, the estimated investment could range from hundreds of billions to 1-2 trillion won. Furthermore, they developed R1, a model with exceptional reasoning capabilities, based on V3. In the development of both V3 and R1, it appears highly probable that data was extracted from OpenAI's models. Despite their denials, several pieces of evidence suggest this. I am unsure if I have included the relevant slides, but let's proceed. For instance, if you input a query and then instruct it to do something unusual, it might respond with something like, 'The policy you have requested violates OpenAI's privacy policy.' This clearly indicates that the data was obtained and used. The impact of this technology lies in its creation. The fact that it was developed in China is significant. Many AI companies have been curious about how to create an AI capable of sophisticated reasoning, similar to OpenAI's GPT-4. While estimations existed, conducting experiments was not feasible due to the immense cost of each new attempt. However, they disclosed details in their technical report, such as 'SFT-GPT,' implying that only this specific method is necessary and other approaches are not. They revealed approximately 80% of the detailed methodology. This allowed numerous companies globally to replicate the technology. The significant cost reduction was also achieved because, unlike OpenAI's GPT-4 development, which required human experts to create extensive COT data—a time-consuming and costly process—Chinese developers could generate data using R1, drastically reducing costs. Consequently, with a relatively modest investment, it became possible to develop AI with top-tier reasoning capabilities, comparable to what the US has achieved. This is not unique to Deepseek; similar capabilities are emerging from other companies.

Companies like Minimax, Moonshot, Deel, and Alibaba Cloud, among others, have begun releasing AI models with comparable performance. This trend is not limited to China; similar advancements are appearing in the US as well. OpenAI's GPT-4 is no longer an exclusive asset. The race has begun for second-generation models. Therefore, the claim that it can be developed for 8 billion won requires careful consideration. While it might be partially true, it is a misleading simplification. The table indicates 'Training Cost,' encompassing 'Pre-training,' 'Context Extension,' and 'Post-training.' It details the GPU hours required, summing up to approximately 2.788 million hours using H800 GPUs. This implies the use of a supercomputer network of 248 H800 GPUs. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the overall development cost.

AI Regulation and National Investment Competition

The actual development cost, including the numerous trial-and-error attempts to achieve this success, labor, and data creation, is not factored in. The paper itself acknowledges this. Considering all the GPUs and other resources, the estimated investment could range from hundreds of billions to 1-2 trillion won. Furthermore, they developed R1, a model with exceptional reasoning capabilities, based on V3. In the development of both V3 and R1, it appears highly probable that data was extracted from OpenAI's models. Despite their denials, several pieces of evidence suggest this. I am unsure if I have included the relevant slides, but let's proceed. For instance, if you input a query and then instruct it to do something unusual, it might respond with something like, 'The policy you have requested violates OpenAI's privacy policy.' This clearly indicates that the data was obtained and used. The impact of this technology lies in its creation. The fact that it was developed in China is significant. Many AI companies have been curious about how to create an AI capable of sophisticated reasoning, similar to OpenAI's GPT-4. While estimations existed, conducting experiments was not feasible due to the immense cost of each new attempt. However, they disclosed details in their technical report, such as 'SFT-GPT,' implying that only this specific method is necessary and other approaches are not. They revealed approximately 80% of the detailed methodology. This allowed numerous companies globally to replicate the technology. The significant cost reduction was also achieved because, unlike OpenAI's GPT-4 development, which required human experts to create extensive COT data—a time-consuming and costly process—Chinese developers could generate data using R1, drastically reducing costs. Consequently, with a relatively modest investment, it became possible to develop AI with top-tier reasoning capabilities, comparable to what the US has achieved. This is not unique to Deepseek; similar capabilities are emerging from other companies.

Companies like Minimax, Moonshot, Deel, and Alibaba Cloud, among others, have begun releasing AI models with comparable performance. This trend is not limited to China; similar advancements are appearing in the US as well. OpenAI's GPT-4 is no longer an exclusive asset. The race has begun for second-generation models. Therefore, the claim that it can be developed for 8 billion won requires careful consideration. While it might be partially true, it is a misleading simplification. The table indicates 'Training Cost,' encompassing 'Pre-training,' 'Context Extension,' and 'Post-training.' It details the GPU hours required, summing up to approximately 2.788 million hours using H800 GPUs. This implies the use of a supercomputer network of 248 H800 GPUs. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the overall development cost.

Strengthening South Korea's AI Competitiveness and Alliance Strategy

From our perspective, it is natural to invest heavily in infrastructure and foster competitive national AI companies through sufficient infrastructure investment. The acting president recently announced a project to do just that. The National AI Committee believes that various policies are needed to ensure that artificial intelligence is well integrated across industries. In the context of US-China competition, other countries in the Middle East, Southeast Asia, and South America also aspire to enhance their AI competitiveness. However, they likely have concerns regarding technological deficiencies and a lack of data in their respective languages. South Korea has extensive experience in data accumulation and acquisition, as well as in developing AI centered around the Korean language. Furthermore, we possess experience in building related AI semiconductors, data centers, and fostering related industries. Therefore, by forming alliances and securing technological leadership, even with limited national power, we can compete effectively with the US and China by joining forces.

Companies like Minimax, Moonshot, Deel, and Alibaba Cloud, among others, have begun releasing AI models with comparable performance. This trend is not limited to China; similar advancements are appearing in the US as well. OpenAI's GPT-4 is no longer an exclusive asset. The race has begun for second-generation models. Therefore, the claim that it can be developed for 8 billion won requires careful consideration. While it might be partially true, it is a misleading simplification. The table indicates 'Training Cost,' encompassing 'Pre-training,' 'Context Extension,' and 'Post-training.' It details the GPU hours required, summing up to approximately 2.788 million hours using H800 GPUs. This implies the use of a supercomputer network of 248 H800 GPUs. The cost of renting one H800 GPU on the cloud is $2 per hour. This figure represents the GPU cost for successful training, not the overall development cost.