The History and Evolution: A Big Data Analysis of the National Innovation Systems in South Korea

: This study is a starting point to analyze South Korean national innovation systems (KNIS) using big data and provide insights for policy makers regarding how they implement the dynamic process of innovation systems. It examines KNIS that has developed over the past 14 years from 2003 to 2016 during the governments of Roh Moo-hyun, Lee Myung-bak, and Park Geun-hye. The aim of this study is to evaluate the KNIS in three ways. The ﬁrst way is to analyze the NIS of the three governments based on data of 470,000 national research and development (R&D) projects, following which the second way is to compare innovative outcomes of the three governments. The last way is to ﬁgure out the characteristics of the KNIS in innovative performance. Our analysis reveals that the KNIS was developed and evolved from 2003 to 2008, maintained until 2012, and gradually declined, even though national R&D investment increased for 14 years. Empirical evidence highlights that policies implemented for more than a decade do not e ﬀ ectively link to economic outcomes, resulting in an imbalance between innovation input and innovation output. This study further argues that the use of NIS concept in South Korea seems to be skewed towards measuring national performance from a narrower perspective.


Introduction
This study is consistent with the views of Freeman (1982Freeman ( , 1987 [1,2] at the beginning of discussions about national innovation systems (NISs). Freeman (1987) [2] questioned how Japan, which depended on imitation and imports of foreign technology in the 1950s and 1960s, was able to achieve economic growth throughout the country with technological growth that surpassed that of the US and Europe in the 1980s. How could Japan grow more quickly than the US and Europe? Several scholars, such as Freeman (1987) [2], Lundvall (1992) [3], and Nelson (1993) [4], carried out comparative studies, particularly of Japan and its institutional differences pursued over many decades, while Freeman (1987) [2] developed the concept of a 'system of innovation'.
Since the 2000s, South Korea has been making efforts to build the NIS to achieve technological innovation and thereby increase economic growth at the national level. The Roh Moo-hyun government (Roh government, 2003(Roh government, -2008 was established the South Korean national innovation system (KNIS), with the aim of achieving a gross national income (GNI) of USD 20,000. As the literature on NIS [1,3,4] outlined, the Roh government considered that NIS to be a core framework and foundation of the long-term economic development of a nation. The target was achieved in 2006, with a GNI of USD 20,823 [5]. Thereafter, the concept of the KNIS emerged as a tool for achieving national goals that were

Theoretical Considerations: Key Concepts
Since the NIS concept emerged in the mid-1980s, it has been used in designing and measuring innovative performance in many countries. NIS can be viewed as an institutional effort to recognize the state as an organic system and effectively promote innovation for the economic development of a country. Freeman (1987: 1) [2] who presented the concept of NIS in his book on Japan, argued that it is 'the institutional network between the public and private sector organizations to carry out related activities and interactions to create, introduce, revise, and advance new technology. ' Freeman analyzed Japan's rapid economic growth in the 1960s and 1980s as a result of policies in the Japanese system in which institutions, such as the Japanese Ministry of International Trade and Industry played a significant role in facilitating innovation and connecting it to national performance.
Although there is no single definition of NIS, Lundvall (1988Lundvall ( , 1992 [3,7] defined NIS as: ' . . . constituted by elements and relationships which interact in the production, diffusion and use of new, and economically useful knowledge . . . a national system encompasses elements and relationships, either located within or rooted inside the borders of a national state.' In a similar vein, Metcalfe (1995) [8] defined NIS as a system of interconnected institutions that jointly and individually contribute Sustainability 2020, 12, 1266 3 of 20 to the development of new technologies and these institutions provided the framework within which government forms and implements policies.
These scholars adopted a broad perspective where NIS not only included institutions relating to exploring and searching processes, but also subsystems-training systems, labor systems, education systems and production and financial systems, all of which also have an impact on learning activities [9,10]. Based on this point, NIS is a system of all organizations and their relationships in a country that affects the creation, diffusion, and utilization of knowledge, which evolves through cultural factors, customs, and social influences [2,3,7].
Nelson instead viewed NIS as a national R&D system focusing on patent systems, government policies, and universities that affect technology innovation. In this regard, Nelson (1993) [4] conducted an empirical study which analyzed and compared the NIS of 15 countries and identified how innovators or organizations created different innovative outcomes. Although the NIS concept was differently defined by scholars, it can still be regarded as an analytical tool to explain the competitiveness of a nation [11]. Interactions and linkages are key factors showing that growth in interactions leads to improved innovative performance. Hence, innovation systems may grow through complementary interactions at three levels: first, firms and knowledge institutions; second, interactions among different markets; and third, interactions between market and nonmarket mechanisms [12][13][14].
In line with the above, the OECD (1997) [15] started to use NIS as an institutional framework to measure national innovation performance through the Science, Technology, and Industry (STI) Scoreboard of Indicators [16,17]. OECD (1997) [15] encouraged member nations to use the concept as it could provide better knowledge of differences among member nations, and it soon became a popular analytical tool for policy makers in many countries [3,18].

Analysis of the NIS
Since its appearance as a new conceptual framework, the NIS has drawn the attention of academics and policy makers for more than two decades. Scholars put effort into theory building, undertaking a major comparative study and case studies covering various nations. Policy makers have widely used the NIS concept as an analytical tool, as it further carries the potential to provide an effective mechanism that shapes the system, instead of viewing the system as a market with anonymous players [19][20][21].
First, in their case study, Arocena and Suzt (2000) [22] analyzed the innovation system of Latin America through innovation industrial surveys held between 1995 to 1998 in Argentina, Chile, Colombia, Mexico, Venezuela, and Uruguay. The study highlights that Latin America's NIS is weak because dominant cultural patterns in Latin America underestimate the impact of knowledge and innovation. Next, Intarakumnerd et al. (2002) [23] conducted a case study to understand NIS in Thailand based on the R&D / Innovation Survey 2000 conducted by Thailand's National Agency for Science and Technology Development Agency, Thailand was recognized as a failure case in establishing an NIS due to a mismatch between the economic structural change and the development level required for NIS. Although the case study in Thailand uses more diverse indicators than that of Latin America in explaining the NIS, these two macro-level case studies seem to have limits in being able to reveal 'what is happening' when implementing the NIS in different countries. Marxt and Brunner (2013) [24] conducted a case study of the Swiss innovation system. Through in-depth interviews and workshops with experts, they analyzed why Switzerland became a leader in innovation in Europe and presented nine suggestions for further development of the NIS. However, the study only shows how the system has emerged and developed in Switzerland by focusing on the analysis of current aspects of the NIS.
Unlike previous studies that adopted qualitative research methods and examined the NIS at a specific point in time, Chen et al. (2011) [25] analyzed the effect of NIS on R&D efficiency from 1998 to 2005 through a panel dataset of 24 countries. Intellectual property rights protection, knowledge stock, and human capital accumulation were used as useful indicators for measuring the NIS. Although it is meaningful that the NIS of 24 countries is compared and analyzed for a long period of time, NIS tends to be simply measured and understood through limited indicators such as patents, royalties Sustainability 2020, 12, 1266 4 of 20 or licensing fees, and journal publications, with no consideration for national characteristics of the countries. In this regard, Balzat and Hanusch (2004) [18] argued that lack or weak development of indicators could be further developed to explain specifics of the NIS (i.e., low income countries, highly industrialized countries, or newly industrialized countries). Guan and Chen (2012) [26] criticized the NIS because research on NIS tended to remain at the theoretical level and further examined the different knowledge innovation processes of 22 OECD member countries at system level. The study of Guan and Chen (2012) [26] is of great significance because it compares and demonstrates the NIS by utilizing the eight indicators extracted from the publications of OECD and the World Competitiveness Report of the IMD, while the dataset of 22 countries is inconsistent in terms of the surveying time for each country. In addition, Chaminade et al. (2012) [27] conducted an empirical analysis of the NIS in Thailand. Indicators adopted by OECD STI Scoreboard of Indicators were used along with a dataset from the years 1999, 2001, and 2003. The scholars found a mismatch between the innovation policy and systemic problems in the Thai NIS through the advanced innovation survey conducted right after the major change of the Thai innovation system, and suggested the survey analysis should be reflected when establishing the Thai NIS.
In South Korea, the South Korea Institute of S&T Evaluation and Planning (KISTEP) developed the COmposite Science and Technology Innovation Index (COSTII) using the NIS framework [28]. Aiming at a comprehensive review of science and technology and measuring the KNIS, COSTII adopted the OECD STI Scoreboard of Indicators [29], which are useful not only to compare the KNIS with the rest of the OECD countries but also for analyzing the strengths and weakness of the KNIS. Since COSTII was introduced in 2006, it has been a popular tool among policymakers to analyze and understand the KNIS based on input-activities and outcomes. The COSTII index, which is published every year, provides national differences among OECD countries with a brief explanation. For example, in 2016, the outcomes of this metric have been ranked as low at 9th place among OECD countries, which contrasts with the high rank of R&D investment (2nd).
As described above, policy makers tend to adopt the NIS as an analytical tool for measuring national competitiveness by using indicators such as GDP, R&D investment, and patents, and this phenomenon reflect innovation processes that are considered as aspects connected to the formal process of R&D [30].

Historical Background of KNIS
Since the OECD (1997) [15] discussed the NIS concept as an analytical device, member countries have actively developed innovation policies based on the NIS approach, including South Korea. In 1996, South Korea joined the OECD, and the overarching framework for science and technology policies was gradually formulated. In 1998, the concept of NIS was widely introduced by government-funded research institutes [31]. In 2001, the Framework Act on Science and Technology was enacted to pursue the development of the national economy and contribute to the improvement of the quality of life of people, through innovation in science and technology, with the government having a legal obligation to construct the KNIS. In addition, a basic plan for national science and technology was established and promoted based on the KNIS [32].
After the legal basis of the KNIS, the Roh government, which was inaugurated in 2003, envisioned the KNIS as 'building a science-based society,' a new paradigm to strengthen growth potential while overcoming the limits of the existing input-driven economy. The Roh government first adopted the perspective of an innovation system to build a new framework for the science and technology policy, which remained as a subpolicy of economic policy. While emphasizing the interactions with the actors involved in innovation systems, the Roh government established policies from developing national strategic science and technology to supporting technology commercialization for Small and Medium-sized Enterprises (SMEs), as shown in Figure 1 [33]. The Lee government inaugurated in 2009 envisioned the KNIS as a 'leap forward to becoming an advanced country (rich people, warm society, strong nation [34].' Unlike the Roh government, which viewed science and technology from the perspective of integration with a society, the Lee government under the principle of being 'small government, big-market (business friendly),' established the KNIS as being centered on the private sector. It was desirable for the government to play a limited role in remedying market failures [34]. In addition, the 577 Initiative was advocated to systematically promote science and technology policy. The main goal of the 577 Initiative was that the total R&D investment of the country reached 5% of GDP, and that competitiveness of science and technology was improved.
The Park government inaugurated in 2013 established a national vision of 'opening a new era of hope with creative science and technology,' which envisioned an KNIS. Unlike the Lee government, which had a passive approach to the KNIS, the Park government advanced the concept of an innovation system and aimed to build an innovative ecosystem. In other words, whereas previous governments had taken the approach of developing and growing the components of innovation systems, the Park government took the approach of activating the organic integration of innovation systems. The definition of NIS mentioned in Section 2.1 makes it clear that the Park government emphasized the interactions and linkages between components. This resulted from the diversification in the expectations of innovative outcomes based on the complexity of economic and social issues that determine innovative environments and the growing maturity of innovative actors. Ultimately, an innovation ecosystem is one where the interactions between components bring forth a virtuous cycle of innovation, and where R&D investment is efficiently aligned with outcomes such as technology transfers, commercialization, and job creation. Five strategic areas were selected for the purpose of establishing the ecosystem. The main strategy was to expand R&D investment, to develop national strategic technology, to strengthen mid-to long-term creative capabilities, to support new industries, and to create jobs [35]. which viewed science and technology from the perspective of integration with a society, the Lee government under the principle of being 'small government, big-market (business friendly),' established the KNIS as being centered on the private sector. It was desirable for the government to play a limited role in remedying market failures [34]. In addition, the 577 Initiative was advocated to systematically promote science and technology policy. The main goal of the 577 Initiative was that the total R&D investment of the country reached 5% of GDP, and that competitiveness of science and technology was improved. The Park government inaugurated in 2013 established a national vision of 'opening a new era of hope with creative science and technology,' which envisioned an KNIS. Unlike the Lee government, which had a passive approach to the KNIS, the Park government advanced the concept of an innovation system and aimed to build an innovative ecosystem. In other words, whereas previous governments had taken the approach of developing and growing the components of innovation systems, the Park government took the approach of activating the organic integration of innovation systems. The definition of NIS mentioned in Section 2.1 makes it clear that the Park government emphasized the interactions and linkages between components. This resulted from the diversification in the expectations of innovative outcomes based on the complexity of economic and social issues that determine innovative environments and the growing maturity of innovative actors. Ultimately, an innovation ecosystem is one where the interactions between components bring forth a virtuous cycle of innovation, and where R&D investment is efficiently aligned with outcomes such as technology transfers, commercialization, and job creation. Five strategic areas were selected for the purpose of establishing the ecosystem. The main strategy was to expand R&D investment, to develop national strategic technology, to strengthen mid-to long-term creative capabilities, to support new industries, and to create jobs [35].   The KNIS has been shaped in accordance to the national vision, which differs from each government as shown in Figure 1. The most recent assessment of COSTII, as shown in Section 2, displayed the science and technology innovation capacity of 30 OECD countries in 2016. South Korea was ranked 5th out of the OECD countries. Under the Roh government, in 2007, South Korea was ranked 12th among OECD countries and this ranking had been maintained until the first term of the Lee government. During the Lee government, the ranking continued to rise to 9th place and South Korea finally established itself as the top five among 30 OECD countries in the Park government.
With regard to the R&D investment displayed in Table 1, since 2006, when the COSTII was first developed and measured, the ranking continued to increase throughout the Roh and Lee governments. After ranking second highest worldwide in 2013, the ranking was maintained through the Park government. The economic outcome ranking, which differs from the R&D investment ranking, stayed constant at around 10th place during the Roh government, but reached 6th place for the first time during the Lee government. This ranking dropped to 7th place during the Park government, staying constant until 2017 when it returned to 6th place again. As of 2017, however, the rankings for R&D investment and economic outcomes still show a large gap compared with those of the leading country. When observing the relative levels of R&D investment and economic outcome in relation with the top-ranking country by setting their levels at 100%, South Korea's R&D investment was 93% and the country's economic outcome was 44.9% in 2017 [29]. Similarly, as mentioned in Section 2.2, IMD research findings point out the huge gap between R&D investment and outcomes. The total expenditure on R&D as a percentage of the GDP rank increased from 6th to 5th place during the Roh government, and jumped to 3rd place during the Lee government. Moreover, during the Park government, it alternated between 1st and 2nd place. Competitiveness, by contrast, rose and fell around 30th place during the Roh government, managed to rise to 22nd place during the Lee government, but fell to 29th during the Park government (Table 2) [36].  In sum, the KNIS has achieved remarkable progress in recent decades in terms of quantitative growth of innovative performance. However, several scholars stated that it has also caused a number of mismatches in the KNIS and some weaknesses that may make the KNIS inefficient include the prevalence of uncompetitive SMEs [37], and underdeveloped linkages for the use of knowledge stock among actors [38].

Data and Analytical Method
This study aims to examine the changes in the KNIS from the Roh government, which first constructed the policy framework based on the perspective of an innovation system, to the recently completed term of the Park government. For the study to capture how KNIS has evolved for 14 years at macro-levels, we used the dataset of national R&D projects extracted from NTIS.
The point of why we utilized the data is that the KNIS can be explained by input investment and output performance, as shown in Figure 1. In the preceding section, the KNIS was built based on the Framework Act on Science and Technology. Under the Framework Act, the basic plan for national science and technology was established in accordance with the national vision of each government, and in order to promote the plan, the government had constructed the KNIS that would lead to innovative performance by investing national R&D projects. The critical factor representing the KNIS is howz'much was invested on national R&D projects and how much innovative performance was created.
The other point is that the data allow a sufficient historical character for analysis to identify the changes in the KNIS for 14 years. As previously mentioned, we obtained a dataset of national R&D projects from the Roh government inaugurated in 2003 to the end of the Park government in 2017. NTIS is the first science and technology R&D information service that integrates and provides national R&D projects information. NTIS widely contains information about 540,000 national R&D projects, such as government investment on R&D, the number of projects, and innovative performance.
As shown in Figure 2, we measure the government investment on R&D, the number of national R&D projects for each government, and innovative performance based on the dataset. Indicators for innovative performance which have been defined as outcomes in the KNIS include the number of papers published, the number of patent applications and grants, technology transfers, royalty income from the technology transfers, the number of commercialized projects, sales from the commercialized projects, and the number of jobs created by the commercialized projects. As innovative performance occurs within two years from the start of a national R&D project on average, the latest available data of 470,000 R&D projects are adopted and classified that cover comprehensive information at the start of each project: the Roh government from 2003 to 2007, the Lee government from 2008 to 2012, and the Park government from 2013 to 2016.
Sustainability 2020, 11, x FOR PEER REVIEW 7 of 21 number of mismatches in the KNIS and some weaknesses that may make the KNIS inefficient include the prevalence of uncompetitive SMEs [37], and underdeveloped linkages for the use of knowledge stock among actors [38].

Data and Analytical Method
This study aims to examine the changes in the KNIS from the Roh government, which first constructed the policy framework based on the perspective of an innovation system, to the recently completed term of the Park government. For the study to capture how KNIS has evolved for 14 years at macro-levels, we used the dataset of national R&D projects extracted from NTIS.
The point of why we utilized the data is that the KNIS can be explained by input investment and output performance, as shown in Figure 1. In the preceding section, the KNIS was built based on the Framework Act on Science and Technology. Under the Framework Act, the basic plan for national science and technology was established in accordance with the national vision of each government, and in order to promote the plan, the government had constructed the KNIS that would lead to innovative performance by investing national R&D projects. The critical factor representing the KNIS is how much was invested on national R&D projects and how much innovative performance was created.
The other point is that the data allow a sufficient historical character for analysis to identify the changes in the KNIS for 14 years. As previously mentioned, we obtained a dataset of national R&D projects from the Roh government inaugurated in 2003 to the end of the Park government in 2017. NTIS is the first science and technology R&D information service that integrates and provides national R&D projects information. NTIS widely contains information about 540,000 national R&D projects, such as government investment on R&D, the number of projects, and innovative performance.
As shown in Figure 2, we measure the government investment on R&D, the number of national R&D projects for each government, and innovative performance based on the dataset. Indicators for innovative performance which have been defined as outcomes in the KNIS include the number of papers published, the number of patent applications and grants, technology transfers, royalty income from the technology transfers, the number of commercialized projects, sales from the commercialized projects, and the number of jobs created by the commercialized projects. As innovative performance occurs within two years from the start of a national R&D project on average, the latest available data of 470,000 R&D projects are adopted and classified that cover

Inputs in KNIS
The previous sections described research methods adopted for this study. Section 5.1 provides the historical context of innovation systems in South Korea while presenting the data analyzed from NTIS. This includes how much each government invested in R&D in terms of public-private expenditure, number of R&D projects, and average government investment per R&D project. Figure 3 shows the amount invested in national R&D projects for each year from the Roh government to the Park government. Over the course of the three governments from 2003 to 2016, R&D investment in South Korea steadily increased, with the exception of 2016.
According to the results of R&D investment by each government, the Compound Annual Growth Rate (CAGR) of R&D investment was the highest at 14.47% between 2003 and 2007 at the time of the Roh government, compared to 9.50% for the Lee government and 4.22% for the Park government. It seems that the Roh government had originally planned and built the KNIS towards developing a science-based society, followed by designing policies and programs. In a similar vein, the Roh, Lee, and Park governments made an effort towards enhancing national competitiveness by increasing R&D investment.
In particular, the Roh government recognized the need for R&D investment to achieve technological innovation at the level of advanced OECD countries and thus rapidly increased its investments. In terms of aligning various actors in organizing the KNIS, R&D projects began to involve not only the government sector but also the private sector, and the collaboration between two sectors has been steadily increasing, except for during the final year of the Park government. It was observed that there are various actors that the government intends to encourage through the NIS construction process to interact with each other to carry out R&D projects. The government investment and private expenditure decreased at the end of the term of the Park government in 2016 and the trend has not been recovered yet.

Inputs in KNIS
The previous sections described research methods adopted for this study. Section 5.1 provides the historical context of innovation systems in South Korea while presenting the data analyzed from NTIS. This includes how much each government invested in R&D in terms of public-private expenditure, number of R&D projects, and average government investment per R&D project. Figure  3 shows the amount invested in national R&D projects for each year from the Roh government to the Park government. Over the course of the three governments from 2003 to 2016, R&D investment in South Korea steadily increased, with the exception of 2016.
According to the results of R&D investment by each government, the Compound Annual Growth Rate (CAGR) of R&D investment was the highest at 14.47% between 2003 and 2007 at the time of the Roh government, compared to 9.50% for the Lee government and 4.22% for the Park government. It seems that the Roh government had originally planned and built the KNIS towards developing a science-based society, followed by designing policies and programs. In a similar vein, the Roh, Lee, and Park governments made an effort towards enhancing national competitiveness by increasing R&D investment.
In particular, the Roh government recognized the need for R&D investment to achieve technological innovation at the level of advanced OECD countries and thus rapidly increased its investments. In terms of aligning various actors in organizing the KNIS, R&D projects began to involve not only the government sector but also the private sector, and the collaboration between two sectors has been steadily increasing, except for during the final year of the Park government. It was observed that there are various actors that the government intends to encourage through the NIS construction process to interact with each other to carry out R&D projects. The government investment and private expenditure decreased at the end of the term of the Park government in 2016 and the trend has not been recovered yet. Figure 4 indicates the number of R&D projects carried out by each government; it has been steadily increasing since the Roh government. Among all governments, the number of R&D projects showed the highest rate of increase in the Lee government. The first year of the Lee government, the number of R&D projects increased by about 4000 compared to the previous year and the last year of the Lee government, the number of R&D projects increased by 8000 compared with the previous year. Even though the number of R&D projects was increasing in the Park government, the increase rate is significantly lower than that of the other two governments.  Figure 4 indicates the number of R&D projects carried out by each government; it has been steadily increasing since the Roh government. Among all governments, the number of R&D projects showed the highest rate of increase in the Lee government. The first year of the Lee government, the number of R&D projects increased by about 4000 compared to the previous year and the last year of the Lee government, the number of R&D projects increased by 8000 compared with the previous year. Even though the number of R&D projects was increasing in the Park government, the increase rate is significantly lower than that of the other two governments.   Figure 5 compares the number of R&D projects with that of the government investment in R&D. The government investment for each project surpassed KRW 300 million in Lee's presidency. However, the average government investment for each project decreased in the last year of the Lee government because the total number of R&D projects shown in Figure 4 increased sharply. In the case of Park government, the average government investment for each project was again returned in KRW 300 million. Compared to the government investment shown in Figure 3, the increase of the government investment had gradually been decreasing, while the number of R&D projects had been increasing across all governments. This confirms Joo's (2015) [39] assertion that the South Korean government was committed to allocating government investment to R&D for as many projects as possible. As a result, 80% of R&D projects remained at a funding low level of KRW 50 million [40]. It is logical to assume that new R&D projects to achieve different national visions designed by each government continued to be created, which may have led to a negative effect on R&D efficiency.

Outputs in KNIS
This section presents innovative outcomes based on government investment: how many papers published, patents applied or granted, technology transfers and commercialized projects generated,  Figure 5 compares the number of R&D projects with that of the government investment in R&D. The government investment for each project surpassed KRW 300 million in Lee's presidency. However, the average government investment for each project decreased in the last year of the Lee government because the total number of R&D projects shown in Figure 4 increased sharply. In the case of Park government, the average government investment for each project was again returned in KRW 300 million. Compared to the government investment shown in Figure 3, the increase of the government investment had gradually been decreasing, while the number of R&D projects had been increasing across all governments. This confirms Joo's (2015) [39] assertion that the South Korean government was committed to allocating government investment to R&D for as many projects as possible. As a result, 80% of R&D projects remained at a funding low level of KRW 50 million [40]. It is logical to assume that new R&D projects to achieve different national visions designed by each government continued to be created, which may have led to a negative effect on R&D efficiency.   Figure 5 compares the number of R&D projects with that of the government investment in R&D. The government investment for each project surpassed KRW 300 million in Lee's presidency. However, the average government investment for each project decreased in the last year of the Lee government because the total number of R&D projects shown in Figure 4 increased sharply. In the case of Park government, the average government investment for each project was again returned in KRW 300 million. Compared to the government investment shown in Figure 3, the increase of the government investment had gradually been decreasing, while the number of R&D projects had been increasing across all governments. This confirms Joo's (2015) [39] assertion that the South Korean government was committed to allocating government investment to R&D for as many projects as possible. As a result, 80% of R&D projects remained at a funding low level of KRW 50 million [40]. It is logical to assume that new R&D projects to achieve different national visions designed by each government continued to be created, which may have led to a negative effect on R&D efficiency.

Outputs in KNIS
This section presents innovative outcomes based on government investment: how many papers published, patents applied or granted, technology transfers and commercialized projects generated,

Outputs in KNIS
This section presents innovative outcomes based on government investment: how many papers published, patents applied or granted, technology transfers and commercialized projects generated, and jobs from the commercialized projects created, as well as how much royalty income from the technology transfers and sales from the commercialized projects was obtained.
The number of papers published in journals based on government investment is shown in Figure 6 and Table 3. At the time of the Roh government, the number of papers published showed a rapid increase both in South Korea and overseas. In order to construct the KNIS, the Roh government figured out the problems of the existing South Korean science and technology administration, following which two major issues were raised; the first was the rational allocation of the R&D budget; and the second was the evaluation of performance. Therefore, the Roh government made efforts to develop an evaluation system to overcome these problems and reflect them in the KNIS, and as a result the number of papers became an evaluation indicator. Along with such efforts to evaluate R&D performance, the number of papers published began to rapidly increase since the end of the Roh government.
Adopting the new evaluation system which included in the number of papers, the number of papers increased the most compared to past years during the first year of the Lee government. Since then, the number of papers published in South Korean journals rose sharply in 2012 at the end of the Lee government, while the number of papers published in SCI journals decreased.
Under the Park government, the number of papers published in South Korean journals declined rapidly, except for in 2014. However, the number of papers published in SCI journals during this time was far greater than during the two previous governments. It seems that world-class research was carried out that led to the publication of papers in SCI journals. In addition to producing the world's first or highest level of performance, R&D performance indicators had gradually been changed towards the evaluating of papers in qualitative rather than quantitative terms (Ministry of Science, ICT and Future Planning 2016) [41]. and jobs from the commercialized projects created, as well as how much royalty income from the technology transfers and sales from the commercialized projects was obtained. The number of papers published in journals based on government investment is shown in Figure 6 and Table 3. At the time of the Roh government, the number of papers published showed a rapid increase both in South Korea and overseas. In order to construct the KNIS, the Roh government figured out the problems of the existing South Korean science and technology administration, following which two major issues were raised; the first was the rational allocation of the R&D budget; and the second was the evaluation of performance. Therefore, the Roh government made efforts to develop an evaluation system to overcome these problems and reflect them in the KNIS, and as a result the number of papers became an evaluation indicator. Along with such efforts to evaluate R&D performance, the number of papers published began to rapidly increase since the end of the Roh government.
Adopting the new evaluation system which included in the number of papers, the number of papers increased the most compared to past years during the first year of the Lee government. Since then, the number of papers published in South Korean journals rose sharply in 2012 at the end of the Lee government, while the number of papers published in SCI journals decreased.
Under the Park government, the number of papers published in South Korean journals declined rapidly, except for in 2014. However, the number of papers published in SCI journals during this time was far greater than during the two previous governments. It seems that world-class research was carried out that led to the publication of papers in SCI journals. In addition to producing the world's first or highest level of performance, R&D performance indicators had gradually been changed towards the evaluating of papers in qualitative rather than quantitative terms (Ministry of Science, ICT and Future Planning 2016) [41].    In other words, the examination duration is approximately one year to 18 months for a patent that did not apply for preferential examination; afterwards, the registration status is decided. In addition, it is mandatory for patent applications to be disclosed automatically after 18 months from their application date. Therefore, because publicized patents were counted based on the aforementioned disclosure, it appears as if the number of patent applications and grants decreased dramatically in 2015 and 2016. By the end of Roh government in 2007, the increase rate of the number of patent applications and grants was significantly higher than in previous years, while the rate gradually decreased since the Lee government. During the Park government, the increase rate continued to decline by 10%, and the rate of patent applications and grants dramatically slowed.   Figure 8 depicts the percentages of patent applications and grants. In the case of the Roh government, the percentage of patent grants was higher than applications, which changed after 2007. In the case of the Lee government, the number of patent applications also increased sharply. It is possible to speculate that researchers tended to focus on increasing the number of patent applications.   Figure 9 compares domestic and overseas patent applications and grants provided by the government. The percentage of overseas patents compared to domestic patents was strikingly low, but the number of overseas patents steadily increased, except for at the beginning of the Lee government. Researchers focused on how R&D performance gained a competitive advantage on the global scale by increasing the number of overseas patent applications and grants. Based on the year-on-year growth rate of domestic and overseas patents, the rate was higher for the domestic patents except for 2009 and 2010 during the Lee government. In 2009 and 2010, the growth rate of overseas patents was 24% and 37%, higher than domestic patents, which was 21% and 13%, respectively. There was obviously a difference between the Lee government and the Park government regarding the growth rate for domestic and overseas patents.  Figure 9 compares domestic and overseas patent applications and grants provided by the government. The percentage of overseas patents compared to domestic patents was strikingly low, but the number of overseas patents steadily increased, except for at the beginning of the Lee government. Researchers focused on how R&D performance gained a competitive advantage on the global scale by increasing the number of overseas patent applications and grants. Based on the year-on-year growth rate of domestic and overseas patents, the rate was higher for the domestic patents except for 2009 and 2010 during the Lee government. In 2009 and 2010, the growth rate of overseas patents was 24% and 37%, higher than domestic patents, which was 21% and 13%, respectively. There was obviously a difference between the Lee government and the Park government regarding the growth rate for domestic and overseas patents. The results in Figures 10 and 11 show the number of technology transfers and the royalty income from utilizing national R&D projects. The striking feature of technology transfers is that the number of technology transfers decreased continuously after the Roh government. During the Lee government, the number of technology transfers tended to decline except for during 2012, and the same results were produced during the Park government. On closer scrutiny of Figure 11, the The results in Figures 10 and 11 show the number of technology transfers and the royalty income from utilizing national R&D projects. The striking feature of technology transfers is that the number of technology transfers decreased continuously after the Roh government. During the Lee government, the number of technology transfers tended to decline except for during 2012, and the same results were produced during the Park government. On closer scrutiny of Figure 11, the royalty income decreased, while the number of technology transfers increased in 2005 during the Roh government. In the case of the Lee government, the number of technology transfers increased sharply in 2012 and was about 7000. In contrast, the royalty income did not change significantly compared to 2010, when the number of 4000 technology transfers was recorded. In the period of 2013-2016, the royalty income on each technology transfer continued to decrease. It can be interpreted that the outcomes of technology transfers gradually deteriorated in quality during this time. The results in Figures 10 and 11 show the number of technology transfers and the royalty income from utilizing national R&D projects. The striking feature of technology transfers is that the number of technology transfers decreased continuously after the Roh government. During the Lee government, the number of technology transfers tended to decline except for during 2012, and the same results were produced during the Park government. On closer scrutiny of Figure 11, the royalty income decreased, while the number of technology transfers increased in 2005 during the Roh government. In the case of the Lee government, the number of technology transfers increased sharply in 2012 and was about 7000. In contrast, the royalty income did not change significantly compared to 2010, when the number of 4000 technology transfers was recorded. In the period of 2013-2016, the royalty income on each technology transfer continued to decrease. It can be interpreted that the outcomes of technology transfers gradually deteriorated in quality during this time.  According to the report which explains technology transfers policies which is presented in Figure 1, technology transfer in South Korea was the result of a plan initiated by the government which centered on the commercialization of SMEs [42]. In the report written by the South Korean Intellectual Property Strategy Institute (KISPI) (2015) [42], 93% of domestic companies that transferred technology from the public are SMEs and only 5% are major companies. In the case of the US, 47% of technology transfers have been to SMEs, 36% to major companies, and 17% to start-up companies. Japan, one of the Asian countries, has a higher percentage of technology transfer to major companies (55%) than to SMEs (44%). Taking all of this into account, it is possible to forecast that technology transfer in South Korea was enacted from a supplier perspective compared to other countries. In other words, there was no interaction between the public and private sector in technology transfer because it was built around the public for supporting SMEs only. Figure 12 depicts the number of commercialized projects among all national R&D projects. During the Roh government, the number of commercialized projects increased steadily. In the Lee According to the report which explains technology transfers policies which is presented in Figure 1, technology transfer in South Korea was the result of a plan initiated by the government which centered on the commercialization of SMEs [42]. In the report written by the South Korean Intellectual Property Strategy Institute (KISPI) (2015) [42], 93% of domestic companies that transferred technology from the public are SMEs and only 5% are major companies. In the case of the US, 47% of technology transfers have been to SMEs, 36% to major companies, and 17% to start-up companies. Japan, one of the Asian countries, has a higher percentage of technology transfer to major companies (55%) than to SMEs (44%). Taking all of this into account, it is possible to forecast that technology transfer in South Korea was enacted from a supplier perspective compared to other countries. In other words, there was no interaction between the public and private sector in technology transfer because it was built around the public for supporting SMEs only. Figure 12 depicts the number of commercialized projects among all national R&D projects. During the Roh government, the number of commercialized projects increased steadily. In the Lee government, except for in 2009, the number of commercialized projects also increased and a total of 17,835 projects led to commercialization in 2012. This was the highest figure across all governments. Figure 13, which demonstrates the sales from commercialized projects, showed that the Lee government had the largest amount of sales among all governments. Additionally, the number of commercialized projects in 2008 was lower than in 2012, but the sales were higher in 2008. During the Lee government, the number of sales reached KRW 200 billion, representing the most striking performance caused by commercialization. However, beginning during the Park government, this number entered a downward curve and finally the number of commercialized projects which had been in the tens of thousands since 2008, sharply decreased to 7743. Although the number of commercialized projects turned positive in 2016, this was at the same level of outcomes as in the last year of the Roh government. The sales were low as well as the number of commercialization projects. Thus, it can be implied that the KNIS was not functioning properly during the Park government.      Figure 14 illustrates the number of jobs created directly by commercialized projects. There was significant correlation between the number of jobs created and commercialized projects in the Roh government. Both of them indicated the same CAGR of 76%; this was also a very high level compared to for the other two governments. In contrast, the Lee government had not exceeded the number of jobs of the Roh government except for in 2009. These findings conflict with the number of commercialized projects and the sales from the projects. In 2009, the number of jobs increased rapidly, but in the rest of Lee's presidency, it was less than 50,000 revealed in the last year of the Roh government. The number of jobs declined during the Park government.

Characteristics of the KNIS
Our main results are reported in Figure 15, which demonstrates innovative performance per government investments of KRW 100 million: the number of papers published domestically and overseas (Papers), the number of patent applications and grants domestically and overseas (Patents), the royalty income from technology transfers (Royalty income), the sales from commercialized projects (Commercialization), and the number of jobs created by commercialized projects (Job creation).
The Roh government constructed the KNIS with a legal obligation and expanded the government investment on R&D with the aim of developing a science-based society. The observable innovative performance came from commercialization and job creation in the Roh government. In 2003, more inputs brought about more outcomes and this tendency seemed to become further strengthened. All indicators of the innovative performance were positive overall in 2006. During the last year of the Roh government, in 2007, the upward trend slowed a little, but the KNIS gradually evolved with the growth of commercialization and job creation.
Most of all, the Lee government reached the highest figures of innovative performance and continued to expand its R&D investment more than the Roh government. The increase rate of innovative performance was lower than the Roh government, except for in the case of commercialization. Interestingly, the overall efficiency of the KNIS was maintained, but the correlation between input investment and output performance became weak in this period.
In the case of the Park government, the level of innovative performance other than the number of papers published showed negative growth, and was lower than the KNIS established in 2003. Although the Park government invested more heavily in R&D than the two previous governments, the R&D investment had not led to increased innovative performance. These empirical results largely confirm that no linear relationship is created between input investment and output performance. The KNIS did not develop satisfactorily (i.e., it declined) during Park's presidency.
There are two findings to note about the KNIS. One point is that the KNIS has been developing constantly since it was constructed in the mid-1990s. All governments made an effort to develop the KNIS and therefore they invested in national R&D projects with expectation of innovative

Characteristics of the KNIS
Our main results are reported in Figure 15, which demonstrates innovative performance per government investments of KRW 100 million: the number of papers published domestically and overseas (Papers), the number of patent applications and grants domestically and overseas (Patents), the royalty income from technology transfers (Royalty income), the sales from commercialized projects (Commercialization), and the number of jobs created by commercialized projects (Job creation).
The Roh government constructed the KNIS with a legal obligation and expanded the government investment on R&D with the aim of developing a science-based society. The observable innovative performance came from commercialization and job creation in the Roh government. In 2003, more inputs brought about more outcomes and this tendency seemed to become further strengthened. All indicators of the innovative performance were positive overall in 2006. During the last year of the Roh government, in 2007, the upward trend slowed a little, but the KNIS gradually evolved with the growth of commercialization and job creation.
Most of all, the Lee government reached the highest figures of innovative performance and continued to expand its R&D investment more than the Roh government. The increase rate of innovative performance was lower than the Roh government, except for in the case of commercialization. Interestingly, the overall efficiency of the KNIS was maintained, but the correlation between input investment and output performance became weak in this period.
In the case of the Park government, the level of innovative performance other than the number of papers published showed negative growth, and was lower than the KNIS established in 2003. Although the Park government invested more heavily in R&D than the two previous governments, the R&D investment had not led to increased innovative performance. These empirical results largely confirm that no linear relationship is created between input investment and output performance. The KNIS did not develop satisfactorily (i.e., it declined) during Park's presidency.
There are two findings to note about the KNIS. One point is that the KNIS has been developing constantly since it was constructed in the mid-1990s. All governments made an effort to develop the KNIS and therefore they invested in national R&D projects with expectation of innovative performance. However, the development level of KNIS was different across each government. It is assumed that there are other factors needed to make the KNIS more dynamic between input investment and output performance. This is linked to the second point. In addition to R&D investment, other socioeconomic and sociocultural factors might have impacts on innovative performance, as displayed in Figure 1. For example, all governments particularly focused on fostering SMEs and accelerating regional innovation although the COSTII did not identify how R&D investment was allocated for SMEs and regional innovation was facilitated in the implementation process. fostering SMEs and accelerating regional innovation although the COSTII did not identify how R&D investment was allocated for SMEs and regional innovation was facilitated in the implementation process.

Discussion
Since the NIS concept was introduced in the mid-1990s, it has been a popular device in policymaking contexts in many countries. In South Korea, highlighting R&D investment, the government introduced of a Plan to Construct a National Innovation System in 2004 [43]. Since then, as analyzed in Section 5, the KNIS has evolved and declined over time in performance, although R&D investment remarkably increased for 14 years. Empirical evidence highlights that policies implemented for more than a decade do not effectively link to economic outcomes, resulting in an imbalance between innovation input and innovation output. We argue three policy directions could describe the imbalanced phenomenon of the KNIS.
The first is a mismatch between the national vision and economic performance. As mentioned in Section 3, the Roh, Lee, and Park governments had different national visions, but differences have not yet been revealed in the implementation process of KNIS. At the time of the Roh government, NIS was established as a government-led public sector with a strong supply-oriented dimension, while the Lee government shifted this focus to the market, deciding that public initiative would hinder growth. The Park government advocated the concept of an innovative ecosystem by taking the government one step further from the innovation system. Although there has been a major change in the views of KNIS as presented in previous section, all the governments have implemented and measured the KNIS based on the identical framework (i.e., COSTII) along with the expansion of national R&D investment. This might lead to a gap in the implementation process between the national vision and economic performance and result in some weaknesses of the KNIS (e.g., weak SMEs) as criticized in Section 3.
The second is the greater emphasis on R&D investment as the direct input that brings about economic outcomes. As analyzed in Section 2, it seems that a strong tendency exists among policymakers to consider innovation performance as directly related to enhancing R&D activities through leading scholars [2,6,44]. In a similar vein, all three governments seemed to adopt a linear perspective to set quantitative goals to be allocated first, and then to implement policies through inputs after the establishment of KNIS. It is assumed that an increase of R&D investment contributed

Discussion
Since the NIS concept was introduced in the mid-1990s, it has been a popular device in policymaking contexts in many countries. In South Korea, highlighting R&D investment, the government introduced of a Plan to Construct a National Innovation System in 2004 [43]. Since then, as analyzed in Section 5, the KNIS has evolved and declined over time in performance, although R&D investment remarkably increased for 14 years. Empirical evidence highlights that policies implemented for more than a decade do not effectively link to economic outcomes, resulting in an imbalance between innovation input and innovation output. We argue three policy directions could describe the imbalanced phenomenon of the KNIS.
The first is a mismatch between the national vision and economic performance. As mentioned in Section 3, the Roh, Lee, and Park governments had different national visions, but differences have not yet been revealed in the implementation process of KNIS. At the time of the Roh government, NIS was established as a government-led public sector with a strong supply-oriented dimension, while the Lee government shifted this focus to the market, deciding that public initiative would hinder growth. The Park government advocated the concept of an innovative ecosystem by taking the government one step further from the innovation system. Although there has been a major change in the views of KNIS as presented in previous section, all the governments have implemented and measured the KNIS based on the identical framework (i.e., COSTII) along with the expansion of national R&D investment. This might lead to a gap in the implementation process between the national vision and economic performance and result in some weaknesses of the KNIS (e.g., weak SMEs) as criticized in Section 3.
The second is the greater emphasis on R&D investment as the direct input that brings about economic outcomes. As analyzed in Section 2, it seems that a strong tendency exists among policymakers to consider innovation performance as directly related to enhancing R&D activities through leading scholars [2,6,44]. In a similar vein, all three governments seemed to adopt a linear perspective to set quantitative goals to be allocated first, and then to implement policies through inputs after the establishment of KNIS. It is assumed that an increase of R&D investment contributed to enhancing innovative capabilities to some extent, however R&D input did not always link to expected innovative outcomes according to Section 5.3. The output indicators fluctuated, although the South Korean government made consistent efforts to increase R&D investment in the past 14 years. We argue that policy is likely to focus on limited input and output indicators as well as technical achievements, which seems to overlook sociopolitical and socioeconomic dimensions in spite of growth in size of the KNIS.
Finally, closely related to the above, policymakers tend to measure KNIS only with a statistical index composed of input and output data (i.e., COSTII) that can be quantified in a framework overlooking the discussion of the various scholars who established and developed the concept of NIS. Policymakers developed huge investment science and technology systems and statistical analysis. These developments may have led to the development of much quantitative survey-based innovation research, as described in Section 2.2. Along with the usefulness of NIS as a policy tool, mapping indicators of national performance regarding R&D efforts and innovation have been a major issue in South Korea for more than a decade. As described in Section 2, NIS is more than a framework which emphasizes a dynamic nature reflecting technological, social, and cultural aspects of a nation. As such, understanding actors and institutions, interactions that facilitate knowledge creation, diffusion, and utilization are crucial for analysis and development of the NIS. We argue policymakers may have been misled to focus on static factors and functions at a specific point in time, but the broader concept seems to be useful for shaping future national policy where specific sociocultural factors and groups co-construct and formulate the framework and concept.

Conclusions
This study examined the history of KNIS for 14 years based on a dataset of 470,000 national R&D projects. The result shows that the KNIS evolved during the Roh government, persisted through the beginning of the Lee government into 2008, and maintained itself until the end of Lee's term. Afterwards, it gradually declined during the Park government. South Korea has been steadily expanding government investment in national R&D projects through three governments from 2003 to 2016, but the outputs indicated that innovative performance levels varied among governments. Since the KNIS was established by the Roh government, innovative performance gradually appeared from 2005 and increased significantly in 2006 and 2007. In the Lee government, the highest CAGR for government investment on R&D was recorded, while innovative performance was maintained. During the Park government, the rate of government investment in R&D had stagnated and innovative performance had declined as well. Throughout the three governments, the KNIS has undergone a phase of evolution, maintenance, and decline.
In this study, we suggest the following policy implications. First, according to empirical analysis, policymakers need to consider the dynamic process of NIS in its implementation process, as the expansion of input (i.e., R&D investment) does not necessarily lead to an increase in output (i.e., innovative performance). It seems that the linear based viewpoint, in which increasing R&D investment simply leads to national economic growth, has prevailed in the policymaking arena and might have resulted in quantitative growth of the KNIS.
Second, in order to understand the dynamic and complex nature of NIS and its implementation, policy makers should consider not only quantitative indicators in measuring NIS but also qualitative indicators reflecting sociotechnical aspects. As scholars advocate the concept of NIS from a broad perspective, this study reveals that NIS is not a fixed framework based on historical review of its 14-year existence. Rather, NIS is considered as an entity that interacts continuously and moves dynamically. Therefore, we suggest that future studies should consider explanatory national and sociotechnical characteristics in analyzing and establishing the NIS. Otherwise, the process leading to innovative outcomes remains as a black box.
This study traces back the dynamic history of KNIS and contributes to existing knowledge by analyzing big data that has rarely been examined by NIS scholars. This is in contrast to previous research, which tends to focus on national comparisons of innovative performance or analyzing the current status of NIS. In particular, the concept of KNIS is mostly used in the policymaking arena to measure innovation activities every year, thereby overlooking its history. Although we attempted to examine its evolution process over 14 years, some limitations still remain. First, in addition to the dataset analyses on R&D investment and outputs during each government relevant to the historical background of the KNIS provided in Section 3, it is necessary to promote an understanding of the KNIS by analyzing other indicators that measure NISs from various perspectives (i.e., the number of new startups created in NIS projects). Second, the NIS framework used in this study does not show how actors and institutions, such as firms, research institutes, and ministries carried out their roles and how they were affected by the three different government policies. Third, although NIS scholars have emphasized interaction and learning, this study lacks explanatory interactivity among actors and therefore does not show how this interactivity enhances innovative outcomes, as indicated in Section 2. In this vein, future research would take account of exploring how different actors (e.g., policy makers and scholars) shape and implement KNIS in order to determine the principal cause of any decline in innovative performance.