Index

• Introduction
• Literature Review
• Materials and Methods
• Results
• Discussion
• Conclusion
• References

Abstract

Educational evaluation is necessary for the continuing development of education by monitoring the direction of educational development toward its set goals. The Ministry of Education in China has set the overall goal of higher vocational education to reach a world-class level by 2035. In the current internationalizing environment, this study accordingly aimed to develop a set of internationalization evaluation indicators for higher vocational education in Shandong Province, China. Previous studies on internationalization evaluation indicators have exhibited differences and limitations in terms of their objectives, measures, and their differentiation between undergraduate-based university and vocational colleges. Most previous studies have been based on reviewing certain aspects of internationalization, whereas few studies have considered the issue from the perspective of internationalization practitioners. This study first reviewed and synthesized previous research to develop a framework of international development elements for higher vocational education in a Chinese context. This was then analyzed by a focus group of 15 people in charge of the international offices (schools) of their higher vocational colleges, who identified 6 first-level indicators, 14 second-level indicators, and 14 third-level indicators. These indicators were then analyzed using a general inductive method. To the best of our knowledge, this is the first attempt to identify internationalization evaluation indicators for higher vocational education from a phenomenological research perspective. Based on the results, we draw forth the theoretical and practical implications for researchers, practitioners, and policymakers.

Keywords: Development elements, Education evaluation, Evaluation indicators, Higher vocational education, Internationalization evaluation.

Received: 18 May 2023 / Revised: 13 July 2023/ Accepted: 21 August 2023/ Published: 25 August 2023

Contribution/ Originality

This study is among the first to employ a qualitative method to construct a novel set of internationalization evaluation indicators for vocational education in the Chinese context. The indicators reflect vocational education’s distinctiveness and governmental alignment, providing practical tools and theoretical support for policymakers and practitioners.

1. INTRODUCTION

Since the 1980s, the definition of internationalization has been widely discussed (Arum & Van de Water, 1992; Francis, 1993; Knight, 2004; Soderqvist, 2002; Van der Wende, 1997). Although a uniform definition of the internationalization of higher education is lacking, most scholars have mentioned some key concepts such as international, globalization, mobility of teachers and students, and educational activities. Generally, internationalization of higher education refers to the process of integrating global perspectives, cross-cultural understanding, and international engagement into goals, functions, and implementation processes of higher education, all of which aim to improve the quality of education and research, to create a more meaningful educational environment for all students and faculty, and make significant contributions to society (Wit, Hunter, Howard, & Egron-Polak, 2015). Neoliberalism, globalization, changes in the labor market, and the development of the knowledge economy have all contributed to the internationalization of education (Tran & Dempsey, 2017). Neoliberalism and globalization sustain the free flow of resources and capital and encourage free competition on a global scale. The labor market and knowledge economy currently have set high standards and international requirements for students’ knowledge, intellectual competence (i.e., critical thinking skills), and professionalism in the global workplace. Consequently, the internationalization of higher education has become inevitable, and higher education must thus compete globally by introducing high-quality educational resources, supporting international students’ mobility, and cultivating first-class international talents.

The improvement of labor quality, increases in the employment rate, and development of the economy and society require a considerable number of middle- and high-level skilled talents (State Council, 2014). Higher vocational education is essential in cultivating technical and skilled talent and artisans who can provide China with a competitive advantage (Xi, 2021). Based on this understanding, central and local governments have recently announced a series of policies to promote the development of vocational education nationwide, including measures for its internationalization. These policies were necessary to establish a vocational education brand with Chinese characteristics and promote vocational education to go global. In China, the overall objective of higher vocational education is to enable the country to become a global leader by 2035 (General Offices of the Communist Party of China Central Committee and the State Council, 2021).

Moreover, educational evaluation is being promoted throughout China as part of its intensified educational reform. In October 2020, the Central Committee of the Communist Party of China and the State Council issued the “Overall Plan for Deepening the Reform of Educational Evaluation in the New Era.” These government agencies aimed to guide the reform of educational evaluation at all levels and in all types of schools and proposed to establish the guiding role of educational evaluation and improve the evaluation of internationalization (Communist Party of China Central Committee and the State Council, 2020). The Ministry of Education has promoted this proposal, from which local governments, education departments, and colleges stand to benefit. Accordingly, as an important component of educational evaluation, internationalization evaluation will be highly valued and taken seriously.

In sum, internationalization is an inevitable process in higher education in today’s globalized economy (Valiulis & Valiulis, 2006). The direction of education development is closely related to its goals and educational evaluation. The Chinese government has promulgated policies to promote higher vocational education, announcing the goal of being the world’s foremost in the field. These policies propose creating new requirements for the internationalization process and adding new curricular content. Meanwhile, to steer education development in the right direction and ensure that national education develops at a pace similar to that of the rest of the world and meets the needs of students and parents, the Chinese government has implemented education evaluation reforms; these reforms emphasize the improvement of internationalization evaluations. To facilitate internationalization, therefore, evaluation indicators are needed to guide and achieve high-quality development. This study aimed to establish a set of internationalization evaluation indicators suitable for higher vocational education in Shandong Province, China.

The purpose of these indicators is to meet the requirements of the current globalized world, further promote education internationalization, achieve the development goals of higher vocational education, and satisfy the needs of society.

2. LITERATURE REVIEW

Research on the internationalization of higher vocational colleges has become increasingly important in recent years (Nurhaeni, Nugroho, & Kusumawati, 2019; Perini & Kämäräinen, 2018; Scheuch et al., 2021; Tran & Dempsey, 2017; Zlatkin‐Troitschanskaia & Pilz, 2022), most of which has focused on the evaluation indicators for the internationalization of universities or research universities (Bound, Turner, & Walsh, 2009; Horn, Hendel, & Fry, 2007; Paige, 2005). Some studies have included vocational colleges in their internationalization evaluation indicators, without distinguishing between undergraduate-based universities and vocational colleges, like the indicators by the International Quality Review Process (IQRP) , Germany’s Centre for Higher Education Development (CHE), the Australian Universities Quality Agency (AUQA), and the American Council on Education (ACE) (Australian Universities Quality Agency, 2011; Brandenburg & Federkeil, 2007; Center for Internationalization and Global Engagement, 2023; de Wit & Knight, 1999).

There have also been studies in some countries, specifically on the internationalization evaluation indicators for higher vocational education. For example, the China Education Association for International Exchange (CEAIE) designed a survey of 93 internationalization indicators to investigate the current status of internationalization in Chinese higher vocational education (Ministry of Education, 2016). In addition, some Chinese researchers have also proposed evaluation indicators for internationalization of higher vocational education (Liu, 2013; Su & Wei, 2010).

As mentioned above, there has been much comprehensive research on indicators for evaluating internationalization in education. Vocational colleges can employ these indicators for self-evaluation and diagnosis. However, the existing indicators have differences and limitations. First, they serve a variety of purposes: IQRP focuses on determining the implementation of established policies and strategies of educational institutions; ACE, on examining the degree of internationalization development in education; AUQA and the Ministry of Education, Culture, Sports, Science and Technology (MECSST) in Japan, on regulating the quality assurance of internationalization in education; CHE, on benchmarking indicators to lay the foundation for ranking higher education institutions; and CEAIE, on the provision of data and information for future policymaking.

Second, except for the indicators designed specifically for vocational colleges by MECSST and CEAIE, indicators such as IQRP and those proposed by ACE do not differentiate between undergraduate-based universities and vocational colleges. The use of the same set of indicators for both types of institutions leads to a lack of specificity and failure to highlight the distinct characteristics of vocational colleges, ultimately affecting the evaluation outcomes. In reality, there are significant differences between vocational colleges and undergraduate-based universities in terms of educational goals, teaching methods, and internationalization efforts. For example, vocational colleges emphasize the development of skills across a wide range of vocational fields through school-based and work-based learning (Organization for Economic Co-operation and Development, 2023), and accordingly internationalization efforts are closely related to the skills and practical experiences of both teachers and students. Meanwhile, undergraduate-based universities engage in academic research and knowledge exploration in all areas of human understanding through research, learning, and teaching (Council on Higher Education, 2013). Therefore, internationalization efforts in their domain are closely related to the knowledge, academic exchange, and shared experiences of teachers and students.

Third, different measures have been employed in constructing the existing indicators. Many studies on internationalization evaluation indicators do not clearly indicate the measures used to construct the indicators. Some studies state that the indicators were established through research on existing literature and analysis of internationalization, such as Paige’s internationalization evaluation indicators for universities. However, some internationalization indicators have used a “model” or “framework.” For example, the CHE follows the “input-process-output” model to establish input and output indicators within the three dimensions of overall aspects: academic research, teaching, and studies (Gao, 2015). A framework is generally understood as the structures used to organize areas of interest, and by building or adopting a framework, indicator designers can still identify the key dimensions of interest (Gao, 2015). For example, before establishing the internationalization evaluation indicators for universities, Paige (2005) had first created a framework of 10 dimensions of internationalization key performance categories, while Gao (2015) established a framework of 6 dimensions for the indicator set for measuring university internationalization. The framework measure can effectively reflect the indicators by identifying internationalization dimensions (Gao, 2015). By comparing the possible applications of “framework” with those of “model,” the authors chose to build a vocational education internationalization elements framework that considered with future internationalization indicators.

Fourth, the design of education internationalization evaluation indicators should involve the participation of internationalization practitioners from colleges. The OECD indicators were developed by two experts in higher education, Jane Knight and Hans De Wit, under the auspices of the Institutional Management in Higher Education and the Academic Cooperation Association of the OECD. The CHE invited directors of international offices from four German higher education institutions to jointly develop the indicators, while other indicators, such as the ACE and the AUQA, were mostly devised by government departments. Many researchers have emphasized the importance of involving various stakeholders from institutions in the development of indicators, as such indicators are more likely to be used in their institutions after their creation (Gao, 2015). In agreement with this point, this study will invite practitioners of internationalization from vocational colleges to jointly identify the indicators.

Most importantly, no study has evaluated vocational education internationalization in Shandong Province, China. Higher vocational education in Shandong has the same characteristics as presented in the aforementioned studies, but it also has distinctive features in policy planning and development priorities. Evidence on these features needs to be supplemented and improved, as previous studies have thus far not contributed to the internationalization of higher vocational colleges in Shandong Province.

2.1. Theoretical Framework

Knight (2004) identified several aspects related to internationalization, such as academic mobility for teachers and students, international cooperation projects, international academic research projects, educational support among countries such as establishing branch campuses, and internationalized courses or teaching processes. Scheuch et al. (2021) argued that the internationalization of vocational education should include vocational education research with an international perspective and projects with international vocational education cooperation partners. Virkkuen et al. (2022) suggested that the elements of internationalization in vocational education include the mobility of teachers and students to other countries, participation in international projects, and participation in international vocational training networks or annual forums.

Among the aforementioned internationalization evaluation indicators like, those from the IQRP, ACE, AUQA, CHE, CEAIE, most were implemented by international organizations or government agencies with convincing reliability and effectiveness. These evaluation indicator systems cover various internationalization indicators, with some containing three to five primary indicators and others containing seven to ten. Indeed, the indicators are significantly different, but this study has identified some commonalities among the indicators mainly focusing on: internationalization strategy and planning (de Wit & Knight, 1999), organizational management (Center for Internationalization and Global Engagement, 2023), faculty involvement in international activities (Australian Universities Quality Agency, 2011), international mobility of students (Brandenburg & Federkeil, 2007), internationalization of disciplines and curriculum (Ministry of Education, 2016), international partnerships (Australian Universities Quality Agency, 2011), and internationalization of research and academic cooperation (Scheuch et al., 2021). These indicators embody the features of internationalization, and can be regarded as the elements of internationalization.

Chen (2004) posited six basic elements in the internationalization of higher education: formulating the internationalization education concept, creating international objectives, encouraging international courses, participating in international exchange between teachers and students, promoting international academic exchanges and cooperative research, and sharing educational resources. Pan (2012) then supplemented the characteristics of the internationalization of higher vocational education based on the following: promoting the vision of internationalization, pursuing the goal to cultivate international talent, offering internationalizing courses, facilitating international travel for students and teachers, internationalizing experiments and practical training, and exchanging vocational education resources. Another main element of internationalization in vocational colleges includes international joint programs and institutions, such as Sino-foreign joint programs and institutions and off-shore programs and institutions (Liu & Tan, 2022; Qin, 2021).

Although previous studies have adopted different dimensions, the existence of common internationalization components has been confirmed through cross-checking (Gao, 2015). The authors identified 22 elements from the aforementioned previous research. After further categorizing and consolidating similar components, a framework of international development elements for higher vocational education was synthesized and established Figure 1. Based on this framework, this study developed a set of evaluation indicators for the internationalization of higher vocational education in Shandong Province.

3. Materials and methods

Qualitative research consists of a set of interdisciplinary, interprofessional, inter-field, and inter-thematic research methods (Denzin & Lincoln, 1994). Creswell (2014) divided qualitative research methods into five groups: ethnography, narrative, phenomenological, grounded theory, and case study.

In this phenomenological study, the indicators were constructed using the qualitative research method. The original data collected included venue notes, focus group recordings, dialogues, emails, audio recordings, and memos. All these data were triangulated to provide a rich description and adduce trustworthy evidence related to the events, phenomena, and people who helped the researchers gain a better understanding of the research objectives (Zhang & Hozer, 2001).

The purpose of conducting a focus group using open-ended questions (see Appendix A: Focus Group Questions) was to collect unbiased data about the participants’ feelings and perspectives on the internationalization process at their institutions. The focus group purposefully included only those participants who were familiar with internationalization. The participants shared some characteristics closely related to the theme of the focus group; the researchers encouraged the participants to share their perspectives in the focus group (Krueger & Casey, 2000). Although the participants came from different cities in Shandong Province, the focus group was conducted at one venue. The questions for the focus group were designed based on a literature review, and it aimed to make the participants feel that their perspectives were valued, allowing them to express their own views and opinions. This procedure facilitated collecting credible and reliable information in a relatively short time.

3.1. Research Location

The study focused on Shandong Province, China. In 2022, the province’s gross domestic product was RMB 8.7 trillion (RMB is the official currency of the People’s Republic of China; RMB 8.7 trillion is equal to $1.27 trillion dollars; Shandong Provincial Bureau of Statistics (2023)), ranking third in China (Xinhuanet, 2023). It has 102 million permanent residents (Shandong Provincial Bureau of Statistics, 2023), making it the second most populous province in China (Peopletech, 2023), and houses 83 higher vocational colleges, ranking third in the country (Ministry of Education, 2022). Hence, Shandong Province is a good fit for research on vocational colleges considering its economic and social development and the high number of vocational colleges.

3.2. Participants

The focus group participants were 15 individuals in charge of internationalization at different vocational colleges who had worked in international education for many years; their demographic details are listed in Table 1. All the participants were selected via purposive sampling and were contacted through email about participating in this study. The inclusion criteria were (1) currently working at the international office (school) of higher vocational colleges in Shandong Province; (2) had at least two years of work experience in internationalization; and (3) held the position of director (dean) or deputy director (associate dean) of the international office at a vocational college.

Name	Title	Level	Province
Participant #1	Vice president	B	Shandong
Participant #2	Associate dean	B	Shandong
Participant #3	Director	A	Shandong
Participant #4	Deputy director	B	Shandong
Participant #5	Dean	A	Shandong
Participant #6	Associate dean	A	Shandong
Participant #7	Dean	A	Shandong
Participant #8	Dean	B	Shandong
Participant #9	Deputy director	B	Shandong
Participant #10	Associate dean	A	Shandong
Participant #11	Director	A	Shandong
Participant #12	Director	A	Shandong
Participant #13	Deputy director	A	Shandong
Participant #14	Deputy director	B	Shandong
Participant #15	Deputy director	B	Shandong

Note:

Level A refers to a high-level construction college conferred by the ministry of education. Level B refers to a normal college with no special rank by the ministry of education.

3.3. Data Collection

This study used the focus group method for data collection, which consisted of the following steps:

(1)   By combining the framework of the international development elements of higher vocational education with a literature review, a draft list of internationalization evaluation indicators was developed.
(2)   Using this draft list, a list of open-ended questions about the indicators was prepared. It included suggestions for revisions to the draft indicators and open-ended questions related to the indicators. The focus group questions are presented in Appendix A.
(3)   Using a purposeful sampling methodology and based on recommendations from the provincial education department, the authors invited 15 participants from the international offices of 15 vocational colleges to participate in the focus group. All the participants signed consent letters and were able to withdraw from the study at any time.
(4)   A moderator who was familiar with evaluation and internationalization was selected.
(5)   The date and location of the focus group was confirmed.
(6)   The focus group schedule was created, the meeting room was prepared for conducting the focus group, all the equipment were checked, and the personnel and time were finalized.
(7)   The participants were informed of the details of the study before the focus group was conducted to ensure they were aware of the nature of the research and the importance of confidentiality and other ethical principles.
(8)   All 15 participants attended the focus group, provided answers to the open-ended questions, and freely expressed their views on the topic. During the process, the moderator solicited the participants’ opinions on relevant issues following the focus group questions. To avoid researcher bias, the moderator only asked questions about relevant issues from the questions and did not express tendentious or guiding opinions. The focus group was divided into two sessions, each lasting 60 minutes. Participants engaged in discussions around the contents of indicators, sharing their thoughts, and exchanging diverse opinions. The entire process was recorded using a Newsmy V03 voice recorder.

3.4. Data Analyses

In this study, based on the general inductive approach, this study employed the coding analysis software (NVIVO) to analyze the audio recording of the focus group. The audio recording was transcribed for systematic analysis.

3.4.1. General Inductive Approach

A general inductive approach was adopted to facilitate the data analyses. This involved systematically analyzing the qualitative data based on specific evaluative objectives. Furthermore, this method involved interpreting the raw data in detail to derive themes, concepts, or models based on the researcher’s interpretations (Thomas, 2006). Compared with other analytical methods, the inductive approach is short, non-technical, and primarily used to analyze qualitative data.

3.4.2. Procedures of the General Inductive Approach

The general inductive approach used in this study included the following procedures:

(1)       Preparing raw data files: The authors sorted through 150 minutes of recordings of the focus group and converted the digital data into a written transcript using a speech-to-text website (Feishu Minutes). The transcript was then translated into English using a translation website (translate.google.com). During the process, all authors cross-checked the transcription and proofread the translated text to ensure the accuracy and reliability of the 53-page English translation.
(2)       Reading the text carefully: The authors read the transcripts in detail to understand the views and opinions of the focus group participants on the draft of the evaluation indicators from the focus group questions.
(3)       Creating categories: Based on the draft of the evaluation indicators, the authors read, analyzed, and created category labels and coded the text using a transcription analysis software (NVIVO).
(4)       Finding overlapping encoded and unencoded text: According to the coding results, all authors found overlapping coded and uncoded texts, verified and cross-checked the coding accuracy, and made necessary adjustments.
(5)       Revising and refining categories: In each category, the authors found new insights and contradictory views regarding the first-, second-, and third-level evaluation indicators.
(6)       Analyzing and summarizing the coding: Analyses and inductions were used to determine the specific content of the evaluation indicators at all levels, and contradictory views with significant differences were noted. Next, the researchers created the second draft of the internationalization evaluation indicators, which was subsequently sent to the 15 participants by email, so that they could reach an agreement on the final version after cross-checking, providing feedback, and making revisions. Finally, with the unanimous approval of the participants, the researchers generated a unique set of internationalization evaluation indicators for higher vocational education in Shandong Province.

4. RESULTS

The qualitative analyses of the data yielded 6 first-level indicators, 14 second-level indicators, and 14 third-level indicators. Table 2 contains the complete list of newly established indicators for evaluating internationalization.

First-level		Second-level	Third-level
Strategy, organizational management, and support		Strategy and planning; Structure and regulations; Support for internationalization.	Internationalization development plan; Mechanism & regulations a; Funding & support services b.
Mobility of faculty		Domestic faculty; Foreign faculty.	Proportion of overseas visiting experience c; Proportion of foreign teachers.
Mobility of students		Domestic students; International students.	Proportion of overseas experienced; Proportion of foreign students.
Internationalization of curriculum		Chinese language+ skills curriculum; International professional standards.	Number of “Chinese language + skills” curriculum items; Number of international professional standards.
International joint programs and institutions		Sino-foreign joint programs and institutions; Off-shore programs and institutions;	Number of Sino-foreign joint programs and institutions; Number of off-shore programs and institutions.
International cooperation and exchanging resources		International cooperation platforms; International cooperation programs; International conferences.	Number of international cooperation platforms e; Number of international cooperation programs f; Number of international conferences g.

Note:

a Mechanism & regulations include establishing an international working committee, a full-time international office, and rules and regulations for internationalization. b Funding & support services include special funds for international exchange and cooperation, English-version websites, and internal indicators for practicing internationalization. c Faculty overseas visiting experience includes overseas study; participation in international conferences or skills competitions; holding managerial and technical posts in international organizations, associations, or overseas enterprises; or obtaining international qualification certificates. d Student overseas experience includes exchange experience, receipt of international vocational qualification certificates, or employment outside China. e International cooperation platforms include international cooperative research platforms (centers), think tank platforms, skills training centers, and practice and training bases jointly established with foreign enterprises and Chinese-funded enterprises located outside China. f International cooperation programs include international academic cooperation programs, international technical skills training programs, and awards in foreign skills competitions. g International conferences include international skills competitions sponsored or undertaken.

For convenience, only the most germane results from the focus group are presented below.

4.1. Strategy, Organizational Management, and Support

All the participants agreed that strategy, organizational management, and support for internationalization were elements that should be included in the college’s development plan to fully reflect the goal of cultivating international talents and the internationalization vision of the college. As one participant stated,

. . . The internationalization indicators are fundamental, and they need to be represented in the top-level design of the college’s development plan . . . after formulating the 14th Five-Year Plan, we made a special plan for internationalization. (Participant #1)

Similarly, many participants considered the establishment of an internationalization committee and an international office essential. One participant noted,

The internationalization of a college is not a matter of just the international office, nor is it a matter of the vice president. It is a matter of a whole college . . . internationalization is involved in all aspects of a college, and its committee needs to be established to coordinate and organize the implementation of the internationalization of the whole college. (Participant #5)

In terms of rules and regulations, many participants agreed that they were necessary. One participant said,

To implement internationalization, there must be rules and regulations to guarantee results. Otherwise, whether a leader or an employee, everyone has different personalities and sometimes they will lose sight of each other, so there must be a standard. (Participant #8)

In terms of financial support, all the participants agreed on the inclusion of this indicator. In line with this, one participant expressed,

The international development of a college cannot be like water without a source; you have to look at the funding. . . . Without financial support, the internationalization of higher vocational education will not go far. (Participant #3).

Regarding the establishment of internal indicators for practicing internationalization, many participants believed that this would help the college function more smoothly. One participant stated,

A college generally encourages internationalization if there are internal indicators, which is a principle they must follow. It will increase the momentum for a department or a school. (Participant #10).

In terms of the English-version website, one participant emphasized its necessity saying,

A college’s internationalization should include at least an English website with sufficient content that is updated frequently. (Participant #15).

4.2. Mobility of Faculty

This indicator represents international faculty mobility in Figure 1. According to the findings, the participants unanimously emphasized the importance of sending more full-time teachers abroad to study, attending international conferences, competing in international skills competitions, obtaining international vocational qualification certificates, and participating actively in international activities. As one participant pointed out.

I feel that teachers in our vocational colleges should go abroad to further their technical skills or obtain their vocational qualification certificates to strengthen vocational education; moreover, employing foreign teachers should be an essential feature. (Participant #2).

Another participant added,

In our college, many teachers have overseas study experience and international backgrounds. It is suggested that this should be taken into consideration. . .. There are also many teachers who go abroad to participate in international conferences and international skills competitions, and some other teachers work in technical positions in overseas companies, which will have a beneficial impact on the internationalization of a college . . . (Participant #4).

4.3. Mobility of Students

This indicator encapsulates students’ international mobility and practical training. Student mobility was emphasized by all the participants, as reflected in one participant’s response:

After graduation, students also go abroad directly through our partner colleges for undergraduate and postgraduate degrees. This should be an indicator . . . and for “going global” education (promoting students’ overseas study experience), China state-owned enterprises have established many overseas enterprises, and many of our students will go to these overseas enterprises for an internship or a job. This should also be an indicator. (Participant #11).

Regarding international students, one participant expressed his thoughts as follows:

International students will participate in training programs at our college. Some will stay for three months or more, but others for less than three months. They can also go to a company for a multi-week internship. This is also a characteristic of vocational education. (Participant #9).

4.4. Internationalization of the Curricula

This indicator was the most discussed by all the participants and was also considered the matter of greatest concern. They all stressed that the curricula should emphasize “Chinese language + vocational skills” (to encourage more international students to learn both Chinese language and vocational skills) and international standards. One participant said,

We need to step up our “Chinese language + skills” policy, a school-managed agenda of our vocational colleges. . . . There should be an increase in the number of “Chinese language + skills” majors or courses, which aligns with one of the characteristics of our vocational colleges [to cultivate more compound language talents to aid the development of overseas Chinese-funded enterprises] . . . (Participant #14).

Another participant added,

We must formulate standards for our majors and courses at higher vocational colleges. Whether these standards have acquired international certification or have been adopted, I think this should be an indicator. . .. Moreover, for this indicator, I think we should promote skills training together with Chinese language training. Through vocational skills training, foreigners can learn the Chinese language and learn [about] our excellent traditional culture. (Participant #7).

4.5. International Joint Programs and Institutions

This indicator represents the content of international joint programs and institutions, and it includes Sino-foreign joint programs and institutions and off-shore programs and institutions. In terms of Sino-foreign joint programs and institutions, one participant said.

In our annual quality report for domestic Sino-foreign joint programs and institutions, we have two evaluation points: the number of Sino-foreign joint programs or institutions and the number of students enrolled. Here, I think the number of Sino-foreign joint programs or institutions and the number of students should be viewed as two separate indicators. (Participant #13).

In terms of off-shore programs and institutions, one participant added,

As for off-shore programs and institutions, a job training session on “going global” was just held yesterday. At the meeting, it was revealed that the province’s vocational colleges have 57 overseas joint education sites, which means we have had fruitful success [in the development of overseas joint programs and institutions]. Furthermore, of the seven workshops approved across the country, three were conducted in Shandong Province, and our province ranked No. 1 in the country. . .. I feel that this has become very important. (Participant #7).

4.6. International Cooperation and Exchanging Resources

There was a unanimous emphasis on international skills training, skills competitions, practical training, international cooperative research, and international exchange of educational resources. One participant noted,

I think as part of our vocational colleges, we may attend fewer academic conferences, and we should participate in more international skills competitions . . . . So, one of our vocational characteristics is more exchanges or competitions related to technical skills . . . . Moreover, the cooperative research platforms approved by the Provincial Educational Department or the Ministry of Education provide us more exchange opportunities with foreign colleges. (Participant #12).

One participant added,

We should choose indicators that best illustrate these bases of technical education and training. . .. After all, practical teaching should account for more than 50% of the proportion [of the subjects taught in vocational education]. It is recommended that this percentage be increased if the practical training base is given [more] weight [in the evaluation system]. Lastly, the cooperative research programs established with foreign institutions are also important components of our work. (Participant #9).

To summarize, the 15 participants identified and agreed on six first-level indicators within the framework of international development see Figure 1. After answering the focus group questions on the content of the second- and third-level indicators, all 15 participants expressed their personal opinions about each first-level indicator. Next, the authors categorized and coded transcript, found instances of overlapping coding, cross-checked the content of the corresponding second-level and third-level indicators, and verified whether the 15 participants agreed with the content of each indicator. The authors found that all the participants were in agreement regarding the content of all the first- and second-level indicators and of more than 75% of the third-level indicators. They disagreed with a few third-level indicators, such as whether the indicator of international conferences needed to be deleted or whether the term “international student” should include students who study in joint programs and institutions operating outside of China. The authors then summarized and produced the second draft of the evaluation indicators and sent it by email to all the participants; opinions were solicited again, particularly regarding differences. Finally, all 15 participants agreed with 31 third-level indicators and established one uniform set of internationalization evaluation indicators for higher vocational education in Shandong Province, China.

5. DISCUSSION

The aim of this study was to investigate the status quo of internationalization in higher vocational education colleges, discern their internationalization rankings within Shandong Province, and provide references for policymakers. To this end, this study adopted the framework approach used in previous research, and proposed a six-element framework for the development of internationalization in higher vocational education. Based on these development elements, a draft of the evaluation indicators was compiled and a list of focus group questions was developed. The findings provide a theoretical foundation and support for the internationalization of higher vocational education. This study was also the first to employ a qualitative research method to construct internationalization evaluation indicators for higher vocational education. While many researchers have previously proposed internationalization evaluation indicators for higher vocational education (Liu, 2013; Ministry of Education, 2016; Su & Wei, 2010), none employed a qualitative focus group analysis. In contrast, the current research adopted the focus group method, discussed with 15 stakeholders in charge of the internationalization of higher vocational education, used the general inductive method to analyze the focus group transcript data, gleaned valuable insights from the data analyses, and created a novel set of internationalization evaluation indicators for higher vocational education. As a novel undertaking in the research field of internationalization evaluation indicators, this pioneering study galvanized the formation of a significant new instrument to evaluate the internationalization of vocational colleges and provided more rigorous approaches and methods for research in this field. The indicators of this study reflect the distinctiveness of higher vocational education. Unlike previous studies, this study specifically focused on internationalization in the context of higher vocational education. Vocational education has unique characteristics, such as serving economic development primarily through the cultivation of technical and skilled talents (Organization for Economic Co-operation and Development, 2023); it therefore merits separate study regarding internationalization. Zhou (2017) argued that an internationalization evaluation should fully consider the needs and objectives of the respective institutions and aim at predicting and modifying future internationalization efforts to achieve the desired goals. Therefore, this study proposed innovative indicators related to international skills competitions, international vocational qualification certificates, international professional standards and curriculum development, and international skills training centers that all complement and further validate the findings of previous studies. In doing so, this research provides another tool for research on the internationalization evaluation indicators for higher vocational education, especially in China.

The indicators of this study encompassed the higher vocational education policies of the central government and the Shandong provincial government. Jamil and Jaffar (2023) mentioned the influence of government policies on the internationalization of vocational education, stating that when adopting internationalization indicators, higher education institutions tend to prioritize the needs of and support for their own country and institution. This study corroborates this research result, and the authors were most impressed by the following three documents mentioned by all focus group participants: Opinions on Promoting the High-quality Development of Modern Vocational Education (General Offices of the Communist Party of China Central Committee and the State Council, 2021); Proposal on Promoting the Improvement of Quality, Training and Excellence in the Whole Province to Build a Highland for Innovation and Development of Vocational Education Opinions (Ministry of Education, 2020); and Opinions of 10 Provincial Departments of Shandong Province on Supporting the Opening of Vocational Education in the New Era (Shandong Government, 2020). The participants unanimously agreed that the indicators should fulfill all the requirements of these three documents for internationalizing higher vocational education in Shandong Province because of the clear related requirements that they lay out. This study created novel indicators, such as the “Chinese language + vocational skills” major and curriculum development, practice and training bases, and international skills training centers, fully reflecting the latest policy orientations in these aforementioned documents.

5.1. Limitations and Future Research Directions

Although this study constructed a new set of evaluation indicators, it has several limitations. First, this qualitative study, conducted using a focus group and a general inductive approach, is limited in terms of data collection and analysis, as well as the natural biases of both the authors and the participants. Second, this study only recruited 15 participants because of geographical and time constraints. Future studies should increase the number of participants and discuss with more people familiar with the internationalization of higher vocational colleges, as this will enhance the generalizability of the results to other populations. Third, this study only determined the evaluation indicators and explored the significance of each. Future studies should confirm the weight of each indicator and restructure the comprehensive evaluation system based on weighted indicators for higher vocational education in Shandong Province.

6. CONCLUSION

This study makes the following practical and theoretical contributions for higher vocational education internationalization in three ways. First, this study provides a research-based frame of reference for policymakers. The indicators were derived from real-world cases and can be applied in practice. Policymakers or government officials can use the indicators for the performance appraisal of higher vocational colleges and to establish a data system under the evaluation indicators, gather related data to evaluate the internationalization level of higher vocational colleges, and provide a better policy foundation for the future international development of higher vocational education.

Second, this study shows that it is practical and feasible to use qualitative research methods in the construction of internationalization evaluation indicators. Based on the theoretical basis of international development elements, this study conducted a qualitative study using the focus group and general inductive method. It fully demonstrates the function of qualitative research in the study of evaluation indicators and provides a new internationalization evaluation instrument that can be adopted for research in similar areas.

Finally, the developed indicators—such as international skills competitions, international vocational qualification certificates, international professional standards and curriculum development, “Chinese language + vocational skills” major and curriculum development, foreign practice and training bases, and international skills training centers—fully reflected the distinctiveness of higher vocational education and China’s national policies, All of which are indispensable elements for the internationalization of higher vocational education. Practitioners and policymakers in this field should incorporate them into their scope of practice and provide more financial support to strengthen policy orientation.

Funding: This research is supported by the Shandong Provincial Education Department (Grant number: [2022]67). Institutional Review Board Statement: The Ethical Committee of the Woosong University, Republic of Korea has granted approval for this study on 9 December 2022 (Ref. No. EAL-WS/1530/9DEC/2022). Transparency: The authors state that the manuscript is honest, truthful, and transparent, that no key aspects of the investigation have been omitted, and that any differences from the study as planned have been clarified. This study followed all writing ethics.

Competing Interests: The authors declare that they have no competing interests.

Authors’ Contributions: All authors contributed equally to the conception and design of the study. All authors have read and agreed to the published version of the manuscript.

REFERENCES

Arum, S., & Van de Water, J. (1992). The need for a definition of international education in US universities in C. B. Klasek (Ed.), Bridges to the future: Strategies for internationalizing higher education. In (pp. 191-203). Carbondale IL: Association of International Education Administrators.

Australian Universities Quality Agency. (2011). Australian universities quality agency audit manual version 8.0. Melbourne: VIC: Australian University Quality Agency.

Bound, J., Turner, S., & Walsh, P. (2009). Internationalization of US doctorate education No. w14792. In (pp. 1-3). Massachusetts, US: National Bureau of Economic Research.

Brandenburg, U., & Federkeil, G. (2007). How to measure internationality and internationalization of higher education institutions: Indicators and key figures. Gütersloh, Germany: CHE.

Center for Internationalization and Global Engagement. (2023). Comprehensive internationalization framework retrieved from American council on education website. Retrieved from https://www.acenet.edu/Research-Insights/Pages/Internationalization/CIGE-Model-for-Comprehensive-Internationalization.aspx

Chen, X. (2004). Some basic issues on the internationalization of higher education in Beijing higher education society for the introduction of foreign intellectuals (Ed.) international symposium on internationalization of Chinese Universities. In (Vol. 1, pp. 1-15). Beijing: International Symposium on Internationalization of Chinese Universities.

Communist Party of China Central Committee and the State Council. (2020). Overall plan for deepening the reform of education evaluation in the new era retrieved from state council of China website. Retrieved from http://www.gov.cn/zhengce/2020-10/13/content_5551032.htm

Council on Higher Education. (2013).  South Africa the aims of higher education retrieved from council on higher education website. Retrieved from https://www.che.ac.za/file/6431/download?token=1HXDcpUT

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.

Denzin, N. K., & Lincoln, Y. S. (1994). Handbook of qualitative research. Thousand Oaks, CA: Sage.

Francis, A. (1993). Facing the future: The internationalization of post-secondary institutions in British Columbia. Vancouver  BC: British Columbia Centre for International Education.

Gao, Y. (2015). Toward a set of internationally applicable indicators for measuring university internationalization performance. Journal of Studies in International Education, 19(2), 182-200. https://doi.org/10.1177/1028315314559030

General Offices of the Communist Party of China Central Committee and the State Council. (2021). Opinions on promoting the high-quality development of modern vocational education. Retrieved from http://www.gov.cn/zhengce/2021-10/12/content_5642120.htm

Horn, A. S., Hendel, D. D., & Fry, G. W. (2007). Ranking the international dimension of top research universities in the United States. Journal of Studies in International Education, 11(3-4), 330-358. https://doi.org/10.1177/1028315306294630

Jamil, H., & Jaffar, K. (2023). Institutional policies and initiatives for the internationalization of HE: A case of Southeast Asia and Pakistan. In L. Waller & S. Waller (Eds.), Higher education: Reflections from the field. In (pp. 1-19). London: IntechOpen Limited.

Knight, J. (2004). Internationalization remodeled: Definition, approaches, and rationales. Journal of Studies in International Education, 8(1), 5-31. https://doi.org/10.1177/1028315303260832

Krueger, R. A., & Casey, M. A. (2000). Focus groups: A practical guide for applied research. Thousand Oaks, CA: Sage.

Liu, B., & Tan, Y. (2022). Research on the implementation path of “Chinese+vocational skills” overseas education in higher vocational colleges. Vocational Technology, 21(2), 1-5.

Liu, W. (2013). Discussion on the eight-dimensional coordinate system to measure the internationalization degree of higher vocational colleges. Vocational and Technical Education, 34(17), 65-68.

Ministry of Education. (2016). Notice on the internationalization development status survey of China higher education retrieved from ministry of education of China website. Retrieved from http://www.moe.gov.cn/srcsite/A20/s7068/201608/t20160819_275761.html

Ministry of Education. (2020). Proposal on the improvement of vocational education quality of the whole province to build a highland for innovation and development of vocational education retrieved from Shandong provincial government website. Retrieved from http://www.shandong.gov.cn/art/2022/12/2/art_305252_10319838.html

Ministry of Education. (2022). List of national higher education institutions retrieved from ministry of education of China website. Retrieved from http://www.moe.gov.cn/jyb_xxgk/s5743/s5744/A03/202206/t20220617_638352.html

Nurhaeni, I. D. A., Nugroho, R. A., & Kusumawati, N. S. (2019). Developing strategic plan for supporting internationalization of technical and vocational education and training in A. G. Abdullah, I. Kustiawan, I. Widiaty, Ana, & T. Aryanti (Eds.). Paper presented at the Proceedings of the 5th UPI International Conference on Technical and Vocational Education and Training (ICTVET 2018). Dordrecht South Holland Atlantis Press.

Organization for Economic Co-operation and Development. (2023). Vocational education and training and adult learning retrieved from OECD website. Retrieved from https://www.oecd.org/education/innovation-education/vet.htm

Paige, R. M. (2005). Internationalization of higher education: Performance assessment and indicators. Nagoya Journal of Higher Education, 5(8), 99-122.

Pan, X. (2012). The connotation, elements and motivation of the internationalization of higher vocational education. Journal of Vocational Education, 2012(33), 10-15.

Peopletech. (2023). Shandong ranked the second largest population in China. Retrieved from https://baijiahao.baidu.com/s?id=1699438009093133822&wfr=spider&for=pc

Perini, M., & Kämäräinen, P. (2018). Book review: Internationalization in vocational education and training. International Journal For Research in Vocational Education and Training, 5(1), 77-81. https://doi.org/10.13152/ijrvet.5.1.5

Qin, W. (2021). An analysis of the overall internationalization process of Sino-foreign joint education in higher vocational colleges. Journal of Zhejiang Fashion Institute of Technology, 20(2), 70-76.

Scheuch, I., Gessler, M., Bohlinger, S., Zlatkin-Troitschanskaia, O., Greppmair, A.-C., Toepper, M., . . . Kühling-Thees, C. (2021). Research on the internationalization of vocational education and training: Current state and future perspectives. Retrieved from https://www.researchgate.net/profile/Ianina-Scheuch/publication/353002976_Research_for_the_Internationalization_of_Vocational_Education_and_Training_Current_State_and_Future_Perspectives/links/613687fa0360302a008420ac/Research-for-the-Internationalization-of-Vocational-Education-and-Training-Current-State-and-Future-Perspectives.pdf

Shandong Government. (2020). Opinions of 10 Shandong provincial departments on supporting the opening-up of vocational education in the new era. Retrieved from Shandong Provincial Education Department website. Retrieved from http://edu.shandong.gov.cn/art/2020/9/17/art_107094_9801112.html

Shandong Provincial Bureau of Statistics. (2023). Shandong statistics yearbook 2022 retrieved from shandong provincial bureau of statistics website. Retrieved from http://tjj.shandong.gov.cn/tjnj/nj2022/zk/zk/indexch.htm

Soderqvist, M. (2002). Internationalization and its management at higher-education institutions: Applying conceptual, content and discourse analysis. Doctoral Dissertation. Retrieved from https://aaltodoc.aalto.fi/bitstream/handle/123456789/11206/a206.pdf?seq

State Council. (2014). Decision of the state council on accelerating the development of modern vocational education. Retrieved from http://www.gov.cn/zhengce/content/2014-06/22/content_8901.htm

Su, Z., & Wei, H. (2010). The theoretical framework of the evaluation indicator system for the internationalization development of higher vocational colleges. Heilongjiang Researches on Higher Education, 2010(7), 84-86.

Thomas, D. R. (2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27(2), 237-246. https://doi.org/10.1177/1098214005283748

Tran, L. T., & Dempsey, K. (2017). Internationalization in vocational education and training: Transnational perspectives. Cham: Switzerland: Springer.

Valiulis, A. V., & Valiulis, D. (2006). The internationalisation of higher education: A challenge for universities. Global Journal of Engineering Education, 10(2), 221-228.

Van der Wende, M. (1997). Missing links: The relationship between national policies for internationalization and those for higher education in general in T. Kalvermark & M. van der Wende (Eds.), National policies for the internationalization of higher education in Europe. In (pp. 10-13). Stockholm: Sweden: Hogskoleverket Studies, National Agency for Higher Education.

Virkkuen, R., Kontturi, M., Heino, M., Hannus, A., Behm, A.-M., Sandbacka, K., . . . Ruffa, L. (2022). Strategy for internationalization of vocational education and training. Retrieved from https://www.tovet.eu/wp-content/uploads/2022/04/READY-Final-version-of-the-strategy-guidebook.pdf

Wit, H. D., Hunter, F., Howard, L., & Egron-Polak, E. (2015). Internationalization of higher education. In J. Thondhlana, E. C. Garwe, H. d. Wit, J. Gacel-Ávila, F. Huang, & W. Tamrat (Eds.), The Bloomsbury Handbook of the Internationalization of Higher Education in the Global South. Brussel: Bloomsbury Publishing.

Wit, H. d., & Knight, J. (1999). Quality and internationalization in higher education. Paris: Organization for Economic Co-operation and Development.

Xi, J. (2021). Important instructions on vocation education by Xi Jinping retrieved from state council of China website. Retrieved from http://www.gov.cn/xinwen/2021-04/13/content_5599267.htm

Xinhuanet. (2023). Local economic data released. Retrieved from http://www.xinhuanet.com/2023-01/28/c_1211722342.htm

Zhang, M., & Hozer, M. (2001). An overview on qualitative research methods. Chinese Public Administration,  2001(11), 39-42.

Zhou, J. (2017). Assessment of internationalization of higher education: Theoretical contributions and practical implications in NAFSA research symposium series. In (Vol. 1, pp. 31-49). Washington, DC: Association of International Educators.

Zlatkin‐Troitschanskaia, O., & Pilz, M. (2022). Internationalization of vocational education and training—Advances and developments. International Journal of Training and Development, 26(4), 581-584. https://doi.org/10.1111/ijtd.12292