Index

Abstract

Situational awareness (SA) is the most important category in single pilot resource management. SA is how a pilot assesses the situation inside and outside the aircraft appropriately to achieve safe and efficient flight safety. SA is an important constituent of a person in information processing and the pilot decision-making process. This study aims to evaluate training in cadet situation awareness. The analysis was based on the aptitude test results as flight training series consisting theoretical stages, simulator and flight practices. Aptitude test is consisting several sections. There were pre-tests and post-tests. Aptitude test results are displayed as cadet score graph, for each section, and the final score will be approached by regression line. Each section average value and the final score are calculated to get changes level in the test score. The results showed an increase in the situational awareness of the cadets based on the aptitude test score.

Keywords: Situational awareness, Pilot cadet, Aptitude test.

Received: 28 March 2022 / Revised: 10 May 2022 / Accepted: 26 May 2022/ Published: 13 June 2022

Contribution/ Originality

This research was first conducted on aviation cadets. This makes the results a reliable tool for prospective pilots’ education system evaluation, before releasing them into the professional world. The data is based on the educational processes series results involving critical situations in the flight process, with computer based pre-posttest.

1. INTRODUCTION

Pilots are a unique occupational group, representing a specific mix of job demands and resources (Demerouti, Veldhuis, Coombes, & Hunter, 2019). Pilots have to store a large amount of important information in their job memory and at the same time deal with other tasks (Novak & Mrazova, 2015). The results of the initial analysis show that there is a direct proportional relationship between the level of task difficulty and the pilot's control behavior (Hebbar & Pashilkar, 2017). Since flying an airplane is a complex job, it is necessary to consider many physiological and psychological factors to evaluate a pilot's cognitive workload (Hebbar, Bhattacharya, Prabhakar, Pashilkar, & Biswas, 2021). Technical training with impaired cues resulted in critical behavioral differences compared to uninterrupted training (Grundy, Nazar, O’Malley, Mohrenshildt, & Shedden, 2016). The results of a study indicate that the psychological training applied does not affect the pilot's performance in terms of precision and accuracy (Socha et al., 2020). Situational awareness (SA) is the most important category in single pilot resource management. SA is how a pilot assesses the situation inside and outside the aircraft appropriately to achieve safe and efficient flight safety (Im, Kim, & Hong, 2021). During flight, pilots must closely monitor flight instruments as one of the important activities that contribute to updating their situation awareness (Lounis, Peysakhovich, & Causse, 2021). SA is an important constituent of a person in information processing and important in the pilot decision-making process (Nguyen, Lim, Nguyen, Gordon-Brown, & Nahavandi, 2019). Situation awareness seeks to answer basic questions about “what has happened”, “what is happening”, or “what will happen”, with the aim of achieving goals such as safety and adjustment (Irwin & Kelly, 2021). SA is a specific representation of the cognitive state and the essence of pilot behavior which includes the cognitive processes of perception, understanding, and prediction (Jiang, Chen, & Kang, 2021). SA is a leading paradigm in studying the human factor as a source of knowledge and in investigating the effects it produces on interactions with the environment (Dalinger, Smurov, Sukhikh, & Tsybova, 2016).

Accurate situational awareness and precise decision-making of a pilot are the main causes of accidents. The aviation accident analysis report by the International Civil Aviation Organization (ICAO) states that in 2018, 58% were high risk accidents, 96% accidents caused human casualties and 82% caused aircraft damage or destruction. (Moon & Lee, 2020). During flying, flight crews need various types of information because the situation around them changes frequently and requires fast and precise information to respond in a safe manner (Craig, 2012). Pilots who use a system awareness briefing are more accurate in reporting the direction of destination, fuel flow, and voltage, compared to pilots who do not use the guidance system (Winter et al., 2019).

Pilot training and recruitment is of fundamental importance in the aviation industry (Adanov, Macintyre, & Efthymiou, 2020). As many as 40% of airline pilots experience burnout and work psychological factors. Severe fatigue related to pilot performance in simulator training (Demerouti et al., 2019). The human factor is still a big problem in runway failures even though many preventive measures have been taken (Suroso & Revadi, 2019). The use of the simulator allows for border-line situation training that cannot be performed in live flight training (Kalavsky et al., 2017). Pilots perform better in the cabin under conditions of low CO2 concentrations (Allen et al., 2019). These points demonstrate the importance of situational awareness training and performance for a pilot.

The concept of situation awareness is especially interesting in aviation operational settings, which involve the operation and control of complex systems in a dynamic environment (Uhlarik & Comerford, 2002). When instructors train new pilots, they can reinforce the desired and identified characteristics of the cadets, thereby increasing self-awareness. These identified strengths can then be used to help each pilot focus on the positive aspects of their personality and skills (Littman-Ovadia & Raas-Rothschild, 2018). Flight instructors and experienced pilots have long held to the intuitive notion that a successful flight results when a pilot “has the big picture”, and conversely when problems arise due to pilot error, it is due to some aspect of missing or incorrect information (Uhlarik & Comerford, 2002). SA has been studied for development in various professional fields, including education (Beck, Meyer, Kind, & Bhansali, 2015; Norman, 2016) online motorcycle taxi driver (Ma'ruf & Jatmiko, 2020) team work (O’Neill, White, Delaloye, & Gilfoyle, 2018; Ouverson et al., 2021) network safety (Li, Huang, Wang, & Li, 2019; Zheng, 2020) and maritime fields (Atik, 2019; Everwyn, Zanuttini, Mouaddib, Gatepaille, & Brunessaux, 2019; Sætrevik, 2013) . Meanwhile, research that study the efforts to improve SA in the aviation sector has been carried out, among others, for the implementation of training programs (Littman-Ovadia & Raas-Rothschild, 2018) SA modelling (Hooey, Gore, & Wickens, 2011; Hooey et al., 2011) relationship to pilot decision making (Kozuba, Sirko, & Pila, 2021) development by Human Centered Design method (Craig, 2012) general aviation training (Bolstad, Endsley, Howell, & Costello, 2002; Endsley & Garland, 2000; Muehlethaler & Knecht, 2016) SA aspects (Kozuba & Piľa, 2015) application of audio tactile(Brill, Lawson, & Rupert, 2015) human-in-the-loop study(Brandt, Lachter, Battiste, & Johnson, 2015) assessment method (Dalinger et al., 2016) training module efficiency for new pilots (Gayraud, Matton, & Tricot, 2017) System Awareness Briefing testing and effectiveness (Chen, Liu, Pang, Wanyan, & Fang, 2021; Winter et al., 2019) the effect of flying learning training materials (Moon & Lee, 2020) conceptual model development (Irwin & Kelly, 2021) correlation evaluation for simulation technology development (Jiang et al., 2021) comparative analysis of agent-based and population-based models (Nasar & Jaffry, 2018) SA loss detection (Pickard, Eidels, Beh, & Blaha, 2021) and the techniques, challenges and prospects of SA air force pilots (Munir, Aved, & Blasch, 2022). Akademi Penerbang Indonesia (API) Banyuwangi (Indonesian Pilot Academy in Banyuwangi district) is an educational institution in the field of aviation that educates and trains pilot cadets. Education and training are carried out using a Cessna 172 SP trainer. A series of education and training are carried out in the training, one of which also leads to the SA aspect. For that we need a study that aims to evaluate the situation awareness training of these prospective pilot cadets.

Figure 1 shows the process of achieving flying safety at the Indonesia Pilot Academy Banyuwangi. Cadets receive situational awareness training that is simulated and conditioned according to the actual flying situation and conditions. The results of training and appropriate flight practice are tested and evaluated using computer-based tests.

2. METHOD

Conditions and situations in a flight operation are very complex. This requires the optimal performance of a pilot in operating an aircraft. Pilot cadets or cadets require a flight operations situation awareness training. This training aims to enable cadets to have a high level of situation awareness when they become pilots. For this reason, an evaluation stage is needed that examines the level of awareness of the situation of the cadets as a measure of the success of the training program. This evaluation stage is a tactical step in determining the level of learning achievement in situation awareness training.

The subjects in this study were cadets of the Indonesian Aviation Academy Banyuwangi which consisted of 46 cadets. The analysis was carried out based on the results of the aptitude test as part of the flight training report and was carried out in 2 stages. This test is a series of learning activities for theory, simulator practice and direct flight practice. This test is called the aptitude test. The aptitude test is a form of representation of situational awareness. This test includes 4 sections based on the exercise performed.

Section 1 is a straight and level section where cadets are assessed in controlling the aircraft at a certain altitude. Section 2 is the rate and turn section where cadets are assessed for controlling aircraft speed and controlling turn movements. Section 3 is the climb section, which is the control of the aircraft during its ascending motion. Section 4 is the descending section, which is the control of the aircraft when it descends.

The test assessment form is as follows:

The scoring is A = 3, B = 2 and C = 1. The final score is determined by the following equation:

The results of data processing aptitude test scores are displayed separately for each test section. The final score is calculated according to Equation 1. These separate data and final scores are displayed in graphical form with point notation for the achieved scores. The graph is equipped with a regression line as a representation of the trendline of the data.

3. RESULT AND DISCUSSION

The results of processing the aptitude test score data for each test section are shown in Table 1 . Figures 2 to 4 . These graphs display the scores of the cadets obtained after taking the pre-test and post-test. This score is a direct score in the form of numbers according to the test assessment guide. The final score is shown in Figure 5 . This score is the result of calculating the final score according to Equation 1 .

The graphs in Figures 2 to 5 show an increase in scores from the pre-test stage to the post-test stage. With the regression line approach, it can be seen that the score increases in general. This increase is the achievement of the cadets where the increase in test results is obtained from the material obtained during education and training. For each section in the aptitude test, the score has increased so that the education and training materials have improved the abilities and expertise of the cadets.

Figure 6 shows the increase in the final score of the aptitude test results from the pre-test stage to the post-test stage. This is in accordance with the increase that occurs in each test section because the final score is the product of the calculation based on the score in each section. The regression line approach shows a clear increase between these two final scores. The distance between the two regression lines indicates that each cadet experienced an increase in the final score.

Section	1st Section	2nd Section	3rd Section	4th Section	Final score
Pre test	2.16	2.10	1.99	2.06	69.18
Post test	2.43	2.37	2.32	2.28	78.21
Differences	0.27	0.27	0.33	0.23	9.03

If the average value is calculated, then the scores for each section and the final score can be seen at Table 2. The average score for each section has increased. This is in accordance with the graphs in Figures to Figure 5 which show the scores of each cadet for each section in the aptitude test. The final score as a result of calculating the score for each section also shows an increase where if it is percentage, the increase in the final score is 13.05%.

Based on the final score, with a passing grade of 60, the pre-test cadets who scored less than the passing grade were 12 people (26.09%). As for the post test, those who scored below the passing grade were reduced to 3 people (6.52%). This shows that the cadets can improve their test results based on the education and training they have received. increase in graduation based on passing grade by 19.57%.

The results of the increase in the score of each section and the increase in the final score indicate that education and training at the Indonesian Aviation Academy Banyuwangi (IAAB) has been able to improve the skills and abilities of cadets. The increase in the results of this aptitude test indicates that the situational awareness of the cadets has experienced a relatively significant increase. The education and training provided to the cadets can improve the situational awareness of the cadets who at the pre-test stage have not met the passing grade score. In general, the increased situational awareness of the cadets is enhanced by education and training.

4. CONCLUSION

The situational awareness of the Indonesian Aviation Academy Banyuwangi (IAAB) cadets has increased according to the aptitude test results. This increase occurred with the education and training provided. Further research related to education and training materials that contribute to optimal increase needs to be carried out as an effort to improve the education and training system.

Funding: This research is supported by Indonesia Pilot Academy Research Grant 2022

Competing Interests: The authors declare that they have no competing interests.

Authors’ Contributions: All authors contributed equally to the conception and design of the study.

REFERENCES

Adanov, L. M., Macintyre, A., & Efthymiou, M. (2020). An exploratory study of pilot training and recruitment in Europe. International Journal of Aviation Science and Technology, 1(02), 44-51.Available at: https://doi.org/10.23890/ijast.vm01is02.0201.

Allen, J. G., MacNaughton, P., Cedeno-Laurent, J. G., Cao, X., Flanigan, S., Vallarino, J., & Spengler, J. D. (2019). Airplane pilot flight performance on 21 maneuvers in a flight simulator under varying carbon dioxide concentrations. Journal of Exposure Science & Environmental Epidemiology, 29(4), 457-468.Available at: https://doi.org/10.1038/s41370-018-0055-8.

Atik, O. (2019). Eye tracking for assessment of situational awareness in bridge resource management training. Journal of Eye Movement Research, 12(3), 1–18.Available at: https://doi.org/10.16910/jemr.12.3.7.

Beck, J., Meyer, R., Kind, T., & Bhansali, P. (2015). The importance of situational awareness: A qualitative study of family members’ and nurses’ perspectives on teaching during family-centered rounds. Academic Medicine, 90(10), 1401-1407.Available at: https://doi.org/10.1097/acm.0000000000000810.

Bolstad, C. A., Endsley, M. R., Howell, C. D., & Costello, A. M. (2002). General aviation pilot training for situation awareness: An evaluation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 46(1), 21–25.Available at: https://doi.org/10.1177/154193120204600105.

Brandt, S. L., Lachter, J., Battiste, V., & Johnson, W. (2015). Pilot situation awareness and its implications for single pilot operations: Analysis of a human-in-the-loop study. Procedia Manufacturing, 3, 3017-3024.Available at: https://doi.org/10.1016/j.promfg.2015.07.846.

Brill, J. C., Lawson, B. D., & Rupert, A. H. (2015). Audiotactile aids for improving pilot situation awareness. Paper presented at the 18th International Symposium on Aviation Psychology.

Chen, H., Liu, S., Pang, L., Wanyan, X., & Fang, Y. (2021). Developing an improved ACT-R model for pilot situation awareness measurement. IEEE Access, 9, 122113-122124.Available at: https://doi.org/10.1109/access.2021.3108438.

Craig, C. (2012). Improving flight condition situational awareness through human centered design. Work, 41(Supplement 1), 4523-4531.Available at: https://doi.org/10.3233/wor-2012-0031-4523.

Dalinger, I., Smurov, M., Sukhikh, N., & Tsybova, E. (2016). Pilot's situational awareness and methods of its assessment. Indian Journal of Science and Technology, 9(46), 1-5.Available at: https://doi.org/10.17485/ijst/2016/v9i46/107534.

Demerouti, E., Veldhuis, W., Coombes, C., & Hunter, R. (2019). Burnout among pilots: Psychosocial factors related to happiness and performance at simulator training. Ergonomics, 62(2), 233-245.Available at: https://doi.org/10.1080/00140139.2018.1464667.

Endsley, M. R., & Garland, D. J. (2000). Pilot situation awareness training in general aviation. Paper presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting Sage CA: Los Angeles, CA: SAGE Publications.

Everwyn, J., Zanuttini, B., Mouaddib, A. I., Gatepaille, S., & Brunessaux, S. (2019). Achieving maritime situational awareness using knowledge graphs: A study. Paper presented at the 1st Maritime Situational Awareness Workshop (MSAW 2019).

Gayraud, D., Matton, N., & Tricot, A. (2017). Efficiency of a situation awareness training module in initial pilot training. Paper presented at the 19th International Symposium on Aviation Psychology.

Grundy, J. G., Nazar, S., O’Malley, S., Mohrenshildt, M. V., & Shedden, J. M. (2016). The effectiveness of simulator motion in the transfer of performance on a tracking task is influenced by vision and motion disturbance cues. Human Factors, 58(4), 546-559.Available at: https://doi.org/10.1177/0018720816639776.

Hebbar, P. A., & Pashilkar, A. A. (2017). Pilot performance evaluation of simulated flight approach and landing manoeuvres using quantitative assessment tools. Sādhanā, 42(3), 405-415.Available at: https://doi.org/10.1007/s12046-017-0613-0.

Hebbar, P. A., Bhattacharya, K., Prabhakar, G., Pashilkar, A. A., & Biswas, P. (2021). Correlation between physiological and performance-based metrics to estimate pilots’ cognitive workload. Frontiers in Psychology, 12, 1-16.Available at: https://doi.org/10.3389/fpsyg.2021.555446.

Hooey, B. L., Gore, B. F., & Wickens, C. D. (2011). Human modelling in assisted transportation (1st ed.). Milano: Springer.

Hooey, B. L., Gore, B. F., Wickens, C. D., Scott-Nash, S., Socash, C., Salud, E., & Foyle, D. C. (2011). Modeling pilot situation awareness. In: Cacciabue, P., Hjälmdahl, M., Luedtke, A., Riccioli, C. (eds) Human Modelling in Assisted Transportation. Milano: Springer.

Im, K. H., Kim, W., & Hong, S. J. (2021). A study on single pilot resource management using integral fuzzy analytical hierarchy process. Safety, 7(4), 1–15.Available at: https://doi.org/10.3390/safety7040084.

Irwin, W., & Kelly, T. (2021). Airline pilot situation awareness: Presenting a conceptual model for meta-cognition, reflection and education. Aviation, 25(1), 50-64.Available at: https://doi.org/10.3846/aviation.2021.14209.

Jiang, S., Chen, W., & Kang, Y. (2021). Correlation evaluation of pilots’ situation awareness in bridge simulations via eye-tracking technology. Computational Intelligence and Neuroscience, 1-15.Available at: https://doi.org/10.1155/2021/7122437.

Kalavsky, P., Rozenberg, R., Socha, L., Socha, V., Gazda, J., & Kimlickova, M. (2017). Methodology of pilot performance measurements. MAD-Magazine of Aviation Development, 5(2), 25-30.Available at: https://doi.org/10.14311/mad.2017.02.04.

Kozuba, J., & Piľa, J. (2015). Selected aspects of pilot situational awareness. OUR SEA: Scientific Journal of the Sea and Maritime Affairs, 62(3 Special Issue), 175-180.

Kozuba, J., Sirko, S., & Pila, J. (2021). Consequences and recurrence of dangerous occurrences in air transport in 1919-2018. Safety & Defense, 7(3), 42-55.Available at: https://doi.org/10.37105/sd.103.

Li, Y., Huang, G.-Q., Wang, C.-Z., & Li, Y.-C. (2019). Analysis framework of network security situational awareness and comparison of implementation methods. EURASIP Journal on Wireless Communications and Networking, 2019(1), 1-32.Available at: https://doi.org/10.1186/s13638-019-1506-1.

Littman-Ovadia, H., & Raas-Rothschild, E. (2018). Character strengths of airline pilots: Explaining life and job satisfaction and predicting CRM performance. Psychology, 9(8), 2083-2102.Available at: https://doi.org/10.4236/psych.2018.98119.

Lounis, C., Peysakhovich, V., & Causse, M. (2021). Visual scanning strategies in the cockpit are modulated by pilots’ expertise: A flight simulator study. PLoS one, 16(2), e0247061.Available at: https://doi.org/10.1371/journal.pone.0247061.

Ma'ruf, F., & Jatmiko, H. A. (2020). Analysis of situational awareness for online taxibike driver in Yogyakarta using QUASA analysis. Scientific Journal of Industrial Engineering, 19(1), 55-63.Available at: https://doi.org/10.23917/jiti.v19i1.10402.

Moon, J. Y., & Lee, J. R. (2020). The effect of studying flight training materials utilizing encoding techniques on situational awareness capabilities of students in PPL training. Journal of the Korean Society for Aviation and Aeronautics, 28(4), 154-163.Available at: https://doi.org/10.12985/ksaa.2020.28.4.154.

Muehlethaler, C. M., & Knecht, C. P. (2016). Situation awareness training for general aviation pilots using eye tracking. IFAC-PapersOnLine, 49(19), 66-71.Available at: https://doi.org/10.1016/j.ifacol.2016.10.463.

Munir, A., Aved, A., & Blasch, E. (2022). Situational awareness: Techniques, challenges, and prospects. AI, 3(1), 55-77.Available at: https://doi.org/10.3390/ai3010005.

Nasar, Z., & Jaffry, S. W. (2018). Trust-based situation awareness: Comparative analysis of agent-based and population-based modeling. Complexity, 2018.Available at: https://doi.org/10.1155/2018/9540726.

Nguyen, T., Lim, C. P., Nguyen, N. D., Gordon-Brown, L., & Nahavandi, S. (2019). A review of situation awareness assessment approaches in aviation environments. IEEE Systems Journal, 13(3), 3590–3603.Available at: https://doi.org/10.1109/jsyst.2019.2918283.

Norman, H. (2016). Modeling learner situation awareness in collaborative mobile web 2.0 learning. Malaysian Online Journal of Educational Technology, 4(1), 32–56.

Novak, A., & Mrazova, M. (2015). Research of physiological factors affecting pilot performance in flight simulation training device. Communications-Scientific letters of the University of Zilina, 17(3), 103-107.Available at: https://doi.org/10.26552/com.c.2015.3.103-107.

O’Neill, T. A., White, J., Delaloye, N., & Gilfoyle, E. (2018). A taxonomy and rating system to measure situation awareness in resuscitation teams. PLoS one, 13(5), e0196825.Available at: https://doi.org/10.1371/journal.pone.0196825.

Ouverson, K. M., Ostrander, A. G., Walton, J., Kohl, A., Gilbert, S. B., Dorneich, M. C., & Sinatra, A. M. (2021). Analysis of communication, team situational awareness, and feedback in a three-person intelligent team tutoring system. Frontiers in Psychology, 12(3), 1–16.Available at: https://doi.org/10.3389/fpsyg.2021.553015.

Pickard, S. R., Eidels, A., Beh, E. J., & Blaha, L. M. (2021). Detecting a loss of situational awareness. Paper presented at the 75th International Symposium on Aviation Psychology.

Sætrevik, B. (2013). Developing a context-general self-report approach to measure three-level situation awareness. International Maritime Health, 64(2), 66-71.

Socha, V., Socha, L., Hanáková, L., Karapetyan, L., Valenta, V., Kušmírek, S., & Olexa, P. (2020). Effect of psychological training on pilot’s performance. Transportation Research Procedia, 51, 252-260.Available at: https://doi.org/10.1016/j.trpro.2020.11.028.

Suroso, H. C., & Revadi, C. E. (2019). The main factors that affect pilot attention and decision making during landing operation leading to runway incursion. Paper presented at the Proceedings of the 2019 1st International Conference on Engineering and Management in Industrial System (ICOEMIS 2019).

Uhlarik, J., & Comerford, D. A. (2002). A review of situation awareness literature relevant to pilot surveillance functions (pp. 1-28). Technical Report DOT/FAA/AM-02/3. Kansas State University, Manhattan.

Winter, S. R., Walters, N. W., Milner, M. N., Garcia, D., Anania, E. C., Baugh, B. S., & Rice, S. (2019). Examining the perception and effectiveness of a system awareness briefing during cruise flight. Collegiate Aviation Review, 37(1), 1–18.Available at: https://doi.org/10.22488/okstate.19.100201.

Zheng, W. (2020). Research on situation awareness of network security assessment based on Dempster-Shafer. Paper presented at the MATEC Web of Conferences.

Views and opinions expressed in this article are the views and opinions of the author(s), World Journal of Vocational Education and Training shall not be responsible or answerable for any loss, damage or liability etc. caused in relation to/arising out of the use of the content.