Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Implementation Challenges of Immersive Technologies in Transportation Simulation

1 Purdue University, School of Aviation and Transportation Technology, West Lafayette, IN 47907, USA ↗
2 Purdue University, School of Aviation and Transportation Technology, West Lafayette IN 47907, USA ↗
3 Coventry University, Faculty of Engineering, Environment & Computing, Coventry CV1 5FB, U.K ↗
Publication year
Digital Object Identifier10.54941/ahfe1004448

Abstract

Innovation, effective management of change, and integrating human factor elements into flight operations control distinguishing features of the aviation sector. Immersive technologies (Augmented, Virtual, and Mixed Reality – Digital Twins technology) can be used in aviation training programs to provide an immersive and interactive learning experience for all aviation professionals. Adapting an aviation immersive technology environment in transportation simulation can allow the implementation of new training approaches in a safe and controlled environment without the risk of actual flight or equipment damage. Digital twins are used to create realistic flight simulations, allowing aviation ecosystem actors to practice their skills in various scenarios and conditions. This helps to improve safety and prepare aviation experts for unexpected events during actual flight. Another use for Augmented, Virtual, and Mixed Reality Simulation in aviation training programs is maintenance training. Moreover, Digital Twins can simulate maintenance procedures on aircraft and aviation systems, allowing SMEs to enhance their knowledge and practice their skills in a safe, cost-effective, and controlled environment. Purdue University School of Aviation and Transportation Technology (SATT) Ecosystems' Artificial Intelligence (AI) research roadmap aims to introduce digital twins in aviation training programs to simulate flight-airport operations and air traffic scenarios. Moreover, Purdue's Artificial Intelligence approach for Augmented, Virtual, and Mixed Reality Simulation / Digital Twins focuses on the potential to improve the effectiveness and efficiency of aviation training programs (CBTA globally) by providing a more realistic and immersive learning experience {lean process for training/certification, transition to AI – Advanced Air Mobility (AAM) environment}. Furthermore, this research focuses on implementing challenges and mitigating residual risk in the 'AI black box.' Results were analyzed and evaluated the Artificial Intelligence certification and learning assurance challenges under the Augmented, Virtual, and Mixed Reality Simulation – Digital Twins aspects.

© 2023. Published by AHFE Open Access. All rights reserved. 125

INTRODUCTION

The utilization of Augmented, Virtual, and Mixed Reality – Digital Twins technology in Transportation Simulation enhances and monitors learners’ performance at individual and cohort levels. Purdue SATT Artificial Intelli- gence technology roadmap aims to enhance training programs by streamlin- ing and personalizing them in an effective -efficient and sustainable approach. The term “implement” refers to the act of putting a plan, idea, or policy into action.

The primary advantages of utilizing the AI SATT platform (Augmented, Virtual, and Mixed Reality – Simulated Air Traffic Control Environment - Digital Twins) incorporate enhancement of training processes, minimiz- ing expenses, augmenting training efficacy, forecasting training requirements for upcoming cohorts of learners, and bolstering the capacity to fulfill future training demands objectively and effectively. The effectiveness of AI SATT training platform is in its ability to remain impartial towards con- tent, enabling it to extract valuable insights and offer recommendations from diverse sources and at different levels. The utilization of a metadata-driven data intake approach is essential for the successful integration of diverse data sources. The AI SATT platform’s primary objective is to optimize the univer- sity academic - training program by offering the most efficient and effective pathway while simultaneously guaranteeing that all performance standards (Student Learning Objectives – AABI) are achieved.

Purdue School of Aviation and Transportation Technology’s (SATT) learn- ing trajectory is consistently improved by student engagement and teacher– instructor–facilitator contributions. Implementing a value-added adaptive training system can be facilitated by utilizing artificial intelligence (AI) and machine learning (ML) techniques to continuously monitor and analyze inte- grated student learning objectives, training content, and learner profiles. Adaptive training represents a significant invention with the potential to enhance the cognitive aptitude for operating intricate systems, such as aircraft flight. Flight simulators offer a comprehensive and immersive environment that facilitates training and provides robust support. Furthermore, generat- ing and collecting substantial data can facilitate the development of artificial intelligence models, enabling the assessment and adjustment of training pro- cesses with a suitable degree of comprehensibility. Purdue SATT digital model is created to facilitate adaptation among pilots, instructors, and air- craft systems within diverse learning environments by employing machine learning techniques, neuroscience principles, and mathematical optimization methods.

Purdue SATT proposed AI technical advancements in flight training are divided into five major thematic areas:

  • Adaptive learning and training analytics powered by artificial intelligence (AI).
  • Training-related behavioral and cognitive methods – Science, Technology, Engineering, and Mathematics, (STEM)
  • AI training operations optimization
  • Flight Operations and Closed-loop Flight Training

The Implementation Challenges of Immersive Technologies 127

  • Independence in Advanced Air Mobility.

METHODOLOGY

The proposed research philosophy in aviation aligns with the ADDIE (Anal- yse -Design-Develop-Implement-Evaluate) methodology, which incorporates the utilization of immersive technology in the aviation ecosystem, AI Stu- dent Learning Objectives (SLO), Aviation Accreditation Board International (AABI) standards and Competency-Based Training and Assessment (CBTA) framework. The present analysis aims to examine the implementation chal- lenges of SATT Artificial Intelligence (AI) research roadmap in learning objectives of flight school students by utilizing data obtained from the Evidence-Based Training (EBT) – Competency Based Training Assessment (CBTA) IATA analysis and AI features, (IATA, 2021).

Purdue research analyzed the Federal Aviation Administration’s (FAA) 14 CFR Part 60, which pertains to the initial qualification and use of flight simulation training devices. Additionally, the ongoing research examines the International Civil Aviation Organization’s (ICAO, 2015) 9625 standards and their relevance to training requirements in the aviation industry. The Training Needs Analysis (TNA) is selected as a systematic process used to identify and assess the training requirements of individuals or groups within an organization. It involves gathering and analyzing data to determine the knowledge, skills, and attitudes of future aviation experts.

Following Honour’s (2006) Task Network Analysis (TNA), the Purdue research team categorized complex tasks into a series of sub-tasks, each of which has a clearly defined AI identity and student learning objective. To effectively comprehend and execute each subtask, the Purdue team identi- fies the requisite skills associated with each subtask, such as reproductive skills, decision-making skills, behavioral skills, and so on, following AABI and CBTA taxonomy.

The Purdue Honours’ Task Network Analysis in Aviation Training exam- ined the following:

1. Cognitive Task Analysis: Examines pilots’ cognitive processes during flight training. It aims to capture the knowledge and decision-making processes involved in aviation operations. 2. Understanding Expertise: Identifies the strategies and heuristics they employ to handle complex situations effectively. This knowledge is then applied to pilot training to enhance the development of expertise. 3. Scenario-Based Training: TNA helps in developing scenario-based train- ing programs. By understanding how expert pilots manage situations (Evidence Based Training / EBT), training scenarios can be designed to expose learners to similar situations and teach them how to handle them effectively. 4. Decision-Making Models: TNA provides insights into the decision- making models used by expert pilots. This information is valuable for training novice pilots to develop sound decision-making skills, especially in high-stress situations.

5. Crew Resource Management (CRM): TNA is used to develop CRM training programs, emphasizing effective communication and teamwork among flight crews. It helps identify the critical cognitive and behavioral components that influence CRM. 6. Training Materials: TNA findings can be used to create training mate- rials, including checklists, procedural manuals, and training scenarios, which align with Immersive Technologies, expert pilot strategies, and best practices. 7. Simulation Training: TNA supports the development of flight sim- ulators and training devices implementing immersive technologies in real-world situations. Simulators can be used to expose pilots to vari- ous scenarios and practice their decision-making skills in a controlled environment. 8. Human Factors Integration: TNA considers human factors aspects, such as workload, situational awareness, and error management, in avia- tion training. It helps ensure that training programs address the critical human performance factors relevant to flight safety. 9. Continuous Improvement: TNA can be used for post-incident analysis and continuous improvement in aviation training. It helps identify areas for enhancement and refinement in training programs and procedures. 10. Safety and Risk Mitigation: By understanding how pilots process infor- mation and make decisions, TNA contributes to identifying and mitigat- ing risks in aviation operations, ultimately enhancing safety, following a Threat Error Management (TEM) – Safety II approach.

Additionally, it is important to determine the minimum levels of fidelity necessary to achieve the desired learning objectives, as outlined by Chapanis (1996).

This study aims to explore learning areas of interest that are related to oper- ations by employing a competency-based training and assessment (CBTA) approach that incorporates artificial intelligence (AI) and follows a thematic academic framework. The introduction of Machine Learning/Deep Learn- ing (ML/DL) software is currently underway. The learning areas adhere to the requirements set forth by the Aviation Accreditation Board International (AABI, 2023).

The implementation challenge lies in enhancing the connection between training tasks and the corresponding tools in the following areas:

  • Accommodate the utilization of modern training tools under the CBTA concept
  • Facilitate enhanced standardization of utilized training tools.

The proposed SATT training syllabus (AI SATT platform) aims to include cutting-edge technologies in aviation training. These technologies include Vir- tual Reality (VR), Mixed Reality (MR), Augmented Reality (AR), Simulated Air Traffic Control Environment, Digital Twins, artificial intelligence (AI) instructor, eye tracking, and their use in various stages of aviation training, ranging from Ab-initio to Type Rating and recurrent levels. The objective is to

The Implementation Challenges of Immersive Technologies 129

enhance or expand upon a certain concept or idea respecting the technology culture of each organization and following Kotter’s 8-step change model.

Figure 1: Adaptation of Kotter’s eight-step process in leading change for AI implemen- tation (Kotter, 1996).
Original PDF Figure 1:

The Purdue research protocol presents virtual reality (VR) applications for both civil and military purposes as the initial stage of VR implementation. SATT, in collaboration with VRpilot, provides Purdue college students with a comprehensive virtual reality (VR) training program that specifically caters to professional flight training. This VR learning experience is designed to effi- ciently address specific training objectives, ensuring rapid implementation. The subject of this training is Normal Procedures for the A-320 and B-737 aircraft models. The training session focuses on abnormal situations and the corresponding emergency procedures and memory items. The main challenge of the implementation of AI in the current phase is the connection of the train- ing syllabus with Competency-Based Training and Assessment (CBTA) in the context of academic aviation training.

The process of eye tracking monitoring involves the observation and anal- ysis of eye movements in order to gain insights into human behavior and cognition. One specific aspect of eye tracking monitoring is the identifica- tion and analysis of errors that occur during eye movements under a specific flight task (Hillmann, 2021). This allows the Purdue research team to imple- ment CBTA in the process of debriefing utilizing a systematic and structured approach to gather information and provide feedback following a specific training event or activity.

The implementation of skills and standardization of immersive technolo- gies at Purdue University facilitates the integration of content efficiency, teaching experience, and the demands of the aviation market. The AT- 38800 course, which pertains to Large Aircraft Systems, was revamped by SATT with the intention of acquiring students with VR training courses that specifically emphasize the A-320/B737 aircraft systems design and philosophy.

The research protocol for integrating artificial intelligence into aviation training on a global scale was established by the Purdue SATT HSI team.

This protocol may be seamlessly integrated into any systems engineering test plan, allowing for the implementation of AI and subsequent evaluation of its effectiveness. The specialists from the Systems Analysis and Test Techniques (SATT) team published the first handbook on the implementation of AI in aviation in September 2023. This handbook collaboration is based on the uti- lization of systems engineering processes and methodologies, which are being employed to effectively integrate human systems within the overall frame- work. The primary objective of this handbook collaboration is to ensure the achievement of successful AI human systems integration in a global CBTA context.

In addition, Purdue’s research demonstrates a strong correlation between reducing operating and training costs and implementing the CBTA-lean methodology (Ziakkas, 2022).

Figure 2: Purdue lean six sigma dimensions (Ziakkas, 2022).
Original PDF Figure 2:

FINDINGS The proposed SATT training syllabus (AI SATT platform) aims to include cutting-edge technologies in aviation training that adhere to the skills spec- ified in the CBTA International Civil Aviation Organization (ICAO) and Aviation-Based Behavioural Indicators (ABBI) criteria.

Following the ADDIE approach, The Purdue research team has identified the following challenges in the implementation of AI in aviation:

  • Analyze: ▶Flight school student learning objectives using the data acquired on Competency Based Training Assessment (CBTA) ICAO analysis, AABI, and VR elements. ▶The present study analyzes the Federal Aviation Administration (FAA)

14 CFR Part 60, which pertains to the initial qualification and use of flight simulation training devices. This analysis also considers the International Civil Aviation Organization (ICAO) 9625 standards and their relevance to training requirements in the aviation industry. The concept of Training Needs Analysis (TNA) refers to the systematic process of identifying and assessing the training requirements of

The Implementation Challenges of Immersive Technologies 131

individuals or groups within an organization. TNA involves gathering and analyzing data to determine the training performance gaps.

  • Design: ▶Applying a thematic, evidence-based, CBTA-AI method. ▶Courses are designed following standards set by the Aviation

Accreditation Board International (AABI). ▶Improve the connection between training activities and Immersive Technologies. ▶Support cutting-edge instructional resources ▶Incorporate the learning goals as a set of scenarios. ▶Curriculum updates to reflect the integration of emerging technologies into flight instruction: Advanced Air Mobility features with aviation partners and regulators - Introduction of Digital Twins in the research of airspace requirements; Virtual/Mixed/Augmented Reality, artificial intelligence instructor, eye tracking, for Aviation training from Ab-initio to Type Rating and recurrent level.

  • Develop: Purdue research protocol includes modules on Normal Procedures Train- ing for the A-320 and B-737 aircraft, as well as Abnormal and Emergency Procedures Training with a focus on memory items. Additionally, there is a module dedicated to cockpit familiarization, which includes flow visual- izations. The immersive technologies training also incorporates multi-crew remote training, allowing for remote collaboration among crew members. For solo training, an automatic crewmember is available. The training pro- gram includes an exam based on Competency-Based Training and Assessment (CBTA). To enhance task sharing, the program emphasizes dynamic pilot fly- ing and pilot monitoring. Eye tracking monitoring is used to detect errors, and debriefing sessions are conducted to review performance.
  • Implement: The implementation of skills and the standardization of immersive tech- nologies at Purdue University enables the integration of content efficiency, teaching experience, and the fulfilment of aviation market needs. The AT- 38800 course, which pertains to Large Aircraft Systems, was revamped by SATT to provide students with an enhanced understanding of VR training courses that specifically concentrate on the A-320 and B737 aircraft models. In continuation of the release of the “Implementation Guide for Artificial Intelligence in Aviation: A Human-Centric Guide for Practitioners and Orga- nizations”, (Ziakkas et al, 2023), Purdue SATT is currently prioritizing the integration of Cognitive Behavioral Training and Assessment (CBTA) inside an artificial intelligence (AI) training setting, with a specific emphasis on the Training Needs Analysis (TNA) findings.
  • Evaluate: To employ AI in aviation training globally and assess results, the Purdue HSI team developed a study methodology that could be readily incor- porated into the systems engineering test schedule. SATT SMEs estab- lished the “Implementation Guide for Artificial Intelligence in Aviation: A Human-Centric Guide for Practitioners and Organizations” network and

collaborated with it utilizing systems engineering concepts and techniques, (Ziakkas et al., 2023).

ANALYSIS Human Resources (HR) experts argue that after identifying shortcomings in HR processes through an examination of the HR performance gap, it is critical to establish job vacancies by developing job descriptions and specifi- cations based on competencies. Following completion of the HR gap analysis, the recruitment strategy must incorporate complete job information that is competency-oriented at all stages, from the first job through the final can- didate selection, considering candidates’ competencies and familiarity with AI – immersive technologies.

Figure 3: Aviation performance gap analysis and change management (Ziakkas, 2022).
Original PDF Figure 3:

Purdue SATT performance gap analysis identifies the TNA – performance requirements considering the EASA AI technology roadmap:

Figure 4: Artificial intelligence in aviation roadmap (EASA, 2023).
Original PDF Figure 4:

The Implementation Challenges of Immersive Technologies 133

Professional flight VR training courses should follow the CBTA competen- cies outlined in the ICAO and ABBI guidelines. Moreover, the implementation of immersive technologies in aviation training including Digital Twins simula- tion, needs a continuous “Operational -Human Factors” evaluation of train- ing with new technologies Vs. traditional ways (ongoing research in Flight Simulators – FSTD A-320/B737 in Purdue Niswonger Simulator Center.

Finally, Purdue continues the research on the use of the ML/DL capabilities in decision-making in Aviation Training and the feasibility of introducing VR training to the Purdue training syllabus from the ab initio phase.

CONCLUSION

The aviation industry is currently looking for new ways to improve the effi- ciency of pilot training. Competency-Based Training and Evaluation (CBTA) is a method of assessing a flight crew’s ability to handle both predicted and unexpected problems.

The goal of this research is to improve training effectiveness by incor- porating immersive technologies in aviation training. The use of dynamic real-time visualization, automatic human profile assessment, and training system adaption technologies has the potential to improve flight training’s overall efficacy and efficiency. This process of digitization includes a vari- ety of immersive virtual technologies and synthetic learning environments. By using these technologies, all persons participating in flight training will obtain a more complete insight into the participants’ performance, ultimately optimizing the training lifecycle.

The identified benefits encompass a multitude of advantageous outcomes, including but not limited to a reduction in instructor hours, the implementa- tion of competency-based training with a focus on evidence-based simulator preparation, enhanced training with increased scheduling flexibility, an aug- mentation in system knowledge (CBTA), an accelerated learning rate, and a notable decrease in negative training instances.

ACKNOWLEDGMENT

The authors thank faculty members of Purdue University for their invaluable feedback contributing to this work.

REFERENCES

AABI. (2023, April 13). https://www.aabi.aero/accreditation/steps-to-accreditation/ Chapanis, A. (1996). Human factors in systems engineering. Wiley Series in Systems Engineering and Management. Andrew Sage, series editor. Hoboken, NJ: Wiley. EASA. (2023). Artificial Intelligence Roadmap: A human-centric approach to AI in aviation. Hillmann, C. (2021). UX for XR: User Experience Design and Strategies for

Immersive Technologies. Apress L. P. Honour, E. C. (2006) “A Practical Program of Research to Measure Systems Engineering Return on Investment (SE-ROI),” Proceedings of the Sixteenth Annual Symposium of the International Council on Systems Engineering, Orlando, FL.

IATA. (2021). CBTA expansion within the Aviation System. 2021.06.02_CBTA expansion within the aviation system_final version. https://www.iata.org/conte ntassets/c0f61fc821dc4f62bb6441d7abedb076/cbta-expansion-within-the-avia tion-system.pdf ICAO. (2015). Manual of Criteria for the Qualification of Flight Simulation Training

Devices - Volume I - Aeroplanes (9625-1) (4th ed., Vol. 1). International Civil Aviation Organization. Kotter, J. P. (1996). Leading change an action plan from the world’s foremost expert on business leadership. Harvard Business Review Press. Ziakkas, D., Plioutsias, A., Pechlivanis, K., 2022. Artificial Intelligence in the avi-

ation decision-making process. The transition from extended Minimum Crew Operations to Single Pilot Operations (SiPO). Artificial Intelligence and Social Computing. AHFE (2022) International Conference. AHFE Open Access, vol 28. AHFE International, USA. http://doi.org/10.54941/ahfe1001452. Ziakkas, D., Vink, L.-S., Pechlivanis, K., & Flores, A. (n.d.). Implementation Guide for Artificial Intelligence in Aviation: A Human-Centric Guide for Practitioners and Organizations. In Amazon. Retrieved October 22, 2023, from https://www.amazon.com/IMPLEMENTATION-GUIDE-ARTIFICI AL-INTELLIGENCE-AVIATION-ebook/dp/B0CL8H14TV/ref=sr_1_1?qid= 1697646031&refinements=p_27%3ADimitrios\protect$\relax+$Ziakkas&s=d igital-text&sr=1-1&text=Dimitrios\protect$\relax+$Ziakkas.

Cover of the proceedings volume Augmented, Virtual and Mixed Reality Simulation

Explore the complete digital volume

Augmented, Virtual and Mixed Reality Simulation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-94-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →