Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The Artificial Intelligence (AI) Certification challenges in Future Single Pilot Operations (SiPO)

1 Purdue University, School of Aviation and Transportation Technology, West Lafayette, IN 47907, USA ↗
2 Coventry University, Faculty of Engineering, Environment and Computing, Coventry, CV1 5FB, UK ↗
Publication year
Digital Object Identifier10.54941/ahfe1003848

Abstract

The aviation industry is characterized by innovation, change management, and human factors implementation in flight operations. The aviation industry anticipates the Single Pilot Operations (SiPO) implementation in commercial airliners. Further de-crewing on commercial airline jets would necessitate using artificial intelligence (AI) in the flight deck to support the pilot duties. This paper outlines human factors and ergonomics (HF/E) certification concerns regarding Human System Integration (HSI). The International Air Transportation Authority's (IATA) Technology Roadmap (IATA, 2019) and the European Aviation Safety Agency's (EASA) Artificial Intelligence (AI) roadmap give an overview and evaluation of current technology trends that will change the aviation environment with the use of AI and the introduction of extended Minimum Crew Operations (eMCO) and Single Pilot Operations (SiPO). A review of the existing research on Artificial Intelligence certification challenges in single pilot operations structured the research themes in cockpit design and users' perception-experience. AI certification challenges in future single pilot operations were examined through interviews with Subject Matter Experts (Human Factors analysts, AI analysts, regulators, test pilots, manufacturers, airline managers, examiners, instructors, qualified pilots, and pilots in training) and questionnaires were sent to a group of professional pilots and pilots in training. In the current regulatory environment, the associated risk-based approach for systems, equipment, and components is primarily driven by a requirements-based "development assurance" methodology during the development of their elements. Although system-level assurance may still necessitate a requirements-based approach, it is acknowledged that design-level layers that rely on learning processes – learning assurance cannot be addressed with only 'development assurance' techniques.Moreover, this research focuses on mitigating residual risk in the 'AI black box.' Results were analyzed and evaluated the Artificial Intelligence (AI) certification and learning assurance challenges under the future single pilot operations aspect.

© 2023. Published by AHFE Open Access. All rights reserved. 629

INTRODUCTION

The new challenges posed by increased air traffic volumes (excluding the Covid-19 distraction) and the growing system’s complexity and the opera- tional environment necessitate a greater emphasis on competitiveness and competency-based training assessment; therefore, implementing new techno- logy (artificial intelligence) could provide lean–six sigma performance-based solutions. It is anticipated that the implementation of new technology will result in new system design and human-machine interactions, which will impact the learning assurance process in terms of:

  • relieving human resources and mental capacity from tasks that a machine can do,
  • allowing reallocation on high added-value tasks, particularly the decision- making process, which critically affects flight safety.

Academic, industrial, and government organizations have collaborated to develop solutions for cockpit workload reduction through initiatives such as Advanced Cockpit for Reduction of Stress and Workload (ACROSS) and Aircrew Labour In-Cockpit Automation System (ALIAS). To alleviate the increased pilot workload, the examined AI systems incorporate knowledge- based capabilities and cognitive and adaptive interfaces (Deutsch, 2005). These are novel ideas in civil aviation, yet they are crucial to the success of SiPO. The system architecture for Extended Minimum-Crew Operations (eMCOs) is presented to facilitate the adoption of SiPO for commercial airliners.

According to the EASA-industrial roadmap, the first certification of assi- stance for pilots is anticipated to occur in 2025. The next step, with a ten-year implementation period, will gradually lead to “complete autonomy”by 2035. The timeline of the industrial roadmap could be summed up in three steps. The first step focuses on crew assistance and augmentation, and a time- frame for implementation has been announced (2022–2025). The second step implements the human-machine collaboration phase from 2025 to 2030, fol- lowing the assistance period. The third step finally introduces and implements autonomous commercial air transport beginning in 2035 (EASA, 2025).

The first and second step of the presented roadmap proposes the two following concepts of operations:

  • Extended Minimum-Crew Operations (eMCOs) — formerly ‘Reduced Crew Operations’— where single-pilot operations are allowed during the cruise phase of the flight, with a level of safety equivalent to today’s two-pilot operations, to be implemented from 2025).
  • Single-Pilot Operations (SiPOs), where, at a later stage, end-to-end single-pilot operations might be allowed, also based on a level of safety equivalent to today’s two-pilot operations, to be implemented from 2030.

The expression ‘black box’ is a specific criticism oriented at AI/ML techni- ques, as the complexity and nature of AI/ML models bring a level of opaqu- eness that make them look like unverifiable black boxes (unlike rule-based software).

The Artificial Intelligence (AI) Certification Challenges 631

Intuitively, the focus of the assurance process needs to move from the accu- racy and completeness/representativeness of the data (training/validation/test data sets) to the learning and its verification. The biggest problem is guaran- teeing that training done on subsets of data can generalize to new data with satisfactory performance.

New approaches to ensuring compliance are needed, and the concept of “learning assurance” is proposed to fill this gap. By revealing as much as possible about the inner workings of AI – SiPO certification, we hope to boost user trust that it can perform as expected.

REVIEWING CERTIFICATION CHALLENGES FOR SINGLE PILOT OPERATIONS Aviation systems SiPO certification should comply with various Accepta- ble Means of Compliance, Guidance Materials, and recommended practices. According to FAR 25.1523 and FAR 25 Appendix D, the criteria for deter- mining minimum flight crew are pilot workload and flight safety when a pilot is incapacitated; in order to obtain certification, SiPO must demonstrate that pilot workload remains at an acceptable level during normal/emergency operations, and that pilot incapacitation does not compromise flight safety. FAR 121 describes the operational requirements for commercial air tran- sport; FAR 121.385(c) stipulates that a minimum of two pilots is required for commercial operations.

Two ICAO documents provide guidance for operations and certification to air operators. The data originates from national and international orga- nizations, such as the International Civil Aviation Organization (ICAO), the United States Department of Defense (DOD), and the Institute for Defense Analyses (IDA). ARINC, ASTM, RTCA, SAE, and NATO Standardization Agreements are also mentioned (STANAGs).

While there are no specific SiPO safety guidelines, the requirements are similar to (and can be derived from) two-pilot operations. FAR 25.1309 and the corresponding Advisory Circular FAR AC 25.1309-1A on demonstrating compliance with fail-safe design are used to derive the system design safety requirements. Aerospace Recommended Practices for avionics system develo- pment and design (ARP-4754A) and safety assessment are provided by SAE (ARP-4761).

The operational and technical requirements for two-pilot aircraft in this category serve as a baseline for future SiPO standards. However, these standards should be altered to address the differences between SiPO and multi-crew operations (MCO), such as:

  • Safety concerns arise when only one pilot is in the cockpit (Weinstein, 1991).
  • Technical and operational requirements for higher levels of ground sup- port (in terms of communication integrity/protocols and work/authority distribution).
  • Human factors and technical requirements for the safe operation of highly autonomous systems (in terms of Human-Machine Interfaces and

Interactions (HMI2) and system integrity/redundancy). Moreover, the AI supplementary to the safety system should follow a certification– verification process considering the learning assurance for the SiPO context.

This study aims to present, identify, and propose the implementation of AI technology in aviation Single Pilot Certification, as well as investigate how AI can impact the transition from multi-crew to eMCO and SiPO, on the premise that a single-pilot human operator having timely and naturally interactive access to data will improve Network Design and Management (NDM) and the relation with learning assurance (Orasanu & Fischer, 1997).

Moreover, the research focused on the relationship between Learning assurance and the certification challenges in SiPO following an Artificial Intelligence technology approach. The study proposes learning assurance for AI systems in connection with the certification of SiPO. As a result, all plan- ned and systematic actions used to demonstrate AI in SiPO will be identified and corrected to the extent that the system satisfies the applicable require- ments at a specified level of performance and provide adequate generalization and robustness guarantees (Ruth, 1982).

The philosophy of interpretivism examined the identified AI technology - areas of interest chosen by theme analysis related to the systematic litera- ture review, surveys, and current AI technologies - based on the case study conducted by the universities of Purdue and Coventry. The framework for the subsequent study approaches and procedures is depicted in Figure 1 (Saunders et al., 2019).

The research followed the single case study strategy. The first step assessed SME (Human Factors analysts, AI analysts, regulators, test pilots, manufa- cturers, airline managers, examiners, instructors, qualified pilots, and pilots in training) perceptions using questionnaires, interviews (Yin, 2014), and literature review under the research questions framework:

  • How do SMEs understand AI learning assurance in aviation?
  • How do SMEs understand traditional V-Cycle in aviation?
  • How do SMEs understand the W-shaped process in aviation?
  • How could the AI’s introduction affect the existing single pilot operations certification practices?
Figure 1: Research framework presentation.
Original PDF Figure 1:

The Artificial Intelligence (AI) Certification Challenges 633

IMPLEMENTATION OF ARTIFICIAL INTELLIGENCE (AI) IN EMCO/SIPO The research team then looked at implementing Artificial Intelligence (AI) technology in the aviation learning assurance process through learning pro- cess verification, traditional V - cycle, and W-shaped model. This was done after conducting a systematic literature review, thematic analysis, and results from interviews and questionnaires. The frequency and relevance of the refe- rences in the literature review demonstrated the importance of the global view of the learning assurance W-shaped model concerning the AI certification procedures (EASA, 2020).

The certification, operational environment, feedback, reporting culture- transparency, flexibility, and learning assurance relationship were also related to the followed taxonomy for AI (Figure 2).

Last but not least, the W-shaped model and the dependability of AI are related (Ruth et al., 1982). The importance of the three identified catego- ries and nine sub-categories of the iterative nature of the learning assurance process in certification is presented in Figure 3.

The quantitative analysis of the opinion surveys and the thematic analy- sis of the interviews validated the systematic literature review results, which indicated a decreasing trend of user resistance when Artificial Intellige- nce implementation in certification was presented as a W-shaped learning assurance model. In the current regulatory framework, the requirements- based ‘development assurance’ technique is the primary driver of the risk-based approach for systems, equipment, and parts during develo- pment. It is acknowledged that the ‘development assurance’ methods can- not handle the design-level layers that rely on learning processes. Howe- ver, the system-level assurance may still necessitate a requirements-based approach.

Figure 2: AI taxonomy in EASA AI roadmap (EASA, 2020).
Original PDF Figure 2:
Figure 3: Iterative nature of the learning assurance process (EASA, 2020).
Original PDF Figure 3:

CONCLUSION

The SiPO is a challenging approach for addressing the anticipated global shortage of qualified flight crews in conjunction with implementing the next- generation CNS+A system. The deployment of such systems is contingent on improving certification and training processes that account for all safety, technical, and regulatory considerations (DARPA, 2014). The suggested AI aviation learning assurance research in SiPO – AI certification and users’ experience was organized based on an examination of the current literature on single pilot operating certification. Results show that the certification shift from Multi-Crew Operations to e-MCO/ SiPO has limits and user resistance related to AI (Liu, 2018). It is essential to consider how to overcome this relu- ctance to change the AI - cockpit design and user interactions. A simple to complicated strategy should be used for the suggested commercially available AI technology application in-flight operations (Stanton & Harris, 2015).

The following findings from this study, which considered the incorporation of AI in the learning assurance related to SiPO certification, are summarized:

  • Adapting assurance frameworks to cover learning processes and addres- sing development errors in AI/ML constituents;
  • Creating a framework for data management to address the correctness (bias mitigation) and completeness/representativeness of data sets used for the ML items training and their verification;
  • Addressing model bias and variance trade-off in the various steps of ML processes.
  • Adapting a Systems Theoretic Early Concept Analysis (STECA) is recom- mended as a follow-up, focusing on a safety-guided design-hazard identi- fication approach.

ACKNOWLEDGMENT

The authors thank faculty members of Purdue and Coventry Universities for their invaluable feedback contributing to this work.

The Artificial Intelligence (AI) Certification Challenges 635

REFERENCES

DARPA, “ALIAS Industry Day Release,” ed: Defense Advanced Research Projects Agency (DARPA), 2014. Deutsch, S., & Pew, R. W. (2005). Single Pilot Commercial Aircraft Operation. EASA. (2020). Artificial Intelligence Roadmap: A human-centric approach to AI in

aviation. Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. FAA, “AC 25.1302–1: Installed Systems and Equipment for Use by the Flightcrew,” ANM-111, Ed., ed. USA: Federal Aviation Administration (FAA), 2013. Human Factors, 37(1), 32–64. https://doi.org/10.1518/001872095779049543 IATA. (2019). Aircraft Technology Roadmap to 2050. Klein, G. (2008). Naturalistic decision making. Human Factors, 50(3), 456–460. Liu, Qiang & Li, Pan & Zhao, Wentao & Cai, Wei & Yu, Shui. (2018). A Survey on

Security Threats and Defensive Techniques of Machine Learning: A Data-Driven View. s.l.: IEEE Access. 6. 12103–12117. 10.1109/ACCESS.2018.2805680, 2018. Orasanu, J., & Fischer, U. (1997). Finding Decisions in Natural Environments: The View From the Cockpit. Ruth, J. C., Godwin, A. M., & Werkowitz, E. B. (1982). Voice Interactive System

Development Program. Advanced Avionics and the Military Aircraft Man/Mach- ine Interface. Saunders, M., Lewis, P., & Thornhill, A. (2019). Research Methods for Business Students Eighth Edition. In Research Methods for Business Students. Pearson Education Limited. Stanton, N. A., & Harris, D. (2015). The future flight deck: Modelling dual, single,

and distributed crewing options. Applied Ergonomics, 53(B), 331–342. https://do i.org/10.1016/j.apergo.2015.06.019. Weinstein, C. J. (1991). Opportunities for advanced speech processing in military computer-based systems. IEEE, 79(11), 1626–1641. https://doi.org/10.1109/5. 118986. White, R. W., Parks, D. L., & Smith, W. D. (1984). Potential flight applications for

voice recognition and synthesis systems. 6th AIAA/IEEE Digital Avionics System Conf., p. 84–2661-CP.

Cover of the proceedings volume Human Factors in Transportation

Explore the complete digital volume

Human Factors in Transportation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-71-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →