Barbara Holder, Christopher Reed (2026). Action-Oriented Pilot Training. In: Advances in Human Factors of Transportation. AHFE Open Access, vol. 224, pp. 86–95. https://doi.org/10.54941/ahfe1007836
Pilot training aims to cultivate the knowledge and skills needed for proficient pilot performance. These include technical and nontechnical aspects. The technical aspects of performance are well defined and understood. The nontechnical aspects, however, are often defined in vague and subjective terms that make them difficult to reliably train and assess. Several frameworks have been developed to attempt to define the essential nontechnical aspects of pilot performance. These include the Joint Aviation Authority’s (JAA) Non-Technical Skills (NOTECHS), Crew Resource Management (CRM), Threat and Error Management (TEM), and to some degree, Competency-Based Training and Assessment (CBTA). To understand how airlines train and assess pilot performance, data were collected at four major U.S. Airlines and one major European Airline. A series of interviews were conducted in the form of discussion groups and one-on-one interviews with Pilots, Check Pilots, Flight Instructors and airline Training and Standards Managers. Observations of Line-Oriented Flight Training (LOFT) sessions and analyses of airline training materials were also conducted. The results suggest the core issues in training today lie with training and assessing nontechnical aspects of performance and integrating nontechnical with technical performance. While NOTECHS, CRM, TEM, and CBTA provide descriptions of nontechnical aspects of pilots’ work, they do not explicitly specify what the pilots need to do. The aim of this research is to make nontechnical performance explicit. In this paper, a pragmatic shift to action-oriented training is introduced. The action-oriented training framework was developed by applying modern cognitive theories to training and assessment.
Pilots have been aviating for over a century, and much has been learned about the skills and knowledge pilots need to safely and effectively operate modern aircraft. The training and assessment process aims to cultivate proficient pilot performance that includes technical aspects (operating the aircraft while following the rules of the airspace), and nontechnical aspects (interacting with people and managing operational variability). The technical aspects are well defined and understood because they are straightforward and easily measurable. However, the nontechnical aspects are not so simple because their appropriate application is highly variable and context dependent.
Over the years, several frameworks have been developed to attempt to define the essential nontechnical aspects of pilot performance. In the
Action-Oriented Pilot Training 87 late 1990s, the European Commission together with the European Joint Aviation Authorities (JAA) sponsored an investigation into developing more robust methods for the evaluation of pilot nontechnical skills (O’Connor et al., 2002). The NOTECHS (short for Non-Technical Skills) framework was an outcome of this effort to create a generic and effective method of nontechnical skill assessment, for use by flight crew instructors and examiners that minimize cultural and corporate differences across JAA operators and nations (Flin et al., 2003). At about the same time in the United States, Crew Resource Management (CRM) was being developed from accident analyses that identified “errors” induced from “failures” in crew coordination, rather than from technical knowledge and skill deficits (Weiner et. al., 1993). Psychologists engaged in a good faith effort to identify the nontechnical problems that contributed to accidents and developed a set of initial categories that focused on interpersonal skills, such as communication and teamwork. Later, CRM evolved to include cognitive skills such as Decision-making, Situation awareness, Monitoring, and Workload management. These categories were aspirational in the sense that they described the behaviors and traits that professional pilots should acquire and exhibit in their work.
The CRM paradigm continued to evolve through the early 2000s, whereby the focus of CRM shifted from that of behaviors to error trapping and management. Reason (1997) introduced the “Swiss Cheese” model that represents accidents as the result of flaws in the defenses of an organization and suggests that the pilots are the last line of defense. Pilots (and other workers) not only detect and manage errors, but they also catch and correct the errors of others. Errors were increasingly seen as inevitable -- therefore, instead of trying to eliminate them, the focus should be on their management. Studies by Helmreich et al., 2003, showed that during line operations, pilots routinely made errors, but they also routinely trapped them.
Threat and Error Management (TEM) emerged in the late 1990s to expand pilot training beyond CRM to include countermeasures for managing threats and errors. TEM began with line-oriented safety audits (LOSA) that identified common threats and errors, and also the countermeasures pilots used to manage them. TEM highlighted the dynamic and challenging complexity of real-world operations and was used to help situate CRM concepts in operational contexts. TEM encompasses the principle that a crew will routinely face expected or unexpected threats and errors during a flight that must be managed. Through thorough planning, and active identification of threats and potential errors, flight crews regularly successfully manage normal operational variability and counter unexpected events during flight.
TEM is often used to inform training content based on findings from safety audits, and to introduce effective techniques to manage threats and errors. Although TEM training has given pilots strategies to manage threats and errors, it doesn’t provide a complete set of nontechnical skills in which pilots must be proficient. Many CRM and TEM programs deployed today often present nontechnical constructs as broad capabilities, rather than as a specific set of knowledge and skill elements. Such specificity is needed to
88 Holder and Reed effectively train and evaluate pilots on the application of nontechnical skills in the operational context, and to standardize training and evaluation by instructors/evaluators.
DATA COLLECTION To understand how effectively airlines train and assess technical and nontechnical pilot performance, data were collected at four major U.S. Airlines and one major European Airline. The objective of these engagements was to examine current methods air carriers use to derive, implement, evaluate, measure, and determine operational effectiveness of behavioral markers/observable behaviors for flightpath management and crew resource management tasks. A series of interviews were conducted in the form of discussion groups and one-on-one interviews with Captains and First Officers (B737, B767, B777, B787, A320, A330, and A350 fleets), Check Pilots, Instructors, and airline Training and Standards Managers. A total of 32 pilots across the 5 airlines participated in the interviews. Observations of Line-Oriented Flight Training sessions and analyses of airline training material were also conducted. Materials included analysis of grade sheets, reason codes, behavioral markers, observable behaviors, instructor guides, and qualification standards. Due to the proprietary nature of airline training assets, details of each airline’s training and assessment materials are not included. To address this limitation, a descriptive account of the four US AQP operators’ and the one European CBTA operator’s assessment practices is presented.
Since the publication of the FAA’s Flightpath Management Advisory Circular (AC) 120-123 (FAA, 2022), many airlines consider FPM the new “aviate” task, encompassing all aspects of flightpath planning, execution, and assurance. In the discussions, participants were queried about FPM and CRM training and assessment practices. Interestingly, for FPM, most of the concerns raised focused on the nontechnical aspects of FPM. The concerns raised regarding nontechnical performance are listed below.
Distraction, Attention Management, Task Prioritization Pilots become distracted by other concurrent tasks and reduce focus on the flightpath, both in the air and on the ground. It was reported that pilots need reminders that the management of the flightpath is always the top priority. Further, the management of distractions and the appropriate allocation of attention to the right thing at the right time were noted weaknesses, especially for the pilot monitoring.
Inconsistently/Insufficiently Defined Nontechnical Skills and Observable Behaviors There was significant variation in the nontechnical skills lists across airlines and inconsistencies in how those skills were defined. Nontechnical skills are difficult to describe and there is little agreement on which skills are essential, so operators tend to develop bespoke definitions and observable behaviors.
Action-Oriented Pilot Training 89 The level of depth in defining the skills also varies. The lack of standard definitions makes nontechnical skills difficult to teach and assess effectively.
Inconsistent Training and Assessment of Nontechnical Performance The interviewed airlines reported that, when hiring, it is assumed that new pilots arrive already having sufficient proficiency in nontechnical skills, even though most cannot articulate exactly what skills those are, nor what constitutes proficiency. Because of this, nontechnical skills are not really trained or assessed in a systematic way. Rather, nontechnical skills training consists primarily of exposing pilots to that airline’s list of nontechnical behaviors and emphasizing that those behaviors are important. In such cases, nontechnical skills are not trained in the operational context, nor are they tied together with technical tasks.
One of the airlines integrated the TEM model and Reason’s “Swiss cheese” model with CRM behaviors from the FAA’s AC 120-51E into a unified model and training concept (Helmreich, 2010; Reason, 1997; FAA, 2004). The training then focused on pilot actions rather than pilot characteristics, traits, beliefs or feelings. The unified model was further integrated into the airlines’ grading scale and qualification standards. This model represents a move in direction of clearly defining nontechnical performance and associated standards of proficiency and integrating nontechnical with technical performance.
Each airline, however, reported that there is a need for more focused nontechnical skills training, especially for instructors. Instructors reported they struggle with remembering the nontechnical skills’ names, and how to tie them to technical performance. Participants agreed with the statement that airlines “need to teach in a manner that highlights nontechnical skills because 95 percent of safety debriefs don’t involve technical skills.” Nontechnical skills are still often treated as secondary to technical skills, despite acknowledgement of their criticality to safety.
Assessment of nontechnical performance remains one of the biggest challenges. Instructors rely on their own expertise and personal subjective judgment of what constitutes good performance. The development and use of observable behaviors that define proficiency were also inconsistent across the airlines, and instructors reported they are overwhelmed when there are too many. During simulator sessions, instructors experience high workload with many different tasks competing for their attention. Not only do they operate the simulator, but they also simultaneously follow a script, perform as ATC, teach, coach, and assess performance. The instructors called for a simpler, more effective grading tool that would help them capture better training data.
Instructor Training Instructors need training on how to teach nontechnical skills in a manner that is operationally useful and how to assess those skills effectively. All operators mentioned the need for nontechnical skills calibration for instructors. Establishing standards for ratings was found to be difficult because of the inherent subjectivity in the way nontechnical skills are evaluated today.
90 Holder and Reed This leads to challenges with Inter-Rater Reliability (IRR) training, despite significant research and resources allocated to mitigate it.
Integration of Technical and Nontechnical Performance At most airlines, nontechnical aspects of defined tasks are not fully integrated with the technical aspects, nor are technical and nontechnical performance graded holistically as a requirement for proficiency. Only one of the airlines interviewed had effectively integrated nontechnical with technical training and assessment. Nontechnical skills should be introduced early in the training and be integrated into technical performance (e.g. maneuvers and procedures) because in real-world operations, they are inseparable. Going forward, nontechnical skills need to be clearly defined, and for every task, the nontechnical components should be made explicit in terms of what the pilot is expected to do.
The European Airline The research team interviewed training managers from one European airline that utilizes ICAO’s Competency Based Training and Assessment (CBTA) framework (ICAO, 2020). CBTA uses a competency model for specifying pilot performance. The research team also wanted to explore this implementation for specifying nontechnical performance. When the airline began transitioning to CBTA, they reported that the ICAO competencies and their associated observable behaviors “did not make sense” to their instructors and course developers. So, the airline created a process for defining what the competencies meant to them. Through a series of discussion groups, they developed a common vocabulary and definition for each of the competencies and associated grading schemes. They reported there was a lot of confusion with the use of the word “competency” itself and that the term interfered with how the instructors’ conceptualized behaviors by oversimplifying notions of performance.
CBTA requires an extensive set of “observable behaviors” (OBs) that are meant to be indicative of competence for each of ICAO’s specified Competencies (IATA, 2023). This airline customized the observable behaviors, as well as their assessment schemes, to meet their specific airline needs. This airline reported that one of their biggest challenges was how to effectively remediate poor performance for a competency deficiency. Further, many of the prescribed OBs were reportedly untrainable because they were abstractions or represented personal characteristics or traits that may or may not contribute to performance. There was also considerable variability in how pilots and instructors interpreted the meaning of the competencies themselves. For example, one instructor might categorize a behavior as “poor communication,” while another may categorize the same behavior as “poor workload management.” As such, this operator found it very challenging to assess behaviors consistently, and to assign them to the appropriate corresponding competency category.
The CBTA Venn diagram assessment methodology was abandoned by this operator because it was judged to be, in their words, “confusing magic math”
Action-Oriented Pilot Training 91 that resulted in scores that were “all over the place.” The CBTA assessment methodology is intended to make assessment more objective by using OBs, but their instructors reported that it has been more confusing and difficult to execute than it was previously. The so called “magic math” of counting the number of behaviors, their frequency, and their effects on performance was reported to not be an effective assessment method. Further, behaviors that are not listed as an OB but are observed as contributing to poor performance are not accounted for in this system.
KEY FINDINGS All the participating operators expressed the desire to improve pilot training to achieve excellent pilot performance. From these findings it is evident that a science-based approach to modernize the characterization, training, and assessment of nontechnical performance is needed. The specific needs are:
1. Modernize nontechnical skill definitions using a science-based approach. This requires moving away from abstract labels (such as Situation Awareness) to actions that can be taught, observed, and evaluated. 2. Integrate nontechnical and technical performance from day-one of training. There is always a nontechnical component of task performance, so the technical and nontechnical components should be trained and assessed together. This will help reframe nontechnical performance as important and safety critical. 3. Improve instructor training and standardization so that instructors can be more effective at teaching and evaluating nontechnical performance across all training sessions. 4. Assessment/grading tools should be improved to be useful in the simulator and debriefing, and the grading simplified to alleviate instructor/evaluator workload. An instructor/evaluator’s first duty is to teach/assess. Tools for collecting grading and assessment data should not significantly interfere with this priority. 5. The grading schemes currently used were all deemed too cumbersome to meaningfully inform training practices or progress. The instructor/ evaluator comments seem to be a rich source of training data that is underutilized. Investigation is needed into methods to harvest information from instructor comments in an effective and timely manner.
Overall, airline training and assessment is effective because the instructors make it work. However, there is considerable room for improvement. In the next section a new training and assessment framework that addresses these needs is proposed.
ACTION-ORIENTED TRAINING The results of this research suggest there is a growing need to modernize the characterization of nontechnical pilot performance and to integrate it with technical performance for training and assessment. In this section, a
92 Holder and Reed new framework is proposed that is intended to replace NOTECHS, CRM, TEM, and the ICAO Competencies. Those frameworks provide abstract descriptions of nontechnical aspects of the pilots’ work, but they do not explicitly define specific, actionable, trainable learning objectives. The pilot’s central function is to control and ensure the aircraft’s flightpath while also managing the variability and complexity inherent in the operation, which is inherently cognitive. The strategies pilot use to manage this complexity are primarily nontechnical.
A pragmatic shift to an action-oriented training framework is proposed here. This framework was named action-oriented training to combat the notion that nontechnical performance, often referred to as “soft-skills”, are not as important as technical performance. The new framework serves two primary objectives. The first is to make nontechnical performance explicit. The second is to integrate technical and nontechnical training and performance assessment.
The framework is inspired by current neuroscience theories that assert cognition is an active process, created through dynamic human-environment interaction (Engel et al., 2016). In the action-oriented paradigm, cognitive processes are intertwined with action. Cognition therefore is enacted such that cognition is a form of practice itself (Noe, 2009; Varela et al., 2017; Clark, 2024). This paradigm asserts that human cognition is embodied, and situated in a context for action, and may be extended through processes, other people, and artifacts. These developments have important implications for how to train, what to train, and how to evaluate performance.
NONTECHNICAL ACTIONS The framework in development aims to clearly specify the nontechnical aspects of performance in terms of foundational nontechnical actions that pilots should do. The nontechnical aspects of performance are formulated as Nontechnical Actions that are broadly applicable, such as Communicate, Plan, Manage Time and Tasks, etc. These are presented in Table 1 along with a notional mapping to CRM behaviors. Each nontechnical action is represented as a verb that indicates what the pilot should do. This is to ensure the focus remains on action and to make the actions recognizable, understandable, and trainable.
Table 1: Nontechnical action areas and CRM behaviors.
Table available in the original PDF paper.
Nontechnical Actions CRM Behavior Communicate Communication
Lead/Participate Leadership and Teamwork Plan Planning and Decision Making
Manage Time and Tasks Workload Management Assure Monitoring/Cross-checking and Situation Awareness
Not applicable
Manage Flight Path (Nontechnical aspects, specific to pilot work group)
Action-Oriented Pilot Training 93
ACTION STRATEGIES Nontechnical actions are operationalized through the application of action strategies. A set of action strategies were defined for each nontechnical action and serve as specific ways to enact the nontechnical action in the work context (Table 2). These were derived by analysis of airline nontechnical skill lists, safety events, Aviation Safety Reporting System Reports, observed LOFT sessions of well-performing pilots, and pilot interviews.
The intent is to provide pilots with foundational actions that build expertise in each area rather than an exhaustive list of all possible actions. The action strategies are specifically applied in a context, and their appropriate application will therefore vary depending on the situation, tasks, and context.
Training should introduce “condition-action” pairs, i.e., when a certain condition occurs, a relevant action strategy should be initiated. This helps link strategies to a context, so they are more likely to be activated when needed.
Nontechnical Actions Communicate Think Out Loud Say what you are thinking
Ask Questions What do you mean? Is that right?
Be Specific Be clear and concise, avoid hints Be Assertive State concern, intervene if necessary
Brief Plans Verbalize intentions Note Exceptions and Bottom Lines Verbalize what’s different from the usual, clarify the limits
Plan “What if” Plan Ask what-if questions, plan for the worst case “What’s Next” Think ahead, ask “What’s next” and plan for it
Choose Conservatively Select a safe course of action Use Reminders for future, nonstandard tasks Make a note, place in visible/associated location
Lead/ Participate
Appreciate the Perspective of Other Workgroups
Say “I get where you are coming from, what do you need?”
Maintain Flight Deck Discipline Set expectations for conduct, set example of professionalism Flightpath First Flightpath is always highest priority Manage Time and Tasks
Slow down Slow airplane, slow pace of operations, add time Complete Tasks Early Move tasks from periods of high- to periods of low-workload
Balance Task Load Ask for help, offer help Delegate and/or shed tasks as necessary Assign tasks to others/automation or drop lower priority tasks as needed to ensure Flightpath.
(Continued)
Table 2: Continued.
Table available in the original PDF paper.
Action Strategies Application Examples
Nontechnical Actions Assure Check Deliberately Look for (not at) something
Check Reasonableness Does the information make sense?
Recheck when Interrupted or Distracted
Go back and check again to verify completed correctly
Challenge Assumptions Be skeptical If unsure, take action to be sure Recognize and Resolve Doubts and Discrepancies
Manage Flight Path
Mentally Fly the Airplane When PM or autopilot on, engage senses/brain/body as if you are flying Check flight path, then resolve Surprised/Confused/Distracted focus on Flight Path
Adjust Automation Appropriately Use Rules, Levels, Combinations Watch for Signs of Reduced Cognitive Capacity Watch for missed targets, fixation, distraction, etc.
Condition and action selection are context-driven, such that the action strategy a pilot chooses to apply will vary by task and context (Table 3). Actions and their effects are visible to instructor/evaluators and therefore may be reliably assessed. The evaluation criteria are specified as timely identification of condition, appropriate choice of strategy, and effective application of the strategy such that the desired outcome is achieved.
In this framework, the nontechnical action areas overlap and support each other. They are not performed independently. The action areas are cross- cutting and apply to most tasks.
Table 3: Condition-action-application examples.
Table available in the original PDF paper.
Condition Action Strategy Application Example When you are about to change the flight/ ground path
Think Out Loud Say what you’re thinking: Taxiing, approaching an intersection. Captain should say “Here’s Bravo, I’m going to turn left.”
When workload is high
Slow Down Slow aircraft and operational pace: Non- normal occurs on descent (plenty of fuel, landing not urgent), ATC offers shortcut to a straight-in approach. Approach setup and brief are not yet done, non-normal checklist not yet complete. Crew should advise ATC they need more time and ask for delay vectors or a hold.
When reviewing flight plan fuel numbers
Check Reasonableness
Use rules-of-thumb to verify sensibility of the information: “We burn about 6000lbs per hour, so for a 2.5 hour flight we need at least 15000lbs. Hmmm why does the release say only 10000lbs? That can’t be right.”
Action-Oriented Pilot Training 95
CONCLUSION
The findings from this research suggest there is an urgent need to reformulate training and assessment of pilot nontechnical performance and to integrate it with technical performance. To address this need, the action-oriented training framework is proposed. This framework was developed to address the shortcomings of the existing programs and to equip the industry with a science-based training and assessment framework that equips pilots with the operational tools needed to enhance safety and operational effectiveness. The next steps for this work are to continue to validate the action-oriented training framework and conduct a live trial with partner airlines.
ACKNOWLEDGMENT
This research is funded by the Federal Aviation Administration (FAA) Human Factors Division (ANG-C1) in support of the FAA Aviation Safety (AVS) organization. Embry-Riddle Aeronautical University Principal Investigator, Dr. Barbara Holder, would like to thank the FAA program manager, Dr. Victor Quach, and the AVS technical sponsor, Dr. Kathy Abbott, for their support of this work. The views expressed in this paper are those of the authors and do not reflect views of Embry-Riddle Aeronautical University (ERAU) or the Federal Aviation Administration (FAA).
REFERENCES
Clark, A. (2024). The experience machine: How our minds predict and shape reality.
Random House. Engel, A. K., Friston, K. J., & Kragic, D. (Eds.). (2016). The pragmatic turn: Toward action-oriented views in cognitive science (Vol. 18). MIT Press. Helmreich, R, L., Klinect, J. R., Wilhelm, J. A. (1999). Models of Threat and Error, and
CRM in Flight Operations In Proceedings of the Tenth International Symposium on Aviation Psychology, (pp. 677–682). Columbus OH: The Ohio State University. International Air Transport Association. (2023). Competency assessment and evaluation for pilots, instructors and evaluators guidance material (2nd ed.). International Civil Aviation Organization. (2020). Procedures for air navigation services training (Doc 9868) (3rd ed.). Federal Aviation Administration. (2004). Crew Resource Management Training
(Advisory Circular No. 120-51E). Federal Aviation Administration. (2022). Flightpath management (Advisory Circular No. 120–123). Flin, R., Martin, L., Goeters, K., Hörmann, H., Amalberti, R., Valot, C., & Nijhuis,
H. (2003). Development of the NOTECHS (non-technical skills) system for assessing pilots’ CRM skills. Human Factors and Aerospace Safety, 3(2), 97–119. Routledge. Noe, A. (2004). Action in perception. MIT Press. O’Connor, P., Hörmann, H., Flin, R., Lodge, M., Goeters, K., & JARTEL Group, T.
(2002). Developing a method for evaluating crew resource management skills: A European perspective. The International Journal of Aviation Psychology, 12(3), 263–285. Reason, J. T. (1997). Managing the risks of organizational accidents. Ashgate. Varela, Francisco J., Evan Thompson, and Eleanor Rosch. The embodied mind, revised edition: Cognitive science and human experience. MIT press, 2017. Wiener, E., Kanki, B. and Helmreich, R. (1993). Cockpit resource management.
Academic Press.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Factors of Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 979-8-950676-00-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional