Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

What About the Next Generation? Assessing Experts’ Judgments of Human Abilities Required for Working in a Future ATC Environment

Department for Aviation and Space Psychology, German Aerospace Center, Hamburg, Germany
Publication year
Digital Object Identifier10.54941/ahfe100713

Abstract

With the implementation of the European ATM Master Plan (SESAR, 2012) job requirements for air-traffic controllers (ATCOs) will alter significantly. Especially the introduction of time-based operations is considered to impose large changes in task operations. A controller-in-the-loop simulation study was designed to assess experts’ judgments on the main human abilities required for working in time-based arrival scenarios with different levels of assistance by an arrival-management system. Five male controllers (mean age: 32.5 years; mean job experience: 10.8 years) licensed for the simulated sector performed on one baseline scenario and four future scenarios. Gaze-data were tracked and questionnaires on situation awareness and workload, as well as a modified Fleishman Job Analysis Survey (F-JAS) were applied. The experts judged the ability to identify problems produced by automation, as well as the ability to stay vigilant as being increasingly required in a highly automated time-based environment. The high number of gaze transitions between aircraft-targets adds objective indication: Under the simulated future conditions ATCOs were hardly able to build up attention guiding expectations which are necessary to keep up with the system- and traffic-parameters. This was also reflected in low situation awareness and high workload ratings.

INTRODUCTION

With the changes that are discussed for future air traffic management (ATM) the roles of air-traffic controllers (ATCOs) will alter significantly. For example, it is assumed that an exchange of accurate data between ATM actors will enable to build an optimum traffic sequence on a pre-tactical level, so significantly less tactical intervention by ATCOs will be needed for traffic control, in both En Route sectors and terminal maneuvering areas (TMA). Within the Single European Sky ATM Research (SESAR) program it is assumed that the ATCOs tasks may generally become more strategic in nature.

Different evolutions of the future ATM systems with different automation design philosophies are considered in literature, implying different ATCOs roles, from scenarios where the human is the assisted by advanced automation (e.g. in SESAR) up to far looking concepts where the computer takes the controller role. From the human-factors design point of view it has to be guaranteed that, whatever the ATCOs future role will be it has to be in accordance with human capabilities. From a personnel selection point of view it is then essential to proactively assess how such evolutions of the human role may change the required abilities to perform in the job, so selection profiles for ATCOs will be modified accordingly in time.

This paper focuses on analyzing the job requirements and their corresponding abilities in a hypothesized future approach control setting. Several scenarios are constructed in which controllers are supported by automation, with different levels of assistance up to the point where the human has to supervise a self-executing automation. The roles under scrutiny all referred to the tactical level. The ATCOs were focused on traffic situations in TMA as displayed at the controller working position (CWP). A change in task allocation between the human operators was also introduced.

As monitoring is supposed to be a main occupation not only in the current but also in a future control tasks (Broach, 2013), ATCOs’ gaze-data were tracked to gain insight into the controllers’ visual sampling strategies. The impact of these working conditions on situation awareness and workload are measured for control.

THEORETICAL BACKGROUND Future Oriented Job Analysis

With the effort that is put into the deployment of advanced automation into ATC the prognosis of abilities for future ATCOs is co-emerging as a relating human factors topic. Alexander et al. (1989; Ammerman et al., 1987) conducted a job analysis in which they assessed how future controller work would change after the introduction of advanced automation into ATC. Nickels (Nickels, Bobko, Blair, Sands, & Tartak, 1995) developed a list of future ATC-work requirements which was the foundation of a selection system for the hiring of air-traffic controllers. Manning and Broach (1992) conducted a strategic job analysis to assess the changes of the ATCO-job as it is predicted to exist in the future as a result of increasing automation. After an analysis of the present job, they gathered information about the kinds of issues, like technological and organizational developments which may affect the job in the future. Eißfeldt (2009) used a technique for future oriented job analysis first introduced by Schneider and Konz (1989). They held several workshops with subject matter experts (SME) to discuss future issues of ATCO and pilots jobs. After being primed with the discussion outputs the SME filled out the Fleishman Job Analysis Survey (F-JAS, Fleishman & Reilly, 1995) once rating the requirements of the job in the present exists and once for the job of the future. Because none of them had ever performed the job using the future automation, it is unclear how valid these SME predictions of the future job requirements are. It may be advantageous to provide the SME with some experience on using new automation to gain higher validity. Eißfeldt, Deuchert and Bierwagen (1999) followed this approach. They assessed future job requirements of ATCOs under simulated future ATC conditions. The simulation features were data-link communication, a stripless system, planning-tools, and automatic conflict detection aids. In a one hour exercise four brief simulation scenarios were performed by the SMEs. Before and after a session they filled out the F-JAS. Eißfeldt et al. (1999) found a general decrease of cognitive requirements for future ATC, with the strongest effects occurring for oral expression and oral comprehension, written expression, and number facility. These decreases were attributed to the effect of data-link communication.

Broach (2013) recently conducted a strategic-job analysis for the tower controller working position. He used the overall profile of abilities derived by Nickels et al. (1995) to compare current aptitude to future aptitude requirements. He performed an extensive review of current aptitude profiles and of assessable documentation on technology changes which are expected to be introduced until the year 2018. Based on the current profile and the expected technology changes he infers a future aptitude profile. Broach resumes that scanning of visual sources will be one of the abilities which will become more important for future performance.

New Roles and Responsibilities for ATCOs As an increased level of computerized support for ATCOs appears to be mandatory to maintain safety and efficiency of the future air traffic operations (Kirwan, 2001), it has to be considered what the future role of the human should be. In the time-based environment proposed in the SESAR ATM target concept the task of tactical vectoring in the TMA in a way that all aircraft (a/c) exactly meet their preplanned times of arrival is supposedly beyond the ATCOs current performance envelope. So for a 4D-plan conformant tactical traffic handling one will have to take advantage of the strength of automation processing power to quickly calculate the optimal trajectories and assign the referring advisories.

In a human-centered approach for TMA design automation is assumed as a means to expand the ATCOs functional envelope by calculating a/c trajectories with respect to the landing time on ground and providing the ATCO with according tactical guidance advisories. Automation is not expected to take over the whole task, as trajectory management in the TMA also requires a level of adaptability, flexibility, and problem-solving skills which currently can only be provided by the human operator. To keep the human within the decision loop in such a system-support setting it has to be guaranteed that automation supports the ATCOs in developing a mental picture of the traffic situation (Mogford, 1997), instead of degrading it or even removing the picture from the human domain into the automation.

Another paradigm is to let the ATCOs chose what is to be done and delegate the task execution to other agents (humans or automation) with the possibility to intervene if required. Such a role change is envisaged in the SESAR concept of self merging and spacing a/c which are suited with an airborne separation assurance system (ASAS). Here the ATCOs delegate certain spacing and positioning tasks in relation to a specified target a/c to the pilot, who takes the responsibility for identifying the target a/c and establishes separation based on instructions from the ground. With the delegation of specific tasks to other agents it is assumed that ATCOs will be relieved so they could adjust to a more strategic work-style in trajectory management (SESAR, 2012).

In some more far-reaching concepts it is supposed that in the long term future automation will take over (Truman & de Graaff, 2007). Arnaldo et al. (2012) discuss the question whether humans will remain within the future trajectory management processes and will perform some strategic functions where their skills come into play, or if tasks which include tactical decisions in the future may imply autonomous and fully automated processes. As the ATCOs would only be able to assess if the automation advisories are reasonable on a safety level, they would rather be reduced on the safety monitor role where the automation controls and the human monitors only, and over-rides when necessary. In this case the ATCO will have the role of the supervisor of automation. Truman & de Graaff (2007) also envisage that the human intervention and control will be at an absolute minimum in such future ATM system which are self- monitoring and self-controlling.

ATCOs’ Monitoring Behavior Broach (2013) resumes that scanning of visual sources will be one of the abilities which will become more important for future ATC job performance. Thus, the strategic job analysis presented here particularly focusses on the ATCOs’ visual behavior. Visual scanning can be equalized with situation awareness on level 1, which is by definition ‘the perception of the elements in the environment within a volume of time and space” (Endsley, 2006). For ATCOs working with advanced automation the elements to be scanned not only comprise all moving elements within the sectors, but also the automation itself. Parasuraman, Sheridan and Wickens (2008) suppose that operators scan the raw information sources less frequently when tasks are delegated to an advanced automation. Rovira and Parasuraman (2010) accordingly assume that in this case ATCOs also exhibit fewer fixations to the radar display compared with manual control, because they invest their freed cognitive resources to focus on secondary tasks. Generally, good monitoring behavior in ATC requires both focused and distributed attention (Moore & Gugerty, 2010). Even while ATCOs are focusing on a specific area of the radar screen, they must continue to be aware for what is occurring around their sector. Consequently, controllers should not neglect to utilize both focused and distributed attention strategies.

METHOD

In the current experiment the approach of Eißfeldt et al. (1999) is followed to assess ATCOs judgments about future job requirements after they experienced some future ATCO roles. Additionally, their interaction with the simulation environment was assessed by tracking their eye movements to gain further objective indication for changes in work- style. As the most important information for an ATCO is related to the moving targets on a situational display, the analysis focusses on attentional switches (transitions) between dynamic area of interests (dAoI) which are defined by the moving zone each a/c blip with related data-block covers on the radar screen (cf. Gross, Friedrich, & Möhlenbrink, 2010). The analysis of fixation transitions within experimental runs is a means to investigate the co- activation of a/c within the overall traffic-representation, the mental picture, which is assumed to underlie the ATCOs monitoring behavior. As the controllers picture is relational in nature (Niessen & Eyferth, 2001), it could be claimed that a number of transitions between a/c reflects the activation of meaningful relations within this overall traffic-representation. Additionally the transition number presumably also indicates the priority of that constellation with regard to expectations or intended actions.

Research Question 1 (explorative): What do experts assess to be the main changes in job abilities required for their working position after performing new roles in highly automated scenarios within their familiar TMA environment?

Research Question 2: Is there evidence for changes in work-style when the new roles imply that central task of TMA management are delegated to cockpit and/or to automation Hypothesis: Under task delegation conditions, the ATCOs use freed resources to switch to a more strategic work style, i.e. to focus on secondary tasks or to attend to what is occurring around their sector. Consequentially, fewer fixations to the radar display compared with manual control and/or more transitions of fixations between distant a/c on the radar screen are expected. A more strategic work style should also be reflected in a comparably higher level of situation awareness. The freeing of resources should also be indicated in lower levels of experienced workload.

To allow a prognosis of what changes are up to come the key features in the European ATM Master Plan (SESAR, 2012) were taken into consideration. Time-based operations are the key concept to be introduced with SESAR Step 1 (interval-based metering). In a time-based environment the initial synchronization of the traffic arriving in high traffic density TMA will be generally ensured via the allocation of controlled times of arrival (CTAs) on initial approach fixes (IAFs). The introduction of a/c capable of meeting a CTA with appropriate accuracy improves the performance and reliability of an Arrival Manager (AMAN) system. AMANs normally are a means to avoid overcrowded TMA by controlling the flow en-route before reaching the initial approach fixes (IAF), where a/c are handed over from area control centers (ACC) to TMA. However, the DLR AMAN research prototype 4D-CARMA (Ehr & Uebbing-Rumke, 2013), which is classified as a strong executing system which is compliant to the SESAR requirements, also is calculating trajectories with respect to the landing time on ground and generates according tactical guidance advisories. These advisories are timely precise shown to the approach controllers at all approach phases on the radar screen. From the ATC-perspective the implementation of the concept of time-based operations may make a strong execution of automated systems necessary.

Experimental Design Several kinds of future scenarios were constructed with the AMAN as the central element to create different roles for ATCOs. The AMAN can be configured to various levels of automation (Sheridan & Verplank, 1978; Endsley, 1987; Endsley & Kaber, 1999). On the five-level LoA hierarchy proposed by Endsley (1987) the AMAN can be configured to the LoAs ‘manual control’, ‘decision support’, and ‘monitored AI’. In the Endsley and Kaber (1999) taxonomy this would correspond to the LoAs ‘manual control’(MC), ‘decision support’(DS) and ‘supervisory control’ (SC) as the second highest level of automation. However, these levels of automation correspond to different ATCOs roles. They can be assigned by using a taxonomy proposed by in the RHEA (Role of the Human in the Evolution of ATM systems; Nijhuis, 2000) project. Accordingly, in four experimental conditions the ATCOs fulfilled the following roles:

  • Experimental run 1 (baseline): ATCO handles traffic according to the first-come-first-serve principle
  • Experimental run 2: ATCO delegates responsibility of certain tasks to suitably-equipped a/c and aircrew
  • Experimental run 3: AMAN predicts what needs to be done for certain tasks and presents proposals to the ATCO
  • Experimental run 4: AMAN controls – ATCO monitors only, and over-rides when necessary
  • Experimental run 5: Automation controls - ATCO monitors only, and over-rides when necessary, with both working together in the control loop during an emergency event.

In the experimental runs only incoming traffic was simulated. All conditions were simulated within the structure of the sector Frankfurt (FRA) Approach (DFAPP) in its configuration valid until October 2011 (i.e., no third landing runway integrated). DFAPP designates the sector in the terminal maneuvering area (TMA) that gets all a/c with destination FRA Airport (EDDF). These flights approach the area via four different arrival routes and Initial Approach Fixes (IAF). These are OSMAX (western sector inbound), ROLIS (northern sector inbound), GEDERN (eastern sector inbound), and PSA (southern sector inbound). The arriving a/c are lined-up on the downwind and then ‘fed’ onto the final approach. The sector controls all levels from surface up to flight-level 115. The arrival routes are designed in a way so as not to interfere with each other. In the experimental trials only a sector configuration for using runways 25 is applied.

Experimental run 1 (baseline) resembled current operations in which controllers had standard systems support. The AMAN was running during the baseline scenario because it is a standard tool in all Langen ACC sectors. Experimental run 2 (ATCO delegates Control to A/C) also resembled current operations from the point of system support, but the concept of a/c controlled merging and spacing was introduced for the piloted aircraft. From the pilots’ perspective the new requirements during in-flight procedures were (1) to meet the target time of overflow at the IAF, calculated on ground by AMAN and displayed in their CDTI, and (2) to follow the Merging and Spacing instructions given by ATC, which comprises to keep up distance and time to their target a/c ahead. In experimental run 3 (ATCO Controls - AMAN Proposes) the AMAN supported the controller by recommending level, speed and turn-to-base advisories and their timing to meet the time-based criteria for the incoming flights. The AMAN advisories for ATCOs were displayed within the corresponding flight data blocks (= Labels) on the radar screen. From the cockpits perspective run 3 functioned as a repeated measurement with the same requirements as in experimental run 2. In experimental run 4 (AMAN Controls – ATCO Monitors) all of the controller’s tasks, i.e. determining speeds, headings, and levels to the a/c and building an arrival sequence are delegated to the AMAN. The work of the controller here moves toward supervisory control. From the pilots perspective in comparison to the baseline requirements change in receiving ATC advisories from ground-based automation via data-link and reading it back via radiotelephony (r/t) to keep the supervising controller informed about their intent. The ATCO monitored the r/t and always was able to intervene via r/t. In experimental run 5 (AMAN and ATCO in Control Loop) again, the controller was in the supervisory role but had to intervene due to a simulated emergency. From the pilots perspective the requirements were analogue to experimental run 4.

In all experimental task-settings the participants were advised to control or monitor the incoming traffic for the whole TMA. Generally the participating ATCOs were advised to guide the down-winding traffic via the standard arrival routes (STARs). All controllers reported to be very familiar with this assignment, as this is a standard setting in real-life during periods of lower traffic load. The participants also worked in a one-man setting without the assistance of a coordinate controller. The experimental baseline scenario (experimental run 1) is oriented at the actual method of operation of approach controllers. The ATCOs tasks were to receive incoming traffic at the IAF, coordinate these a/c according to the traffic flow and provide sufficient lateral and vertical separation between a/c, as well as guiding them to the final approach. The AMAN was active, but just displaying a planned sequence on a time ladder. To the subsequent experimental runs the CTA and merge-&-space operation, a strong executing AMAN, and CPDLC-communication were selectively introduced. For the first future condition (experimental run 2) the AMAN was active, but not in a strong executing mode. The job includes all core-tasks of the baseline, added by the tasks related to the merge and space procedure, like instructing the pilots and monitoring the on-going operations. For the next, more advanced future condition (experimental run 3) the AMAN was set into strong executing mode. In this setting the ATCO is advised to generally instruct the pilots with the AMAN advisories, if no safety concerns would stand against it. Also the core tasks to determine headings, level, and speeds according to the intended approach sequence where suspended as AMAN made recommendations to follow. However, the list again was expanded by the new core-task to evaluate AMAN advisories before communicating them via r/t. For the most advanced future condition (experimental run 4) the AMAN was set into a self-executing mode. Almost all core-tasks are now performed by the AMAN via data-link communication with the pilots. In the role of supervisory controller, the new core-tasks are to monitor AMAN performance on the AMAN display, monitor a/c behavior on the radar screen, monitor r/t communication, and to intervene to take manual control when necessary should emerge. This most advanced future scenario was performed a second time with shuffled a/c signs and a provoked emergency event that forced the ATCO to perform tactical interventions (experimental run 5).

Simulator Preparation The Human-in-the-Loop Simulations were performed at DLR’s integrated Air-Ground Simulator AviaSim (Bruder et al., 2013). The ATCO is seated in front of four displays with the radar screen as the main window in front. On the left the electronic flight strip bay is located. On the right the 4D-CARMA generated timeline for the arriving a/c, a display for the advisories the automation currently is sending to a/c, as well as a scale showing distances of a/c on the final approach (the so called mileage-metering scale) are shown. The head-up display shows labels of a/c approaching the TMA on a timeline according to their controlled times of overfly (CTOs) at the referring IAFs. All a/c are simulated by pseudo pilots except for three that are simulated by fully equipped simulation cockpits operated by airline pilots. Pseudo pilots operate the non-piloted a/c according to standard procedures until they are transferred to the TMA. At the beginning of the scenario a pseudo ACC-controller overtakes communication with piloted a/c and transfers it to the ATCO when the a/c reaches the TMA border.

Dependent Variables and Procedure The experiment was conducted in 5 sessions over three days. In total five controllers from Deutsche Flugsicherung GmbH (DFS) Area Control Center (ACC) Langen with valid licenses for TMA Frankfurt and 15 pilots from various airlines participated. Only controller data will be reported here. The controller sample was all male with a mean age of 32,5 years (SD: 6,42), and a job experience of 10,8 years in the mean (SD: 6,85). A session comprises four runs of a future scenario, one baseline scenario, and five training runs for the controllers or three training runs for pilots, respectively. A single simulation run involved one controller, three pilots and two pseudo pilots. The controllers were working the whole Frankfurt Terminal Maneuvering Area (TMA) in a one-man operation modus. Pilots were starting airborne within the lower airspace and entered TMA via three different initial approach fixes (IAF). The controllers, pilots and pseudo-pilots communicated using r/t or CPDLC/Data-Link transmissions.

For the controllers the first exercise day started with an introduction of questionnaires, calibration on the eye- tracking system and an about 90-minutes in-depth walk-through the F-JAS dimensions. The second part of the first day included two initial trainings runs of about one hour to get acquainted with the simulator and the Arrival Manager (AMAN) interface (first trainings run) and to familiarize with pseudo-pilots interactions (second training run). The second day started with a recapping trainings session in which the merging and spacing procedures were practiced with pseudo-piloted a/c. This was followed by a checkup of the controllers’ dynamic vision abilities. Afterwards the first joint trainings scenario was executed to familiarize controller, pilots, and the pseudo-pilots cooperation. This was followed by a second joint training in which merging and spacing procedures were practiced with the piloted aircraft. The measured traffic sample contained 24(+x) flights with destination Frankfurt (EDDF), with at least six a/c entering the TMA on each of the four IAF during the scenario run. The scenario contained a mix of a/c of heavy and medium weight class planned to land in staggered mode on the two parallel runways 25R and 25L, which have to be operated dependently. Next, the controllers executed the five experimental runs, one at the end of day two and four on day three. For comparison reasons, the same traffic sample was used for all runs. However, to avoid familiarization with the traffic sample, all a/c signs were shuffled or substituted between each run. Furthermore, pilots switched working positions after each run to let them experience different traffic situations. Experimental run four and five were always performed consecutively (en bloc) and the baseline run was always performed first, i.e. at the end of day one. The runs 2, 3 and the block 4+5 were counterbalanced to avoid practice effects. Each run was performed only after a break.

Quantitative and qualitative data were gathered during and after each simulation run. As a subjective measure of workload and situation awareness the instantaneous self-assessment (ISA; Kirwan & Flynn, 2001) was used. Controllers were prompted four times during a scenario (appointed according to the flight phases of the piloted aircraft, i.e. RTA IAF, Merge, Follow, and Turn) to give their ratings on current situation awareness and workload on two three-point scales. During the simulation runs eye-movements were tracked to gain indication for participants’ visual sampling behavior, like the change from focused to distributed sampling. The transitions (switches) of fixations between a/c (dAoIs) within trials are used as an indicator for the actual relational structure of the ATCOs mental picture. The fixation duration on objects is used as an objective measure of situation awareness related to these objects. Immediately after each simulator run, the participants filled out (a) a 14-Component Version of the Situation Awareness Rating Technique (SART; Taylor, 1990) , (b) the NASA Task-Load Index (NASA tlx; Hart & Staveland, 1988), a questionnaire to assess the impact on mental workload (AIM; Dehn, 2013), and the SHAPE Automation Trust Index (SATI; Dehn, 2013). At the end of the last experimental run of exercise day two and exercise day three the participants filled out a modified version of the Fleishman Job Analysis Survey (F-JAS; Fleishman & Reilly, 1995) and cognitive interviews were held directly afterwards to gain in-depth information about experienced job requirements. Additionally, a questionnaire on user satisfaction and acceptance was completed at the end of day three.

RESULTS

The controllers agreed on several usability scales that the system was easy to learn, and that the time needed to learn using the system was adequate. The controllers also agreed on the statement that the scenarios they had worked on were interesting and inspiring. They regarded the traffic-load of the scenarios to be comparable to the current real- life traffic.

Questionnaire on Job Ability Requirements The ATCOs judgments on the abilities required for future job performance were assessed once for the presented future scenarios (overall) and compared to the ability-profile the experts initially identified for the baseline scenario. These ability-requirement profiles are put together from a of 37 ability-requirement scales the ATCOs filled-in, with 32 of them selected from the Fleishman Job Analysis Survey (F-JAS), supplemented by five scales covering additional requirements developed by DLR in a format similar to F-JAS. A paired-sample t-test on the single scales

Figure 1: Mean and standard errors of SMEs (n=5 ATCOs) F-JAS type scale ratings on the 37 ability requirement dimensions plotted for the (a) the simulated baseline scenario and (b) the simulated future scenarios revealed a significant effect of future changes (versus baseline) on required Problem Sensitivity (cognitive scale; future increase) and Number Facility (cognitive scale; future decrease) abilities (see Table 1). With respect to practical relevance the marginal differences in required Spatial Orientation (cognitive scale; future decrease),
Original PDF Figure 1:

Perceptual Speed (cognitive scale; future increase), Situation Awareness (interactive/social scale; future increase), Resistance to Premature Judgment (interactive/social scale; future increase), and Vigilance (future increase) abilities may also be considered (see Table 1). A Wilcoxon signed-rank test on the rank of means for the 38 ability requirement scales showed no fundamental change in the ability requirement profiles from current (baseline) to future scenarios. Figure 1 presents the means and standard errors of SMEs’ F-JAS-type scale ratings related to the baseline and the future scenarios on the 37 ability requirement scales. For the baseline scenario time sharing received the highest mean rating and is thus considered as the most important ability required for the current job, which is a typical finding.

Table 1: Mean ratings of F-JAS type scale ratings for the baseline and future scenarios. Only those scales are listed for which a paired-sample t-test yielded a significant or marginal within-group effect
Original PDF Table 1:
Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

Baseline Future T df P (2 way) Problem Sensitivity 4,92 6,44 - 3,919 4 .017

Number Facility 4,04 2,56 .2,929 4 .043 Spatial Orientation 4,20 2,46 2,400 4 .074

Perceptual Speed 4,66 5,56 - 2,535 4 .064 Situation Awareness 6,56 7,00 - 2,269 4 .086

Resistance to Premature

3,90 5,42 - 2,513 4 .077

Judgment Vigilance 4,76 6,56 - 2,126 4 .066

In the resulting future ability-requirement profile, Situation Awareness1 - which received the highest mean ratings - followed by Stress Resistance2, Vigilance3, Problem Sensitivity4, Emotional Control5, Selective Attention6, Auditory Attention7, Resilience8 and Perceptual Speed9 received higher values than the ability of Time Sharing10 (cf. Figure 1). The increasing importance of Situation Awareness, Vigilance, Problem Sensitivity, and Perceptual Speed should be considered as important developments in task requirements, as they are ranked in the top ten required future abilities when compared by their mean ratings (cf. Figure 1). Significant future decreases (see Table 1) were only determined for scales of minor importance (namely Number Facility and Spatial Orientation).

Gaze Analysis Transitions. In Figure 2 for each experimental condition the state-transition matrices containing the total number of gaze transitions between a/c are displayed. The referring a/c are labeled according to their position in the landing

1 Ability to perceive the current automation-traffic interaction, understand the reasons behind, and project the future development.

2 Capability of dealing with stress situations in such way that control is maintained and the objective achieved.

3 Ability to maintain attention and alertness over prolonged periods of time

4 Ability to tell when something is wrong or likely to go wrong.

5 Ability to stay in self-command in emotionally challenging situations, when irritating, unexpected, difficult or stressful factors occur.

6 Ability to concentrate on a task without getting distracted.

7 Ability to focus on a single source of auditory (hearing) information in the presence of other irrelevant and distracting sounds.

8 Ability to rapidly recover normal energy and enthusiasm following a discouraging situation.

9 Involves the degree to which one can compare letters, numbers, objects, or patterns, both quickly and accurately. This ability also includes comparing a presented object with a remembered object.

10 Ability to shift back and forth between two or more sources of information.

sequence. For each experimental condition the degree an a/c was co-activated (and thus related) with the other a/c can be read out by first tracing the respective line and then the respective column of the a/c. For example, to determine for the baseline condition the degree the a/c in the fifth position of the landing sequence is co-activated with other a/c, the counted transitions for the preceding a/c can be read out in the line labeled ‘5th’ (counts for a/c ordered in their spatial distance to the a/c under consideration are 26, 14, 34 and 11), the transition numbers for the succeeding a/c can be traced in the column correspondingly labeled ‘5th’ (counts for a/c ordered in their spatial distance to the a/c under consideration are 37, 24, 12, 14, 5, 13, 0, 2, 0, 0, 0). To simplify the interpretation the colors indicate the suspected co-activation of these relations in the ATCOs mental picture, from red (indicating high relevance) to white (medium relevance) to blue (minor or no relevance). Obviously, for all experimental conditions the ATCOs allocated much attention to relations of a/c neighbors on the final approach. Maximum transition numbers between neighboring a/c are counted for (a) the experimental condition in which the ATCO delegated responsibility of certain tasks to suitably-equipped a/c and (b) the condition where the AMAN controlled and the ATCO monitored only, and over-rode when necessary. However, for both of these conditions, and especially for the supervisory control condition, even relations between distant a/c (preceding and succeeding) received more attention than in the baseline condition.

Position in Experimental 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th 13th 14th 15th 16th Conditions Final Sequence

First Come First Serve (Baseline)

Time-Based Operations

Medical Emergency

ATCo Delegates to A/C AMAN Proposes AMAN Controls – ATCo Monitors AMAN and ATCo in Control Loop During

1st 2nd 60 3rd 52 19 4th 58 9 36 5th 14 34 14 26 6th 2 9 15 22 37 7th 3 3 5 8 24 29 8th 6 5 11 6 12 16 60 9th 1 0 2 3 14 5 20 35 10th 0 2 1 5 5 4 24 34 28 11th 1 3 2 5 13 12 27 23 34 70 12th 0 0 0 0 0 3 3 7 0 16 14 13th 0 0 0 2 2 2 5 11 8 27 28 9 14th 0 2 0 1 0 3 3 3 3 4 7 17 17 15th 0 0 0 0 0 4 10 7 1 4 6 2 10 20 16th 0 0 0 0 0 0 1 1 3 2 2 1 0 1 5 1st 2nd 101 3rd 57 48 4th 22 18 92 5th 18 25 39 26 6th 0 20 15 68 67 7th 6 22 33 9 97 28 8th 5 7 26 41 26 19 127 9th 3 8 6 11 21 3 28 106 10th 1 3 5 6 15 27 24 51 122 11th 0 1 1 5 12 11 29 12 31 118 12th 0 0 0 2 1 1 4 9 7 21 14 13th 0 0 0 0 7 3 8 19 29 21 54 8 14th 0 0 0 0 3 0 12 9 7 19 6 15 24 15th 0 0 0 0 2 0 1 5 0 6 8 1 5 9 16th 0 0 0 0 0 0 1 0 2 6 9 9 2 7 4 1st 2nd 54 3rd 31 16 4th 3 10 24 5th 2 3 2 6 6th 0 10 9 17 28 7th 1 2 2 6 30 14 8th 0 1 0 6 9 8 25 9th 13 9 10 10 13 15 3 5 10th 0 0 0 3 1 1 0 4 2 11th 0 3 1 5 4 5 13 24 3 20 12th 0 0 0 0 0 4 5 5 0 10 5 13th 0 0 0 2 2 1 1 0 2 4 2 3 14th 0 0 0 0 1 1 4 0 0 2 4 3 0 15th 0 0 0 0 2 1 3 2 0 0 0 2 0 4 16th 0 0 0 0 1 0 0 0 0 1 0 1 1 0 0 1st 2nd 143 3rd 54 66 4th 7 22 66 5th 18 43 36 37 6th 5 10 34 42 23 7th 6 10 24 20 82 24 8th 2 14 15 21 64 20 113 9th 2 4 6 14 23 8 76 102 10th 7 6 7 9 19 7 21 62 61 11th 4 6 15 18 18 19 50 63 107 157 12th 0 0 0 0 4 3 3 22 10 28 28 13th 0 0 0 4 3 4 15 23 9 64 45 26 14th 0 0 0 2 4 6 7 11 9 19 23 29 35 15th 0 0 0 0 2 2 5 8 3 17 2 8 9 20 16th 0 0 0 0 1 0 4 7 4 10 3 8 8 8 13 1st 2nd 36 3rd 25 9 4th 5 6 12 5th 0 0 0 1 6th 1 0 1 19 0 7th 1 4 4 0 3 5 8th 1 0 4 0 1 2 7 9th 1 2 10 0 3 6 40 65 10th 1 0 2 5 1 5 8 47 39 11th 0 0 0 1 1 2 1 7 10 31 12th 0 0 0 0 0 1 0 6 15 5 2 13th 0 0 0 2 0 0 3 7 11 8 3 12 14th 0 0 0 0 0 0 1 2 1 0 0 0 14 15th 0 0 0 0 0 0 0 2 1 1 0 1 1 4 16th 0 0 0 0 0 0 3 8 11 5 2 1 6 1 1

Figure 2. State-transition matrices containing the total number (summed up for N= 5) of gaze transitions between dynamic areas of interest (blip and data block) representing the single a/c on the radar monitor during an experimental run

Figure available in the original PDF paper.

Fixations per screen. The numbers of fixations counted for the radar display for all experimental conditions are all higher compared to the baseline (cf. Figure 3). This is contrary to the directed hypothesis predicting that the number of fixations counted for the radar display should decrease when trajectory management tasks are delegated to other agents. As the data was not normally distributed the most appropriate way to statistically test the differences from the baseline was Wilcoxon signed-rank test. There was a significant increase from baseline to the 3rd condition (ATCO Controls - AMAN Proposes) in the number of fixations counted for the radar screen, Z = -2,023, < .05, and also to the 5th condition (AMAN and ATCO in Control Loop: Z= - 2,023, p< .05). The 4th condition (AMAN Controls – ATCO Monitors) only marginally differed from the baseline (Z= -1,753; p= .080). Thus, there were changes in visual sampling strategies (and work style) during scenarios, but not as expected. As flight strips information has relevance for the ATCOs planning of future incoming traffic the numbers of fixations counted for the strip-bays display should increase in case the ATCOs have used free attentional resources to focus on some more strategic considerations. Figure 3shows that under all experimental conditions the ATCOs only rarely attended to this planning information. However, there was a significant increase from baseline to the 5th condition (Z= - 2,032, p= .042) indicating some planning activity was involved during the emergency handling. As also can be seen in Figure 3 the AMAN (4D-CARMA) screen was more attended to under the 4th (Z= -1,753; p= .080) and 5th condition (Z= - 2,023, p= .043). This change in visual sampling is mainly attributable to the ATCOs’ close monitoring of the HMI the automation was displaying its instructions given to the cockpits.

Figure 3. Number of fixations on each of the three monitors of the controller working position for each scenario run
Original PDF Figure 3.

Control Variables Repeated Measures Analyses of Variance (rANOVA) were used to statistically compare the questionnaire outcomes for the various runs. Partial eta-squared ([Equation or mathematical symbol available in the original paper PDF] p

2) is given as a measure of effect size. For each analysis p < .05 was used.

Workload. For each run workload measures with NASA TLX were obtained (SME). An ANOVA on the NASA TLX Overall Scores for the five runs showed only a marginal effect [F(4,16) = 2,694, p < .10, [Equation or mathematical symbol available in the original paper PDF] p

2 = .402]. The within subject single contrasts showed that experimental run 5 imposed significantly higher workload compared to the baseline condition [F(1,4) = 9,476, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .703]. No effect for the five runs on the AIM total score was measured. Single contrasts indicate significantly higher demands on monitoring for experimental run 2 (ATCO Delegates to A/C) compared to experimental run 1 (baseline), F(1,4) = 16,000, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .800. A 4 (Moment of Measurement) x 5 (ATCO Role)-factorial rANOVA for the ISA scores showed a significant effect for the factors ‘Moment of Measurement”, F(3,6) = 14,600, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .970.

2 = .880. These findings contradict the hypothesis trajectory management task delegation would release the ATCO. There is a distinctive trend for a workload increase with the scenario progressing. The experimental factor (ATCO Role) had no significant effect on workload. Single contrasts revealed a significant higher ISA workload ratings only for the experimental run 2 (ATCO Delegates to A/C) compared to run 1 (baseline), F(1,4) = 64,000, p < .05, [Equation or mathematical symbol available in the original paper PDF] p Situation Awareness. An ANOVA for the 5 runs showed a significant effect for the 10-D SART scale values, F(4,16) = 4,063, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .504. Single comparisons with the baseline run revealed significantly lower situation awareness under 3rd experimental run (ATCO Controls - AMAN Proposes: F(1,4) = 9,732, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .709), as well as under the fourth experimental run (AMAN Controls – ATCO Monitors: F(1,4) = 24,500, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .806). This is contrary to the directed hypothesis which predicts the allocation of attentional resource released by tasks delegation to enhance situation awareness. A 4 (Moment of Measurement) x 5 (ATCO Role)-factorial ANOVA for the ISA scores on ‘situation awareness’ showed a significant effect for the factors ‘Moment of Measurement’, F(1,931, 3,861) = 8,826, p < .05, [Equation or mathematical symbol available in the original paper PDF] p

2 = .815. There is a distinctive trend for situation awareness to decrease with the scenario progressing. There was no significant effect for the experimental factor (ATCO Role).

CONCLUSIONS

The ability analysis revealed the specific job requirements which occur for ATCOs when moving from the current to a future TMA-task profile. The SME experienced higher sensitivity with regard to problems produced by automation will be required (problem sensitivity). In their opinion a system-supervising ATCO has to continuously assess if the automation instructions are reasonable or not. This requires higher abilities to stay alert over a long period of time (vigilance). In the passive supervisor roles ATCOs experienced they were unable to from expectations on traffic behavior on the basis of their own intentions, so they were more heavily occupied with tracing and integrating automation behavior into their mental picture to understand what is happening and project what the system is going to do. According to the SMEs’ appraisal these new job requirement more heavily goes back to the abilities to obtain situation awareness.

The gaze transitions between dynamic areas of interest representing the single a/c on the radar monitor are interpreted to indicate the relational structure of the ATCOs current mental picture. Higher transition numbers show a/c relations which are more closely monitored, relate to a/c which are more frequently updated, and thus refer to a/c representations which are more highly activated in the ATCOs mental picture. The transition counts show most highly activated a/c relations for neighboring a/c on the final approach. They are most often monitored in succession. The highest counts for transitions between those focal a/c relations were observed for scenarios (a) where the ATCO delegated separation tasks to a/c and (b) when the ATCOs monitored the automation controlling the traffic. These findings may indicate how monitoring behavior is changing when it is no longer guided by the ATCOs intentions, but by their expectations and recurrent checks of other agents (pilots or automation) actions. This increase in focal switches between relevant a/c constellations reveals more frequent updates of focal elements in the ATCOs mental picture. The increase in number of transitions between distant a/c also indicates higher distributed- attention requirements for both scenarios. The ATCO more frequently had to update what else was going on in the sector when other actors also were in the decision loop.

As flight strips information has relevance for the ATCOs planning of future incoming traffic, it was expected that the numbers of fixations counted for the strip-bays display should increase in case the ATCOs delegates tasks and has free attentional resources to focus on some more strategic considerations. Under all experimental conditions the ATCOs only rarely attended to this planning information. So the ATCOs generally only took the tactical level into account. Only in the experimental condition the ATCO had to handle an emergency a/c gaze data suggest some planning activity was involved. However, questionnaire data did not proof a workload-reducing effect of task delegation. In fact, ATCOs experienced higher workload which might be attributable to higher monitoring efforts. Even for those experimental future conditions where the ATCOs supposedly updated their mental picture more frequently questionnaire data indicate that situation awareness was deteriorating. Thus, changes in visual sampling strategies and work style can be attributed to the introduction of the future experimental conditions, but they were not as they were expected in the hypothesis.

It may be questioned if receiving tactical advises from automation to comply with 4D-plan requirements is a means to expand the ATCOs functional envelope. When ATCOs in our simulation were advised by automation when and how to act, they were hardly able to assess if the automation advisories were reasonably 4D-plan compliant. As the ATCOs actually had to decide on situational matters which they could not assess, they were only acting as a transducer of system advisories to the pilots. This might have caused their experienced decrease in situation awareness. One could argue the system could equally well directly interact with the receiver in this case, namely the a/c. However, the monitoring of such a system would require keeping up with a large quantity of rapidly changing system parameters. Based on the current data this would imply a shift of job requirements for TMA ATCOs to a higher demand for vigilance, paired with high demands for perceptual speed, problem sensitivity and situation awareness. If such a shift already can be expected with the introduction of 4D-trajectory management will depend on system design decisions. Whatever these decisions may be it is recommended here that the human performance assessment in the system validation activities should also encompass the use of job analysis tools. This is to ensure that ATCO selection will sooner than too late look out for employees who are capable to fill in the roles the new ATM system assigns to the human operator.

REFERENCES

Alexander, J. R., Alley, V. L., Ammerman, H. L., Fairhurst, W. S., Hostetler, C. M., Jones, G. W. et al. (1989). FAA air traffic control operations concepts Volume VII: ATCT tower controllers Washington, DC: Federal Aviation Administration Advanced Automation Program (AAP-110). Ammerman, H. L., Becker, E. S., Bergen, L. J., Claussen, C. A., Davies, D. K., Inman, E. E. et al. (1987). FAA air traffic control

operations concepts Volume V: ATCT/TCCC tower controllers Washington, DC: Federal Aviation Administration Advanced Automation Program (AAP-110). Arnaldo, R., Sáez, F. J., & García, E. (2012). HALA!: Towards Higher Levels of Automation in ATM. Position Paper - State of the Art and Research Agenda. In. Broach, D. (2013). Selection of the Next Generation of Air Traffic Control Specialists: Aptitude Requirements for the Air Traffic

Control Tower Cab in 2018 (Rep. No. DOT/FAA/AM-13/5). Oklahoma City, OK: Federal Aviation Administration, Civil Aerospace Medical Institute. Bruder, C., Eissfeldt, H., Grasshoff, D., Gürlük, H., Friedrich, M., Hasse, C. et al. (2013). Aviator II: Simulator-based Research on Operational Monitoring and Decision Making for Human Operators in Future Aviation (Rep. No. Forschungsbericht 2013-04). Köln: Deutsches Zentrum für Luft- und Raumfahrt (DLR). Dehn, D. (2013). The SHAPE Questionnaires. http://www.eurocontrol.int/sites/default/files/content/documents/nm/safety/safety-

shape-leaflet.pdf. Ehr, H. & Uebbing-Rumke, M. (2013). Interface AviaSim 4D-CARMA (Rep. No. IB 112-2010/43). Braunschweig: DLR. Eissfeldt, H. (2009). Aviator 2030 - Determining ability requirements in future ATM systems. In 15th International Symposium on Aviation Psychology Dayton, OH. Eissfeldt, H., Deuchert, I., & Bierwagen, T. (1999). Ability Requirements for Future ATM Systems Comprising Datalink: A

Simulation Study Using an EATCHIP III based Platform (Rep. No. Forschungsbericht 1999-15). Köln: Deutsches Zentrum für Luft- und Raumfahrt (DLR). Endsley, M. R. (1987). The application of human factors to the development of expert systems for advanced cockpits. In Human Factors and Ergonomics Society (Ed.), (pp. 1388-1392). Santa Monica, CA. Endsley, M. R. (2006). Situation awareness. Salvendy, G. Handbook of human factors and ergonomics. 3, 528-542. New York,

Wiley. Endsley, M. R. & Kaber, D. B. (1999). Level of automation effects on performance, situation awareness and workload in a dynamic control task. Ergonomics, 42, 492. Fleishman, E. A. & Reilly, M. (1995). Administrator's guide to the Fleishman Job Analysis Survey (F-JAS). Potomac, MD:

Management Research Institute. Gross, H., Friedrich, M., & Möhlenbrink, C. (2010). Development of a process for the evaluation of eye-fixation data using dynamic areas of interest. Braunschweig: DLR. Hart, S. G. & Staveland, L. E. (1988). Development of NASA-TLX (task load index). Results of theoretical and empirical

research. In P.A.Hancock & N. Meshkati (Eds.), Human Mental Workload (pp. 139-181). North Holland: Elsevier. Kirwan, B. (2001). The role of the controller in the accelerating industry of air traffic management. Safety Science, 37, 151-185. Kirwan, B. & Flynn, M. (2001). Controller conflict resolution strategies for CORA2. In 4th USA/Europe ATM R&D Seminar Santa Fe. Manning, C. A. & Broach, D. (1992). Identifying ability requirements for operators of future automated air traffic control

systems (Rep. No. DOT/FAA/AM-87/26). Washington, DC: Federal Aviation Administration. Mogford, R. H. (1997). Mental models and situation awareness in air traffic control. International Journal of Aviation Psychology, 7, 331-341. Moore, K. & Gugerty, L. (2010). Development of a novel measure of siutation awareness: The case for eye movement analysis.

In HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING (pp. 1650-1654). Santa Monica, CA: Human Factors & Ergonomics Society. Nickels, B. J., Bobko, P., Blair, M. D., Sands, W. A., & Tartak, E. L. (1995). Separation and Control Hiring Assessment (SACHA) final job analysis report Washington, DC: Federal Aviation Aviation Administration Office of the Assistant Administrator for Human Resources Management. Niessen, C. & Eyferth, K. (2001). A model of the air traffic controllers' picture. Safety Science, 73, 187-202. Nijhuis, H. B. (2000). Role of the Human in the Evolution of ATM (RHEA): Final Report (Rep. No. RHEA/NL/SPR/01).

Amsterdam: NLR. Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2008). Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making, 2, 141-161. Rovira, E. & Parasuraman, R. (2010). Transitioning to Future Air Traffic Management: Effects of Imperfect Automation on

Controller Attention and Performance. Human Factors, 52, 411-425. Schneider, B. & Konz, A. M. (1989). Strategic Job Analysis. Human Resource Management, 28, 51-63.

SESAR (2012). The European ATM Master Plan - The Roadmap for Sustainable Air Traffic Management.

https://www.atmmasterplan.eu/ Sheridan, T. B. & Verplank, W. L. (1978). Human and computer control of undersea teleoperators. Cambridge, MA: MIT Man- Machine Systems Laboratory. Taylor, R. M. (1990). Situation awareness rating technique (SART): The development of a tool for aircrew systems design (Rep.

No. AGARD-CP-478). Neuilly Sur Seine: NATO-AGARD. Truman, T. & de Graaff, A. (2007). Out of the Box - Ideas About the Future of Air Transport, Part 2 Advisory Council for Aeronautic Research in Europe (ACARE).

Cover of the proceedings volume Advances in Human Aspects of Transportation: Part II

Explore the complete digital volume

Advances in Human Aspects of Transportation: Part II

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2098-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →