Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Influence of Head up Display on Visual Fatigue and Eye-Hand Discoordination in Runway Incursion Scenarios

1 Beijing Institute of Computer and Electronic Application Technology, Beijing, China ↗
2 Beijing Wuzi University Logistics School, Beijing, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1001566

Abstract

In contrast to Primary Flight Display (PFD), Head-up Display (HUD) has not been found to conducive to the detection of runway incursions, altough visual fatigue and eye-hand discoordination have both been cited as important factors in this regard. Background:Few studies that assess the impact of HUD on visual fatigue and eye-hand coordination. Method:In addressing this deficit, we performed a simulation flight test that detects runway incursion by deploying the A320 cockpit visual simulation system that integrates HUD. Results:The results demonstrate the percentage of eye-hand discoordination with HUD was 18.75%, thereby exceeding the 8.40% recorded in the application of PFD by 10.35%. Hand manipulation duration and frequency analysis indicates that it helps to reduce the pilot’s physical load. The results also demonstrate that the linear fitting coefficient of pupil diameter change over time when HUD is applied is -0.000055 less than the 0.000009 evidenced when PFD is applied. Conclusion:These results lead to the conclusion, flight operating error and visual fatigue are more easily triggered by HUD during the landing phase. Application:The results of this study may contribute to the evaluation of the application and design of HUD, along with the analysis of pilot’s visual fatigue and performance runway incursion prevention.

INTRODUCTION

Runway safety was identified as a global safety priority by the second edition of the Global Aviation Safety Plan, which was published by the International Civil Aviation Organization (ICAO) in the second edition of the Global Avi- ation Safety Plan (ICAO, 2016). The International Air Transport Association (IATA) has defined runway incursion as: “[A]ny occurrence at an airport involving an aircraft , vehicle, person, or object on the ground that creates a collision hazard or results in loss of sepa- ration with an aircraft taking off, intending to take off, landing, or intending to land.” (IATA, 2002).

The ICAO and Runway Safety Partners also regard runway in incursions as a high-risk category. Although, runway incursion accidents reported d in recent years is very low, the number of runway incursion incidents remains

© 2022. Published by AHFE Open Access. All rights reserved. 43

high (ICAO, 2017). Runway incursions can be divided into four catego- ries: 1) operational deviations; 2) operational errors, 3) pilot deviations; and 4) vehicle or pedestrian deviations. (IATA, 2002). Pilot deviations are the most frequently occurring category (Prinzel, 2004). The factors that contribute to runway incursions can be grouped into three categories: human performance limitations (which include false perceptions, loss of situational awareness and memory lapses), latent conditions and, meteorology threats.

FAA Runway Safety Statistics show there were a total of 1745 runway incursions in the US in Flying Year (FY) 2017. Pilot deviation (PD) accounting for 65 % of this total. (FAA Runway Safety Statistics, 2017). Airbus statistics indicate that 55% of aircraft accidents result in total loss and 50% of fatal accidents occur during the approach and landing phases. (Airbus Industries, 2000). Studies reveal that most civil aviation accidents are evidenced during these phases, with casualties and property loss occurring as a result. This is why both phases have become a key focus of flight accident prevention and safety supervision.

Previous studies have established that detection performance is found to be worse when a HUD is used during the flight landing phase. (Steven et al., 2000). Most previous studies that evaluated HUD impact on event dete- ction tended to focus on pilot performance, and therefore engaged, inter alia, eye tracking (Lisa, 2004), mental workload (Wickens, 2003), miss rate (Wickens, 2009) and response times (Steven et al., 2000). In contrast, the impact of HUD on eye-hand coordination and visual fatigue remained under-researched.

Fatigue is closely connected with increased risk (Dawson, et al., 2012; Dawson, et al., 2014). It impairs cognitive function (Härmä, 2006), leads to accidents (Cabon, et al., 2012) and reduces work performance. (Fu, et al., 2016; Filtness & Naweed, 2017). Fatigue affects pilot attention and response speed (Borghini, et al., 2014), damages information processing, and influences the driver’s ability to react to emergencies and retards sen- sorimotor function retards sensorimotor function. Visual fatigue is a widely used standard for display evaluation. Visual scanning tasks that are exposed on monitor for a long period of time often resulit in visual fatigue, with this being related to the overloading of visual tasks. Fatigue has a greater impact on eye movement mechanics and increases the task time. It is associated with declines in performance (Park, et al., 2017), headaches and reduced saccade speed.

Eye hand coordination is a basic human ability o that consists of cognition, eye movement, limb movement and vision. Studies affirm that a transforma- tive relationship relating to time and space is evidenced between the hand and eye in natural human behavior. (Hayhoe, et al., 2003). Vision positi- vely affects the guidance of manual movement (Steinman, et al., 2003) and also improves their accuracy. While the position of the hands provides visual feedback, visual saccades are often 50-100 milliseconds ahead of manual movement (Vercher, et al., 1994).

In natural circumstances, the eye, hand and head are in the context of ongoing behavior, in continual motion. Normal behavior requires the spatial

Influence of Head Up Display on Visual Fatigue and Eye-Hand Discoordination 45

H V1

V O

Figure 1: Runway incursion scenario.
Original PDF Figure 1:

and temporal coordination of movements (Pelz, et al, 2001). Skilled motor behaviors involve the ability to predict the consequences of actions, while motor learning involves the acquisition of new maps that related to motor commands and desirable sensory outcomes. (Sailer, et al, 2005). Most studies that address the learning of visually guided manual skills examine goal- directed conduct that occurs under novel load or visuomotor conditions (Flanagan et al., 2003). Some studies argue that visually guided manual tasks are controlled by action schema that bring in sensorimotor routines for manual action and also incorporate accompanying routines that specify task-specific eye movements that support the planning and control of manual action. (Flanagan & Johansson, 2003). Some degree of coupling between the oculomotor and limb motor systems has been demonstrated by assessing the relative onset times of eye and arm movement, in addition to the demon- stration of a gap effect for arm movement reaction times. When in flight, reasonable eye hand coordination could improve flight quality and flight errors, while helping to ensure flight safety.

This would appear to suggest that further investigations are required in order to illustrate how HUD impacts on visual fatigue and eye hand discoordination during runway incursion scenarios.

METHODS

Runway Incursions Scenario ICAO divides runway incursions into several recurring scenarios, which most commonly include: a) an aircraft or vehicle crossing in front of a landing aircraft; b) an aircraft or vehicle crossing in front of an aircraft taking off; c) an aircraft or vehicle crossing the runway-holding position marking; d) an aircraft or vehicle inadvertently entering an active runway; e) a breakdown in communications resulting in a failure to follow instructions from air traffic control instruction; f) an aircraft passing behind an aircraft or vehicle that has not vacated the runway. (ICAO, 2007). In referring to runway incursi- ons performed by NASA, we selected a runway incursion scenario in which aircraft traveling at a speed of V1 passes in front of a landing aircraft at a speed of V and a height of H. (NASA, 2000). Figure 1 sets out this scenario in more detail.

Figure 2: Linear fitting coefficient of pupil diameter.
Original PDF Figure 2:
Table 1. Runway incursion experiments.

Table available in the original PDF paper.

Task Display Condition Phase of Flight Scenario Duration (min) 1 HUD Landing Runway incursion 5 2 PFD Landing Runway incursion 5 3 HUD Landing Normal 5 4 PFD Landing Normal 5

Subjects Twenty-three male flying cadets (M = 25 yrs., SD = 0.87) were recrui- ted from Civil Aviation University of China. All participants were required to take flight training until they were able to independently and skillfully complete airfield traffic pattern, in accordance with the Instrument Flying Handbook, the Air Flight Manual and, the Pilot Operation Handbook. In addition, they would need to be evaluated and approved against the Catego- ries I and II criteria for using HUD that are set out in the A320 Flight Crew Operating Manual.

Experiments Different types of displays (HUD and PFD) and landing scenarios (normal scenarios and runway incursions) would affect eye-hand coordination and visual fatigue. The experimental tasks were designed using a single variable method to analyze how HUD and PFD affect experimental results. The expe- rimental tasks, which were performed upon the basis of the A320 Cockpit Vision Simulation System, are set out in Table 1.

Procedure During training, each subject flew a minimum of four trials, the intention of becoming familiar with the A320 visual simulation system. Once each parti- cipant had mastered the ability to fly, they then began the experimental trials. All subjects were given a three-minute briefing that explained the tasks. Sub- sequent to the briefing, an open Tobii Eyetracker and high-resolution camera

Influence of Head Up Display on Visual Fatigue and Eye-Hand Discoordination 47

were used to record data relating to eye movement and hand manipulation. In order to ensure the validity of the experiment results, participants did not know if the runway incursions occurred before the test. In operationali- zing the premise that the equipment was operating as normal, subjects began the formal experiment, acting in accordance with the experimental sequence designed by standard Latin square. Each participant established a smooth approach and landing, operating at a height of 1200ft without turbulence. The indicated airspeed was 140 knots, auto-throttle was enabled, flaps were set to 30 degrees and the landing gear was down. Once the invasion of the aircraft is detected, subjects must decide to ‘go-around’. At the beginning of the recording of hand manipulation, an artificial mark was added with the intention of synchronizing the eye movement information and the hand motion video.

RESULTS

Pupil Diameter The results of tasks one and two that fourteen subjects successfully detected the invading aircraft with HUD, with the number rising to 17 when PFD was applied. Eleven of 23 subjects detected runway incursions on tasks one and two. The pupil diameters of these subjects were analyzed through the application of the linear fitting method.

As Figure 2 demonstrates, the linear fitting coefficient of pupil diameter using HUD is substantially negative and less than the value of PFD which is basically greater than zero. The linear regression equation for pupil diameter changes during runway incursion with HUD and PFD is shown in Eq.1.

yHUD = −0.000055t + 3.25 yPFD = 0.000009t + 3.25 (1) Where yHUD and yPFD respectively represent the pupil diameter of HUD and PFD, t is the sampling point.

This proved that pupil diameter is a reliable indicator for evaluating wor- kload and visual fatigue. And, it is known that progressively smaller pupil diameters indicate increased fatigue. As Eq.1 demonstrates, during the dete- ction of runway incursion, the pupil diameter associated with the use of HUD presents a downward trend. Conversely, when PFD is used, there is a slight increase in the trend. It can be extrapolated that the use of HUD to detect runway incursions is more likely to cause visual fatigue than PFD, during the landing phase. Because visual fatigue endangers flight safety, HUD is not therefore conducive to aviation safety when an aircraft is landing.

In an attempt to analyze the significance of HUD’s and PFD’s effect on pupil diameter, we performed ANOVA analysis on pupil diameter data obtai- ned from the left and right eyes of eleven subjects. The significance of left and right are respectively p1 = 0.040 and, p2 = 0.025, both of which are less than 0.05. It is therefore concluded that PFD and HUD significantly affect have a significant effect on the pupil diameter when detecting runway incursion. Visual fatigue is easily affected by the flight display, and the HUD’s influence is negative.

Figure 3: Linear fitting coefficient of pupil diameter.
Original PDF Figure 3:

Hand Manipulation Duration and Frequency When in flight, pilots exert control by adjusting the flying flaps, pedals, rocker and throttle sticks. Good flight control can improve flight accuracy and quality. In contrast to the traditional PFD, HUD can reduce the pilot’s workload by using advanced flight display technology this affects the pilot’s hand manipulation behavior and contributes to different control behaviors in the runway incursion scenario. The duration and frequency were analyzed and the results of the statistical analysis are set out in Figure 3.

Figure 3 depicts the manipulation frequency and duration that occur when HUD and PFD are used to detect runway incursion during the landing phase. The average manipulation frequency is 0.46 times/sec (SD = 0.43 with a 95% confidence interval (0.16, 0.77)), which is lower than the 0.58 time- s/sec (SD = 0.39 with a 95% confidence interval (0.33, 0.83)) that occurs when PFD is applied. When compared against PFD, it becomes apparent that the use of HUD to detect runway incursions has a lower manual control fre- quency during landing phases. The analysis of manipulation duration reveals that in the case of HUD it is 5.28 sec, which is lower than PFD (5.89 sec). It can be speculated that the application of HUD reduces the physical load on pilots by helping them to detect a runway incursion during the landing phase. The results of ANOVA show a significance value of P = 0.527 >0.05, which makes it possible to infer that HUD and PFD do not significantly affect manipulation frequency and duration during the detection of a runway incursion.

Eye-Hand Discoordination Hand-eye cooperation entails that pilots will, in referring to the flight instru- ment and noticing that the flight attitude is due to deviate from the expected setting, take pilots take a reasonable action to maintain or approach the fli- ght attitude. Once pilots detect that the flight has deviated from the desired state, no effective action will permit movement away from the expected state. This is considered to be Eye-Hand Discoordination (EHD).

During the experiments, a video camera was used to record hand mani- pulation behavior during the application of HUD and PFD to detect runway incursion. The Number and duration of manipulations that subjects used to adjust an airplane’s heading, pitch and roll were extracted by Noldus

Influence of Head Up Display on Visual Fatigue and Eye-Hand Discoordination 49

Figure 4: EHD of HUD and PFD.
Original PDF Figure 4:

Observer upon the basis of video frames. Eye movements and manual data were synchronized with artificial markers in the video. Figure 4 sets out the statistical analysis of EHD.

The preceding Figure illustrates the total number of action and EHDs that arise when HUD and PFD to detect runway incursion during the landing phase. As can be seen, the EHD of HUD is 20.45% (SD = 0.18, 95% confide- nce interval = (0.08, 0.29)). This compares with 9.16% for PFD (SD = 0.08, 95% confidence interval = (0.04, 0.13)), meaning that the EHD of HUD is 11.29% higher than PFD. This confirms that HUD is more likely to lead to EHD, with the result that the possibility of pilot disoperation increases, along with the impairment of flight safety.

CONCLUSION

This research focuses on exploring how HUD affects visual fatigue and EHD detections of runway incursion during the landing stage of flights. With the intention of analyzing changes in eye and manual operation, an experiment was conducted that sought to detect runway incursions through the applica- tion of an A320 simulation platform. A comparison was made against PFD, and the use of HUD to detect runway incursions during the landing phase. The following conclusions were then obtained: (1) The pupil diameter of the subject showed a downward trend, and this contributed to visual fatigue and the impairment of flight safety. (2) There is a lower manipulation frequency and duration, and this helps to reduce the pilot’s physical load. (3) High EHD can easily trigger flight pilot errors and is not conducive to flight safety and quality.

ACKNOWLEDGMENT

Supported by R&D Program of Beijing Municipal Education Commission (KM202210037001).

REFERENCES

Borghini, G., Astolfi, L., Vecchiato, G., Mattia, D., & Babiloni, F. (2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neurosci Biobehav Rev, 44, 58–75. Cabon, P., Deharvengt, S., Grau, J. Y., Maille, N., Berechet, I., & Mollard, R. (2012).

Research and guidelines for implementing fatigue risk management systems for the French regional airlines. Accident Analysis and Prevention, 45(1), 41–44. Dawson, D., Chapman, J., & Thomas, M. J. W. (2012). Fatigue-proofing: a new approach to reducing fatigue-related risk using the principles of error manage- ment. Sleep Medicine Reviews, 16(2), 167–175. Dawson, D., Searle, A. K., & Paterson, J. L. (2014). Look before you sleep: evaluating

the use of fatigue detection technologies within a fatigue risk management system for the road transport industry. Sleep Medicine Reviews, 18(2), 141–152. Filtness, A. J., & Naweed, A. (2017). Causes, consequences and countermeasures to driver fatigue in the rail industry: the train driver perspective. Applied Ergonomics, 60, 12–21. Flanagan, J. R., & Johansson, R. S. (2003). Action plans used in action observation.

Nature, 424(6950), 769–771. Flanagan, J. R., Vetter, P., Johansson, R. S., & Wolpert, A. D. M. (2003). Prediction precedes control in motor learning. Current Biology Cb, 13(2), 146–150. Fu, R., Wang, H., & Zhao, W. (2016). Dynamic driver fatigue detection using hidden

Markov model in real driving condition. Pergamon Press, Inc. Härmä, M. (2006). Workhours in relation to work stress, recovery and health.

Scandinavian Journal of Work Environment & Health, 32(6), 502-514. Hayhoe, M. M., Shrivastava, A., Mruczek, R., & Pelz, J. B. (2003). Visual memory and motor planning in a natural task. J Vis, 3(1), 49–63. Horrey, W. J., Wickens, C. D., & Alexander, A. L. (2003). The effects of head-up

display clutter and in-vehicle display separation on concurrent driving performa- nce. Human Factors & Ergonomics Society Annual Meeting Proceedings, 47(16), 1880–1884. International Air Transport Association. (2002). Runway Incursion Prevention Program. Icao Nam/Car/Sam Runway Safety/Incursion Conference, Mexico City, 22 to 25 October 2002. International Civil Avation Organization. (2007). Manual on the Prevention of

Runway Incursions. International Civil Aviation Organization. (2016). Global Aviation Safety Plan 2017-2019. International Civil Aviation Organization. (2017). Runway Safety Programme Glo-

bal Runway Safety Action Plan. Nakayama, M., Nowak, W., Ishikawa, H., Asakawa, K., & Ichibe, Y. (2014). Discovering irregular pupil light responses to chromatic stimuli using waveform shapes of pupillograms. Eurasip Journal on Bioinformatics & Systems Biology, (1), 18. National Aeronautics and Space Administration. (2000). Runway Incursion Pre-

vention: Cockpit displays could help reduce accidents. Langley Research Center, FS-2000-09-53-LaRC. Nie, B., Huang, X., Chen, Y., Li, A., Zhang, R., & Huang, J. (2017). Experimental study on visual detection for fatigue of fixed-position staff. Applied Ergonomics, 65, 1–11. Park, S., Choi, D., Yi, J., Lee, S., Lee, J. E., & Choi, B., et al. (2017). Effects of display

curvature, display zone, and task duration on legibility and visual fatigue during visual search task. Applied Ergonomics, 60, 183–193.

Influence of Head Up Display on Visual Fatigue and Eye-Hand Discoordination 51

Pelz, J., Hayhoe, M., & Loeber, R. (2001). The coordination of eye, head, and hand movements in a natural task. Experimental Brain Research, 139(3), 266–277. Prinzel, Lawrence, J., & III. (2004). Head-up displays and attention capture. NASA

Technical Memorandum, 62(4), 273–273. Sailer, U., Flanagan, J. R., & Johansson, R. S. (2005). Eye-hand coordination during learning of a novel visuomotor task. Journal of Neuroscience the Official Journal of the Society for Neuroscience, 25(39), 8833–42. Stasi, L. L. D., Contreras, D., Cañas, J. J., Cándido, A., Maldonado, A., & Catena, A.

(2010). The consequences of unexpected emotional sounds on driving behavior in risky situations. Safety Science, 48(10), 1463–1468. Steinman, R. M., Pizlo, Z., Forofonova, T. I., & Epelboim, J. (2003). One fixates accurately in order to see clearly not because one sees clearly. Spat Vis, 16(3-4), 225–241. Thomas, L. C., & Wickens, C. D. (2004). Eye-tracking and Individual Differences in

off-Normal Event Detection when Flying with a Synthetic Vision System Display. Human Factors and Ergonomics Society Meeting (Vol. 48, pp. 223–227). SAGE Publications. Vercher, J. L., Magenes, G., Prablanc, C., & Gauthier, G. M. (1994). Eye-headhand coordination in pointing at visual targets: spatial and temporal analysis. Experimental Brain Research, 99(3), 507–523. Wickens, C. D, Becky L. Hooey, Brian F. Gore, et al. (2009). Identifying Black Swans

in NextGen: Predicting Human Performance in Off-Nominal Conditions. HFES- HUMAN FACTORS. 51(5), 638–651.

Cover of the proceedings volume Human Error, Reliability, Resilience, and Performance

Explore the complete digital volume

Human Error, Reliability, Resilience, and Performance

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-09-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →