Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluating the Effectiveness of Mixed Reality as a Cybersickness Mitigation Strategy in Helicopter Flight Simulation

Royal Netherlands Aerospace Center, Anthony Fokkerweg 2, 1059CM, Amsterdam, The Netherlands ↗
Publication year
Digital Object Identifier10.54941/ahfe1003570

Abstract

The advent of Virtual Reality (VR) in flight simulation promises to provide a cost-effective alternative for flight crew training compared to conventional flight simulation methods. However, it has been noted that the use of VR in flight simulation can lead to a greater incidence of cybersickness, which could jeopardize the effectiveness of flight training in VR. To optimally leverage the benefits that VR in flight simulation can bring, it is critical that this higher likelihood of experiencing cybersickness is countered. Even though a variety of theories for the causes for cybersickness in VR have been formulated, one of the most widely-accepted theories hinges on the principle that the sensory conflict between the visual sensory inputs from the virtual environment and the motion that is sensed by the vestibular system can result in cybersickness. Minimizing this sensory conflict can therefore be a strategy to mitigate cybersickness. The use of Mixed Reality (MR), in which the virtual environment is visually blended with the actual environment, could potentially be used for this strategy, based on the idea that it provides a visual reference of the actual environment that corresponds with the motion that is sensed, thereby reducing the sensory conflict and, correspondingly, cybersickness.The objective of this research is to investigate the effectiveness of MR, as an alternative for VR, for the mitigation of cybersickness in helicopter flight simulation. Since the idea of using MR as a cybersickness mitigation strategy is rooted in the idea of reducing the mismatch between visual and vestibular sensory inputs, the effectiveness of MR in combination with simulator motion is investigated as well. Arguably, MR could deteriorate immersion and reduce simulation fidelity, which may hamper the ability of the pilot to adequately fly in the virtual environment. Based on this premise, it is expected that a sweet spot exists where cybersickness is reduced, while fidelity remains sufficient to perform the flying task satisfactorily. In addition to evaluating the effectiveness for cybersickness mitigation, the impact of MR on pilot performance is also investigated.A human-in-the-loop experiment was performed that featured a total of four conditions, designed to assess the impact of both MR and motion on the cybersickness development and pilot performance. The experiment was performed in a simulated AgustaWestland AW139 helicopter on a Motion Systems’ PS-6TM-150 motion platform (6DoF), combined with a Varjo XR-3 visual device. In the experiment, Royal Netherlands Air Force helicopter pilots (n=4) were instructed to fly a series of maneuvers from the ADS-33 helicopter handling qualities guidelines. The Pirouette task in ADS-33 is the main focus for the results analysis because it is expected that near ground dynamic maneuvering affects cybersickness more severely compared to more stable and high altitude performed tasks.The cybersickness is evaluated by means of MIsery SCale (MISC) scores that were reported by the participants after the maneuver, and through a qualitative assessment triggered by VR comfort criteria. Pilot performance is assessed by examining the flight trajectories, helicopter inputs, and relevant helicopter outputs.The experiment campaign was completed in the last quarter of 2022, for which the results analysis is still ongoing. Preliminary results suggest that MR in the absence of simulator motion can have a beneficial effect on the development of cybersickness, as lower MISC scores were observed compared to the other conditions. However, the addition of motion with MR seems to have the opposite effect on cybersickness. Key in this result is that participants mentioned that being able to observe the simulator motion washout – the sensory-subliminal movement which allows the simulator to return to the neutral position – was an important sickening factor. The paper features a more in-depth analysis of the MISC scores and post-exposure sickness questionnaire results to further support the findings regarding the effect of MR and motion on cybersickness.While the analysis of the pilot performance results requires additional effort, current analysis of the flight trajectories and helicopter outputs suggests that MR may have a detrimental effect on pilot performance. The analysis of the pilots’ control input, however, still has to be completed. Next to an analysis of the flight trajectories and the helicopter outputs, the paper’s results also includes an analysis of the pilots’ control input – both in the time and frequency domains – to allow for a more detailed analysis on how the mitigating measures affect the individuals’ helicopter handling qualities and to which extent this influences the experienced cybersickness.

INTRODUCTION

The advent of Virtual Reality (VR) in flight simulation promises to provide a cost-effective alternative for flight crew training compared to conventional flight simulation methods. However, it has been noted that the use of VR in flight simulation can lead to a greater incidence of cybersickness compared to conventional simulation methods, which could jeopardize the effectiveness of flight training in VR (Chang, Kim, & Yoo, 2020). To optimally leverage the benefits of VR in flight simulation, it is critical that this higher likelihood of experiencing cybersickness is countered. While a variety of theories for the causes for cybersickness in VR have been formulated, one of the most widely- accepted theories hinges on the principle that the sensory conflict between visual sensory inputs from the virtual environment and the motion that is sensed by the vestibular system can result in cybersickness (Chang, Kim. & Yoo, 2020; Palmisano, Allison, & Kim, 2020; Reason & Brand, 1977). The- refore, minimizing this sensory conflict can be an effective strategy to mitigate

© 2023. Published by AHFE Open Access. All rights reserved. 131

such sickness (NATO Science & Technology, 2021). The use of Mixed Reality (MR), in which the virtual environment is visually blended with the actual environment, could be used as a strategy, as it provides a visual reference of the actual environment that corresponds with the motion that is sensed, thereby reducing the sensory conflict and, accordingly, cybersickness.

The objective of this human-in-the-loop experiment is to investigate the effectiveness of MR as an alternative to VR in mitigating cybersickness in helicopter flight simulation. Since the idea of using MR as a cybersickness mitigation strategy is rooted in the idea of reducing the mismatch between visual and vestibular sensory inputs, the effectiveness of MR in combination with simulator motion is investigated as well. Arguably, MR could deteriorate immersion and reduce simsulation fidelity, which may hamper the ability of the pilot to adequately fly in the virtual environment. Hence, the impact of MR on pilot performance is also investigated. Based on this premise, it is expected that a sweet spot exists where cybersickness is reduced, while fidelity remains sufficient to perform the flying task satisfactorily. Based on the aforementioned literature, it is hypothesized that the use of MR leads to less experienced cybersickness compared to the baseline condition, but that the presence of MR hampers pilot performance.

METHOD

Participants The experiment is completed by a total of four Royal Dutch Air Force helico- pter pilots with a mean age of 35.8 years (SD = 6.83 years) and an average of 1634 flight hours (Min = 400, max = 4000). Three pilots mainly fly the Chinook, while one pilot has experience flying both the AW139 and the NH90.

To assess the susceptibility to cybersickness of the participants, the Motion Sickness Susceptibility Questionnaire (MSSQ) was used (Goldin, 2006). The four participants reported a mean MSSQ score of 8.75 (SD = 7.89), impl- ying that the mean score of the participants falls between the 40th and 50th percentile for motion sickness susceptibility, indicating that the participants are slightly less susceptible to motion sickness than the population average (Golding, 2006). Hence, it is assumed that they can complete the experiment without excessive sickness.

Materials

Simulation Setup The experiment has been performed in a simulated AgustaWestland AW139 helicopter on a Motion Systems’ PS-6TM-150 motion platform combined with a Varjo XR-3 visual device. The PS-6TM-150 motion platform is a compact 6 Degrees of Freedom (DoF) system, for which the motion was cued by means of a classical washout motion cueing algorithm (Reid & Nahon, 1985). The Varjo XR-3 is an enhanced MR device, which has a high-resolution focus area (27

  • x 27
  • ) and two RGB cameras mounted to provide video input. The virtual environment is generated in Unity version

Evaluating the Effectiveness of Mixed Reality as a Cybersickness Mitigation Strategy 133

2020.3.4. To standardize each pilot’s experiment flight profile, a scenario is designed with multiple Mission Task Elements (MTE) from the ADS-33E- PRF (Baskett, 2000). Despite the fact that multiple MTEs were flown in the scenario, the ADS-33E-PRF pirouette MTE is the main focus for the results analysis. In the pirouette MTE, the pilot is tasked to fly a circular trajectory, keeping the heading aligned with the center pole at a fixed distance and hei- ght. A 15-knots wind at a fixed direction was included in the scenario. It is expected that the near-ground, dynamic maneuvering in the pirouette MTE affects cybersickness more severely compared to more stable and high altitude performed tasks, due to greater levels vection leading to increased sensory conflict (Lawson, 2014; Zelie & Qadeer, 2019). A visual representation of the pirouette flight task is presented in Figure 1. The corresponding pirouette performance specifications can be found in Table 1.

Figure 1: Pirouette MTE procedures and performance limits (Baskett, 2000), combined with the virtual environment implementation of the pirouette MTE course.
Original PDF Figure 1:
Table 1. ADS-33E-PRF pirouette MTE performance specifications (Baskett, 2000).

Table available in the original PDF paper.

Performance Specification Desired Performance Adequate Performance ± 10 ft ± 15 ft Maintain distance to circle center (100 ft)

± 3 ft ± 10 ft

Maintain radar altitude / height (15 ft) Complete full circle within 45 seconds (∼8 kts) 60 seconds (∼6 kts)

The implementation of the pirouette MTE parcourse in the virtual envi- ronment is shown in Figure 1. The black line denotes the reference flight trajectory, while the inner and outer edges of the adjacent, thick orange bands denote the limits for desired and adequate performance respectively, as stipulated in Table 1.

Cybersickness Measurements Since it is known that motion sickness susceptibility can vary to an extent within a population, the experiment participants will complete Motion Sickness Susceptibility Questionnaire (MSSQ) developed by Golding (2006) prior to participating, in order to evaluate these variations in motion sickness susceptibility. The results of this questionnaire can be used to explain ano- malies or noteworthy outliers in the cybersickness results. Moreover, partici- pants will complete the Misery Scale (MISC) after each run. This scale runs from 0 (“No problems”) to 10 (“Vomiting”).

Research Design Four conditions were designed to test for the impact of visual referencing using MR and motion on cybersickness and flight performance. A visual overview of the four experiment conditions is presented in Figure 2.

Figure 2: Experiment conditions, as seen from the pilot’s eye reference point.
Original PDF Figure 2:

Evaluating the Effectiveness of Mixed Reality as a Cybersickness Mitigation Strategy 135

The MR Reference conditions display a visual projection for which the out- of-the-window view in the virtual helicopter cockpit has an 80–20 ‘virtual environment versus real world environment’ transparency ratio, facilitated by the mounted cameras of the MR device. This ratio is chosen arbitrarily and may require additional research into the effectiveness of different tran- sparency ratios on performance and cybersickness. As seen in Figure 2, both the virtual and actual worlds are clearly distinguishable in the MR Reference conditions. The No Motion – No Visual Reference condition is referenced to as a baseline.

All participants complete the four experiment conditions in one day, using a predetermined counterbalanced order. Prior to the experiment, participants are briefed on the use of the MIsery SCale (MISC; Bos, Mackinnon, & Pat- terson, 2006), and fly a familiarization run through the virtual environment in the No Motion – No Visual Reference configuration. After the familiari- zation run, the participants can rest until they report a MISC score of 0, to ensure there are no carryover effects from the familiarization.

In each experimental condition, participants complete the virtual course, featuring all selected MTEs – including the pirouette. The participants are instructed to complete the MTEs twice to attain a total scenario duration of approximately twenty minutes, which was selected to allow for suffici- ent time for cybersickness to develop. Upon completion, the participants are prompted to report their MISC score. Reported cybersickness should not be too severe for the recovery time to be limited to such an extent that it can affect the cybersickness development in the next condition. As such, the expe- rimental condition is aborted once a participant reached MISC 3, implying light cybersickness symptoms, such as stomach awareness and dizziness.

After completing a condition, participants are instructed to rest to recover from any cybersickness symptoms. Once the participant reports a score of MISC 0, the next condition is initiated.

Dependent Measures and Data Analysis For the present paper, no formal statistical analyses are conducted due to the limited sample size and the exploratory nature of this experiment. Howe- ver, several parameters are assessed. For cybersickness, descriptive statistics and visualizations of these are used to investigate the differences in repor- ted MISC scores between conditions. To assess performance, several relevant pirouette performance parameters, including the height deviation, distance deviation, and deviation from the heading corresponding to a given position are analyzed. In addition, flight trajectories are mapped for each condition and compared to the performance requirements as detailed in Table 1.

RESULTS

Cybersickness The average MISC scores per condition (Figure 3) show that the MR static condition performs best in terms of average reported MISC score. This is followed by the baseline condition, the VR motion condition and finally, with the highest average MISC score, the MR motion condition. Note that for both

Figure 3: Average MISC scores with standard error for all four conditions.
Original PDF Figure 3:

the VR and MR conditions, the impact of motion is negative concerning the MISC. The standard error is similar for all conditions (SE ≈0.25), implying a relatively high inter-personal variability.

Pilot Performance

Figure 4 shows boxplots for the pirouette MTE height, distance, and hea- ding deviation computed for all available datapoints. Reviewing the pirouette MTE performance specifications, presented in Table 1, it can be observed from Figure 4 that for all pilots adhering to the performance margins is a challenging task. For each condition, large deviations (beyond adequate performance standards) in height, distance to centre pole, and heading are detected. Thus, it is appropriate to say that considerable sway, heave, and surge occurred independent of the condition. For height and heading control,

Figure 4: Pirouette MTE performance results.
Original PDF Figure 4:

Evaluating the Effectiveness of Mixed Reality as a Cybersickness Mitigation Strategy 137

it can be observed that the MR Reference – No Motion condition resul- ted in the lowest deviations from the desired performance specifications. In contrast, the distance deviation appears to be relatively high in the MR Refe- rence – No Motion condition, compared to the other conditions. Motion appeared to negatively impact performance in terms of height and heading control.

Figure 5 visualizes the top-down view of the flight trajectories for the pirouettes flown, for both the first and the second attempt in a condition. The MR Reference – No Motion condition reveals a more severe deviation from the reference flight path compared to the other conditions for at least three out of four participants, thereby reinforcing the findings reported for distance deviation in Figure 4. The motion appears to positively affect the pirouette performance in terms of the accuracy of the flight trajectories, particularly in the MR Reference conditions.

Figure 5: Pirouette MTE flight trajectories for the experiment conditions.
Original PDF Figure 5:

CONCLUSION

The aim of this research was to investigate the effectiveness of MR, as an alternative to VR, as a mitigation measure for cybersickness in helico- pter flight simulation. Due to the exploratory character of the research and

small sample size, trends are discussed instead of results based on statistical analyses.

It was expected that the use of MR would mitigate cybersickness, while leading to a deterioration in pilot performance, when compared to VR. For the limited number of participants, the MR Reference – No Motion condi- tion resulted in the least severe cybersickness, indicated by the lowest MISC scores. Mixed results were found for the impact of MR on pilot perfor- mance. While no noteworthy differences could be observed for the height and heading deviations between the MR Reference and No Visual Refere- nce conditions, a deterioration in the distance deviation, i.e., the accuracy of the flight trajectories in the horizontal plane, was found for the MR Refe- rence – No Motion condition. These findings therefore show that the MR Reference – No Motion condition can be effective for cybersickness mitiga- tion, while potentially having detrimental side effects on some aspects of pilot performance.

It was found, however, that the MR Reference – Motion condition led to the most severe cybersickness of all conditions, based on the MISC score. The substantial difference in the MISC score between the Motion and No Motion conditions for the MR Reference is likely to be due to the additional sensory conflict introduced by the visual perception the wash- out of the motion platform, as was reported by the participants. Since motion that is coherent with the perceived visual corresponds to a low sensory conflict, resulting in less severe cybersickness, this could be an indication that the simulated motion was not coherent enough with the displayed visual. In terms of pilot performance, motion itself seemed to nega- tively affect pilot performance in terms of height and heading deviations, while not being detrimental for distance deviation from the reference flight trajectory.

For the purpose of cybersickness mitigation, it can be concluded that the use of MR can be beneficial compared to using VR only in the absence of simulator motion, due to the additional sensory conflict of visually percei- ving the motion washout in MR. As MR without simulator motion therefore seems promising for cybersickness mitigation, but does seemingly lead to a deterioration in pilot performance when it comes to the accuracy of the flown flight trajectories, it would be interesting to investigate the optimization of the MR reference to the physical world for both cybersickness and performa- nce. An optimization study that, for example, focuses on the effect of different MR transparency ratios in order to find a sweet spot where cybersickness is reduced while the pilot performance is not affected could therefore constitute an interesting future research avenue.

ACKNOWLEDGMENT

This research is part of a Royal Netherlands Aerospace Center internally funded research program: Immersive Simulation. The presented results are part of a series of publications. We thank the involved Royal Dutch Air Force pilots who participated in the experiments.

Evaluating the Effectiveness of Mixed Reality as a Cybersickness Mitigation Strategy 139

REFERENCES

Baskett, B. (2000). ADS-33E-PRF Aeronautical Design Standard Performance Specification – Handling qualities requirements for military rotorcraft. Defense Technical Information Center. Bos, J. E., Mackinnon, S., & Patterson, A. (2006). Motion sickness symptoms in

a ship simulator: Effects of inside, outside, and no view. Aviation, Space, amd Environmental Medicine, 76(12), 1111 - 1118. Chang, E., Kim, H. T., & Yoo, B. (2020). Virtual Reality sickness: A review of causes and measurements. International Journal of Human–Computer Interaction, 36(17), 1658–1682. https://doi.org/10.1080/10447318.2020.1778351 Golding, J. F. (2006). Predicting individual differences in motion sickness suscepti-

bility by questionnaire. Personality and Individual Differences, 41(2), 237–248. https://doi.org/10.1016/j.paid.2006.01.012 Lawson, B. (2014). Motion sickness symptomatology and origins. (Hale, K. & Stanney, K. M. Eds.) Handbook of Virtual Environment: Design, Implementation, and Applications, 531–600. Boca Raton, FL: CRC Press NATO Science & Technology. (2021). Guidelines for mitigating cybersickness in

Virtual Reality systems. Peer-Reviewed Final Report of the Human Factors and Medicine Panel/Modeling & Simulations Group Activity Number 323 (NATO STO-TR-HFM-MSG-323). Palmisano, S., Allison, R. S., & Kim, J. (2020). Cybersickness in head-mounted displays is caused by differences in the user’s virtual and physical head pose. Frontiers in Virtual Reality, 587698. https://doi.org/10.3389/frvir.2020.587698. Reason, J. T., & Brand, J. J. (1977). Motion sickness. The American Journal of

Psychology, 90(1), 1822–188. Reid, L. D., & Nahon, M. A. (1985, December). Flight simulation motion-base drive algorithms: Part 1, developing and testing equations. UTIAS Report No. 296. Zelie, B., & Qadeer, A. (2019). Vestibular and mulit-sensory influences upon

self-motion perception and the consequences for human behavior. Frontiers in Neurology, 10, 1664–2295. https://doi.org/10.3389/fneur.2019.00063

Cover of the proceedings volume Human Factors and Simulation

Explore the complete digital volume

Human Factors and Simulation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-59-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →