Troy Weekes, Kazuhiko Momose, Thomas Eskridge (2023). Neuroergonomics of Cursor Control Devices in Spacecraft Cockpits for Spaceflight Participants. In: Neuroergonomics and Cognitive Engineering. AHFE Open Access, vol. 102, pp. 182–191. https://doi.org/10.54941/ahfe1003018
The commercial space transportation industry is rapidly growing with increasing numbers of spaceflight participants (SFPs). These private individuals receive considerably less training than astronauts before embarking on space missions, which presents an urgent need to develop the cognitive ergonomics that simplify spacecraft cockpit design. Neuroergonomics is an emerging area within cognitive ergonomics, which Parasuraman described as “the study of brain and behavior at work”. This experimental study investigated the neuroergonomics of cursor control devices (CCDs) for spacecraft cockpits by applying electroencephalography (EEG) power indices as objective measures of concentration, relaxation, effort, fatigue, arousal, valence, and absorption during task performance. Data for this study were collected from a sample of twenty-seven participants who performed a Fitt’s cursor control task in PsyToolkit with a counterbalanced device sequence of four different CCDs, i.e., touchpad, touchscreen, joystick, and numpad. The devices were affixed to, and configured in the variable positioning Adaptive Spaceship Cockpit Simulator, which was used to simulate the microgravity environment of space using head-down tilt (HDT). The index of difficulty of the cursor control task trials was varied according to Fitt's law across easy, medium, and difficult levels. The orientation of the simulator varied between upright and HDT orientations. We administered a HDT treatment before the experimental trials in the HDT orientation to induce the physiological effects associated with increased intraocular pressure, which results from microgravity. A HDT recovery period was administered after the experimental trials in the HDT orientation. Participants completed a subjective questionnaire to capture perceived effort at the end of each experimental track. Using the Flow Choice Architecture, we processed EEG signals to compute EEG power indices for a Multivariate Analysis of Variance. There were significant findings in concentration across CCDs during the two orientations. The HDT orientation demanded more concentration than the upright orientation across the devices. This result indicated that there was additional cognitive workload induced by manipulating the CCDs in the HDT orientation. There were significant differences in fatigue across the two orientations. The HDT orientation was associated with greater fatigue levels. An important finding in the subjective questionnaire was the perceived effect of the HDT orientation on cognition. The touchpad consistently demonstrated differences relative to the other CCDs. Task difficulty did not significantly impact any of the EEG indices. No significant interactions were observed in the EEG indices across the orientations, devices, and task difficulty levels. A striking result emerged during the HDT recovery period where most participants exhibited a sleepy-like EEG signature characterized by a consistently high relaxation index. Overall, these results indicated that computational neuroergonomics may produce objective insights about the human spaceflight experience related to orientation and cursor control devices. We recommend that strategies to enhance spacecraft cockpit design include neuroergonomics of CCDs, control devices, and user interfaces, in general.
Neuroergonomics of Cursor Control Devices in Spacecraft Cockpits 183
INTRODUCTION
Recent advances in passenger-carrying commercial spaceflight have created the need for individuals of varying backgrounds and capabilities to safely per- form control tasks in spacecraft cockpits. Spacecraft are unique vehicles that operate under variable gravitational conditions and orientations depending on the different phases of the spaceflight mission.
Our research problem centers on quantifying how well spaceflight par- ticipants (SFPs) can perform control tasks in terrestrial and microgravity situations using different cursor control devices (CCDs). How might we design spacecraft cockpits with optimal CCDs based on neuroergonomics to have the highest positive impacts and least negative impacts on the task performance of SFPs?
Control tasks in vehicle settings have traditionally been evaluated using physical and cognitive ergonomics. The application of cognitive approaches demonstrates a trend to directly measure the neural dynamics underlying human behavior and task performance. Our quantitative approach uses the combined frequencies of neural signals to model the effects of work tasks, orientations, and technologies on the human brain.
Parasuraman (2003) introduced neuroergonomics as the study of the brain and behavior in naturalistic work contexts by examining brain signatures related to human task performance. Neuroergonomics merges neuroscie- nce and ergonomics to explain the neural mechanisms underlying cognitive and motor functioning (Parasuraman & Wilson, 2008). Neuroergonomics is well-positioned as a tool for scientific inquiry that can provide a deeper understanding of human performance by exploring how the brain works during different tasks and situations.
This paper seeks to provide evidence of the cognitive factors at play during human spaceflight control tasks by investigating the neuroergonomics of the CCDs that are commonly used in spacecraft cockpits. We computed seven power indices from electroencephalography (EEG) data to gain objective insi- ghts into the neurocognitive and neuroaffective states of SFPs performing a Fitts (1954) law cursor control task with different CCDs in a spacecraft cock- pit simulator environment. We hypothesized that certain CCDs have better neuroergonomics. Further, we predicted that the effects of microgravity on visual processing as simulated by head-down tilt (HDT) have a significant impact on the neuroergonomics of CCDs for SFPs.
RELATED WORK
This paper focuses on SFPs, who are not crew but space tourists and consu- mers of commercial space transportation (FAA, 2006). The spacecraft cockpit presents a potentially challenging work environment considering the indivi- dual differences among SFPs. The controls of the cockpit user interfaces (UIs) are peripheral devices that transfer human inputs into spacecraft operations. We hypothesized that gravity effects and device type would affect the way SFPs perform cursor control tasks and would be observed in neural signals.
Computational neuroergonomics analyzes time-based and frequency- based EEG features of brain activity to determine the effects of performing a task in a given situation and context (Johnson & Krusienski, 2018). Table 1
Table 1. Experiments highlighting EEG indices and their neural correlates.
Table available in the original PDF paper.
EEG Index Neural Correlates References (Grammer et al., 2021) Concentration Higher frontal beta power Lower global theta power Higher global beta/theta ratio Lower frontal theta/alpha ratio
(Freeman et al., 1999) (Prinzel et al., 2000) (Berka et al., 2007) (Teplan et al., 2014)
Relaxation Higher global alpha power Lower beta power Higher theta power Lower global delta power
(Boksem et al., 2005) (De Gennaro et al., 2007) (Cheng & Hsu, 2011)
Effort Higher beta/alpha ratios (Keller, 2007) (Berka et al., 2007) Fatigue Higher fronto-central delta power Higher fronto-central theta power Lower global relative beta power Arousal Higher frontal alpha power Lower parietal delta power (Ota et al., 1996) (Reuderink et al., 2013) Valence Asymmetry in frontal alpha power Lower frontal theta power
(Nacke, 2009) (DeLosAngeles et al., 2016)
(Tomarken et al., 1990) (Alves et al., 2008) (Reuderink et al., 2013) Absorption Lower theta power Lower central beta power Lower central gamma
shows the neural correlates mapped to the neurocognitive and neuroaffe- ctive states found in the computational neuroscience literature. We combined the power spectral density (PSD) features from five frequency bands in multivariate functions.
METHOD
Research Questions Our research objective was to measure the effects of two important factors of ergonomics, i.e., work tools and environment. Due to the known effects of microgravity on visual perception, we anticipated that CCD task perfor- mance would decrease, and neural states would be negatively affected. We aimed to differentiate the neuroergonomics of different CCDs using a ran- domized controlled trial experiment. The following research questions (RQ) were defined according to the objectives of the study:
RQ1: What are the effects of orientation on the EEG indices?
RQ2: What are the effects of CCD on the EEG indices?
Experiment Design The experiment was based on a three-way (2 × 4 x 3), repeated measures design with three independent variables: (i) two seating orientations - upright and HDT, (ii) four CCDs - touchpad, touchscreen, joystick, and numpad, and (iii) three levels of task difficulty - easy, medium, and difficult. The dependent variables included EEG indices derived from the neural correlates identified
Neuroergonomics of Cursor Control Devices in Spacecraft Cockpits 185
in Table 1, i.e., concentration, relaxation, effort, fatigue, arousal, valence, and absorption.
Participants Twenty-seven healthy right-handed volunteers (15 males, 10 females, 1 non- binary, 1 preferred not to say; M = 22.5 years, SD = 5.2) wearing shirtsleeves were recruited to the experiment. Inclusion criteria were as follows: (i) height between 5 feet and 6 feet 3 inches, (ii) weight under 280 pounds, (iii) no eyesight conditions related to intraocular pressure, (iv) not using medications which might cause drowsiness, (v) no blood circulation problems, and (vi) not pregnant. The research study was reviewed and approved by the IRB at Florida Institute of Technology (IRB Number: 22-114).
Experiment Setting Adaptive Spaceship Cockpit Simulator (ASCS) The experiment was conducted in the ASCS (Doule, 2018), which was ado- pted for the wider research plan centered on participants in various seated orientations while wearing spacesuits and normal sleeve clothing. The ASCS was configured for right-handed participants seated in upright (0
) and HDT (34
) orientations.
PsyToolkit Cursor Control Task A cursor control task was customized for use with three CCDs connected via USB to a Raspberry Pi touchscreen display. The display presented ran- domized target squares of varying sizes and distances from four randomly sequenced starting locations using PsyToolkit (Stoet, 2010; 2017). The task was based on Fitt’s Law (Fitts, 1954), and trials involved measuring the time and accuracy with which the cursor could be controlled with the CCD to click on the starting square followed by the center of the target square.
Cursor Control Devices (CCDs) The participants used the following four devices to complete the control task: (i) touchpad - Apple Magic Trackpad, (ii) touchscreen - Raspberry Pi Touchscreen, (iii) joystick - Logitech Freedom 2.4 Cordless Joystick, and (iv) numpad - Jelly Comb 2.4G Number Pad. The CCDs were affixed to the control device platform of the ASCS by Velcro tapes that facilitated easy adju- stment and removal of the devices throughout the experiment. The order of the devices was counterbalanced to mitigate practice effects across the participants.
Muse EEG Headband Prior studies using the Muse EEG headband have found the device to be non- invasive and efficacious for scientific research (Cannard, 2021). The study participants wore a Muse EEG headband (Muse MU-02-BK-EN) that measu- red their neural signals. EEG signals were sampled at 256 Hz and transmitted via Bluetooth to a local database on a Windows desktop computer using a Bluetooth Low Energy Device dongle. Figure 1 depicts the active sensors of
Figure 1: EEG sensors in frontopolar, anterior frontal, and temporoparietal regions.Figure 2: FCA pipeline transforms raw EEG into labelled averages of normalized indices.
the headband that were located at AF7, AF8, TP9, and TP10 with a refere- nce sensor at FPz and two bridged grounds based on the 10–10 international sensor placement convention (Krigolson, 2017).
Flow Choice Architecture (FCA) EEG signals from the Muse headband were processed by the Flow Choice Architecture (FCA) (Weekes, 2021). FCA was configured with the expe- rimental procedure for each participant to guide the experimental sessi- ons. The FCA pipeline in Figure 2 utilized a series of steps to transform raw EEG signals into event-based averages of EEG indices to measure the neuroergonomics of the CCDs during specific task trials.
Procedure
Figure 3 shows the experiment procedure, which comprised the following six sessions: (i) briefing, (ii) familiarization, (iii) first orientation condition, (iv) second orientation condition, (v) post hoc questionnaire, and (vi) debriefing.
Neuroergonomics of Cursor Control Devices in Spacecraft Cockpits 187
Figure 3: Experimental sessions in Tracks A and B, excluding briefing and debriefing.Figure 4: Box plots of EEG indices: (A) concentration, (B) relaxation, (C) effort, (D) fatigue, (E) arousal, (F) valence, and (G) absorption, grouped across four devices then split into two orientations (n = 21). The differences of means between subgroups with the same device are depicted by the solid lines across orientations.
Data Analysis We conducted a three-way repeated measures MANOVA (α = .05) with post hoc pairwise comparisons. Six participants were excluded due to incomplete data, leaving 21 participant’s data for analysis.
RESULTS
Effects of Orientation on the EEG Indices
Figure 4 shows that orientation had significant effects on concentration, F(1,19) = 7.06, p <.01, η2 = .02 and fatigue, F(1,19) = 4.31, p <.05, η2 = .01. The HDT condition resulted in more concentration and fatigue.
Figure 5: Box plots of EEG indices: (A) concentration, (B) relaxation, (C) effort, (D) fatigue, (E) arousal, (F) valence, and (G) absorption, grouped across two orientati- ons then split into four devices (n = 21). The differences of means between subgroups with the same orientation are depicted by the solid lines across devices.
Effects of Device on the EEG Indices
Figure 5 shows that device significantly affected the EEG indices as follows: concentration, F(3,17) = 8.03, p <.01, η2 = .02; relaxation, F(3,17) = 10.09, p <.001, η2 = .08; effort, F(3,17) = 6.48, p <.005, η2 = .005; fatigue, F(3,17) = 6.46, p <.001, η2 = .05; arousal, F(3,17) = 13.00, p <.001, η2 = .10; and absorption, F(3,17) = 10.96, p <.001, η2 = .08.
Pairwise comparisons revealed that the numpad required significantly more concentration (p <.0001) than the joystick. The touchpad was signi- ficantly more relaxing to use than the joystick (p <.0001), numpad (p <.05), and touchscreen (p <.01). However, the touchpad required significantly more effort than the joystick (p <.01), numpad (p <.01), and touchscreen (p <.01). The touchpad also generated significantly more fatigue than the joystick (p <.0001), numpad (p <.05), and touchscreen (p <.01). The fatigue effect was corroborated by the increased effort using the touchpad. The touchpad generated significantly less arousal than the joystick (p <.0001), numpad (p <.0001), and touchscreen (p <.0001). The touchpad promoted significan- tly more absorption than the joystick (p <.0001), numpad (p <.01), and touchscreen (p <.01). Surprisingly, there were no significant effects on vale- nce across the devices despite the consistently lower means in the HDT orientation.
DISCUSSION
Different orientations and CCDs impacted the control task performance of participants in the ASCS. Orientation plays a critical role in determining the neuroergonomics of CCDs. During the HDT recovery period, most partici- pants exhibited a sleepy-like EEG signature characterized by a consistently
Neuroergonomics of Cursor Control Devices in Spacecraft Cockpits 189
high relaxation index. This finding is notable considering the impacts of dro- wsiness and decreased vigilance that are likely to degrade subsequent task performance.
The selection of control devices for spacecraft cockpits influences the neuroergonomics experienced by SFPs. Devices with poor neuroergonomics should be avoided to mitigate their negative impacts on task performance. The touchpad consistently generated differences relative to the touchscreen, joystick, and numpad. The neuroergonomics of the touchpad may be due to the following factors: (i) balance between challenge and skill, (ii) forgiving, and direct gestural touch input for cursor control, (iii) small, scaled, and well mapped cursor control envelope, and (iv) high accuracy in large and fine cursor control inputs.
Five participants agreed (0 - strongly agreed, 5 - somewhat agreed, n = 27) that there was an effect of the HDT orientation on their cognition. Eleven participants agreed (2 - strongly agreed, 9 - somewhat agreed, n = 27) that the HDT orientation affected their breathing. The perceived impacts of the HDT orientation on cognition and breathing need to be investigated in future work in spite of the objective EEG indices, which already indicate potential adverse impacts on SFPs, since these effects of microgravity on SFPs will be ubiquitous.
Overall, the results of this study suggest that computational neuroergo- nomics produce strong and objective insights about the human spaceflight experience. In this case, EEG-based neuroergonomics enabled the research- ers to observe the near-real time operational state of humans during situated task performance. This is especially significant when considering the com- plex control tasks that SFPs may be required to perform during spaceflight missions.
Limitations In this computationally intensive type of investigation, the acquisition and analysis of EEG data were problematic due to several sources of error. One source of error was variable signal quality, which is caused by movements and electrical noise.
The EEG indices measured across the participants were relatively consi- stent given the prevalence of individual differences in EEG datasets. However, the Bluetooth connection malfunctioned occasionally, which resulted in data being lost or not recorded for 6 of the 27 experiment sessions. The reduced dataset limited our ability to fully explore the range of individual differences.
The Muse EEG headband limited the data collection to four sensors. In spite of the sensors being well positioned to examine asymmetrical brain regions, the sparse spatial coverage of EEG signals restricted analyses from exploring neural correlates related to other brain regions.
The participants’ subjective responses about the effects of the HDT orien- tation on cognition and breathing may be influenced by recall bias depending on the orientation track. Future work should analyze the impacts of orienta- tion track on the dataset, and utilize sensors in subsequent studies to collect objective measures of breathing rates, heart rate variability, and galvanic skin response.
CONCLUSION
With increasing focus being placed on space tourism and human spaceflight with SFPs, it is becoming more important to validate the human-centered design of spacecraft cockpits to ensure mission success. This study applied computational neuroergonomics to human-system integration with the goal of enabling SFPs to operate control devices safely and productively on their spaceflight missions. The results provide meaningful interpretations of the participants’ neurocognitive and neuroaffective states. The touchpad device exhibited the most favorable neuroergonomics and HDT orientation negatively impacted the neuroergonomics of the SFPs. We recommend that strategies to enhance spacecraft cockpit design include neuroergonomics of devices and UIs, in general (Momose et al., 2023).
ACKNOWLEDGMENT
This study was funded by the Federal Aviation Administration (FAA) Center of Excellence (COE) for Commercial Space Transportation (CST) via FAA Cooperative Agreement 15-C-CST-FIT-016. The authors thank the FAA COE CST for funding. The opinions expressed are those of the authors and do not necessarily reflect the views of the FAA.
REFERENCES
Alves, N. T., Fukusima, S. S. & Aznar-Casanova, J. A. (2008), “Modelsof brain asymmetry in emotional processing”, Psychology & Neuroscience1, 63–66. Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V.
T., Olmstead, R. E., Tremoulet, P. D. & Craven, P. L. (2007), “EEG correlates of task engagement and mental workload invigilance, learning, and memory tasks”, Aviation, Space, and Environmental Medicine 78(5). Boksem, M. A., Meijman, T. F. & Lorist, M. M. (2005), “Effects of mental fatigue on attention: an ERP study”, Cognitive Brain Research25(1), 107–116. Cannard, C., Wahbeh, H. & Delorme, A. (2021), “Validating the weara- ble Muse headset for EEG spectral analysis and frontal alpha asymmetry”, in2021 IEEE International Conference on Bioinformatics and Biomedicine, IEEE, pp. 3603–3610. Cheng, S. Y. & Hsu, H. T. (2011), Mental fatigue measurement using EEG,
IntechOpen. De Gennaro, L., Marzano, C., Veniero, D., Moroni, F., Fratello, F., Curcio, G., Ferrara, M., Ferlazzo, F., Novelli, L., Pellicciari, M. C. et al. (2007), “Neurophysi- ological correlates of sleepiness: a combined TMS and EEG study”, Neuroimage 36(4), 1277–1287. DeLosAngeles, D., Williams, G., Burston, J., Fitzgibbon, S. P., Lewis, T. W.,
Grummett, T. S., Clark, C. R., Pope, K. J. & Willoughby, J. O. (2016), “Electro- encephalographic correlates of states of concentrative meditation”, International Journal of Psychophysiology 110, 27–39. Doule, O. (2018), Adaptive Spaceship Cockpit Architecture, PhD Thesis, Florida Tech. FAA. (2006), “Human space flight requirements for crew and space flight participants: Final rule”, Online athttp://edocket.access.gpo.gov/2006/pdf/E6- 21193.pdf.
Neuroergonomics of Cursor Control Devices in Spacecraft Cockpits 191
Fitts, P. M. (1954), “The information capacity of the human motor system in controlling the amplitude of movement”, Journal of Experimental Psychology 47(6), 381. Freeman, F. G., Mikulka, P. J., Prinzel, L. J. & Scerbo, M. W. (1999), “Evaluation
of an adaptive automation system using three EEG indices with a visual tracking task”, Biological Psychology 50(1), 61–76. Grammer, J. K., Xu, K. & Lenartowicz, A. (2021), “Effects of contexton the neural correlates of attention in a college classroom”, NPJ. Science of Learning 6(1), 1–4. Johnson, G. D., & Krusienski, D. J. (2018). “Computational EEGanalysis for brain-
computer interfaces.” Computational EEG Analysis: Methods and Applications, 193–214. Keller, B. M. (2007), “Cognitive avionics toolset for operator state classification based on physiological signals”, in 2007 IEEE/AIAA 26th Digital Avionics Systems Conference, IEEE, pp. 6–A. Krigolson, O. E., Williams, C. C., Norton, A., Hassall, C. D. & Colino, F. L.
(2017), “Choosing muse: Validation of a low-cost, portable EEG system for ERP research”, Frontiers in Neuroscience 11, 109. Momose, K., Weekes, T. R. & Eskridge, T. C. (2023), “Simulation Study of Human Input Devices in a Variable Gravity Environment for Commercial Space Transportation”, AHFE International (unpublished manuscript). Nacke, L. (2009), Affective ludology: Scientific measurement of user experience in
interactive entertainment, PhD Thesis, Blekinge Institute of Technology. Ota, T., Toyoshima, R. & Yamauchi, T. (1996), “Measurements bybiphasic changes of the alpha band amplitude as indicators of arousallevel”, International Journal of Psychophysiology 24(1-2), 25–37. Parasuraman, R. (2003), “Neuroergonomics: Research and practice”, Theoretical
Issues in Ergonomics Science 4(1-2), 5–20. Parasuraman, R. & Wilson, G. F. (2008), “Putting the brain to work: Neuroergonomics past, present, and future”, Human Factors 50(3), 468–474. Prinzel, L. J., Freeman, F. G., Scerbo, M. W., Mikulka, P. J. & Pope, A. T. (2000), “A
closed-loop system for examining psychophysiological measures for adaptive task allocation”, The International Journal of Aviation Psychology 10(4), 393–410. Reuderink, B., Mühl, C. & Poel, M. (2013), “Valence, arousal and dominance in the EEG during game play”, International Journal of Autonomous and Adaptive Communications Systems 6(1), 45–62. Stoet, G. (2010), PsyToolkit: A software package for programmingpsychological
experiments using Linux. Behavior research methods, 42, pp. 1096–1104. Stoet, G. (2017), “Psytoolkit: A novel web-based method for runningonline questionnaires and reaction-time experiments”, Teaching of Psychology 44(1), 24–31. Teplan, M., Krakovskà, A. & Špajdel, M. (2014), “Spectral EEG features of a short
psycho-physiological relaxation”, Measurement Science Review14(4), 237–242. Tomarken, A. J., Davidson, R. J. & Henriques, J. B. (1990), “Restingfrontal brain asymmetry predicts affective responses to films”, Journal of Personality and Social Psychology 59(4), 791. Weekes, T. R. (2021), Flow Choice Architecture: Cognitive Augmentation of Know-
ledge Workers, PhD Thesis, Florida Tech.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Neuroergonomics and Cognitive Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-78-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional