Satinder Singh Virdi, Yisi Liu, Daniel Zhang (2022). EEG-based stress recognition in competency assessment using maritime navigation simulator. In: Human Factors in Transportation. AHFE Open Access, vol. 60, pp. 675–683. https://doi.org/10.54941/ahfe1002504
Human factors receive increasing attention in maritime operations as it is found that human errors cause up to 90% of maritime incidents. One of the solutions to enhance safety in maritime navigation is to consider both technical and non-technical skills during training and assessment. Technical skills are always the first and foremost step in training or assessment. However, it may not be enough to solely conduct the navigation safely and efficiently by technical skills. Non-technical skills such as situational awareness, workload and stress management, and decision making also play a critical role in safe navigation and should not be overlooked. In addition, the seafarers spend a long time on board as part of their typical work pattern, and due to the current pandemic, they might be stranded at sea for even longer. It has been urged that ship owners and managers take the right actions to care for seafarers' mental well-being, such as stress and fatigue. To fulfil the needs of non-technical skill assessment and monitoring of mental well-being, electroencephalogram (EEG) is utilized to recognize the seafarers' mental stress when performing specific tasks in a full-mission Advanced Navigation Research Simulator (ANRS). The EEG-based recognition has unique advantages over the other biosensors. For example, it has high temporal resolution and reflects the mental states that are not identifiable from facial expressions with acceptable accuracy. A simulator allows the researchers to design and implement navigation scenarios with different weather conditions, traffic density, vessel types, emergency alarms, etc. This is ideal for conducting experiments for human factors study. An experiment has been designed and conducted in ANRS to validate the use of EEG for maritime navigational competency assessment. Participants with varied maritime backgrounds and roles were invited for data collection, e.g., navigating officers and experienced marine Pilots. Demanding events such as engine failure, close quarter situation, and other bridge equipment failure were marked during the EEG data recording. The results show that the EEG based stress recognition correlates with the demanding events in the experiment and can reflect the difficulty level of the challenging circumstances. The proposed EEG-based recognition can be used to study the human element in competency assessment using a maritime navigation simulator to ensure objective competency assessment.
Navigation safety is crucial as maritime accidents could cause loss of lives and economic benefits, pollution to the marine environment, etc. (Soares and Teixeira, 2001). Regulations have been made under different internatio- nal conventions, and advanced technologies have been developed to prevent maritime accidents. However, the accidents rate is still high (Baniela and Ríos, 2011), and most of them are caused by human errors (Baker and McCafferty, 2005). (Hasanspahi´c et al., 2021) has applied the Human Factor Analy- sis and Classification System for Maritime Accidents (HFACS-MA) method to analyze the historical marine accident reports from 1 January 2010 to 31 December 2019 and identifies that the most frequent accident causal factor was the condition of an operator(s). Some examples in the state of operator(s) category include fatigue, stress, and non-technical skills such as situation awareness, leading to low performance and unsafe situations (Roth- blum et al., 2002). It is also noticed that stress needs special attention as people under extremely high stress tend to have more human errors and are one reason for fatigue (Smith et al., 2007). To improve navigational safety, more attention should be paid to the condition of the navigator. For example, non-technical skills such as situational awareness, fatigue/stress management, and decision making are recommended during training and assessment.
Due to the nature of sea service, the seafarers spend a long time on board as part of their typical work pattern, and due to the current pandemic, they might be stranded at sea for even longer. It has been emphasized that ship owners and managers should take the right actions to care for seafarers’ men- tal well-being, such as stress and fatigue (MPA Singapore). One of the ways to monitor mental states is through biosignals. It has been proven that bio- signals such as heart rate (Özsever and Tavacio˘glu), electroencephalogram (EEG) (Lim et al., 2018) could be used to monitor mental workload and stress.
In our work, EEG is used to measure the stress of the navigator and marine Pilots as it has high temporal resolution and reflects the mental states more intuitively – directly from a human brain. An experiment was designed and carried out in a full-mission Advanced Navigation Research Simulator (ANRS) for EEG data collection. A navigation simulator allows the researchers/trainers to replicate scenarios similar to the one in real life with different factors, e.g., adverse weather conditions, traffic density, and emergency conditions. From the analysis results, we see great potential to use EEG to monitor the stress level of the navigators in competency assessment.
The paper is organized as follows. In the related work section, stress and maritime navigation safety and stress measurement at the workplace have been reviewed. The methodology section introduced the algorithm used for the EEG-based stress recognition. The experiment section shows the details of the experiment designed for data collection, including participants, device, scenarios, and experiment procedure. The results section presents the pre- liminary analysis using data from two pilots, followed by the conclusion section.
EEG-Based Stress Recognition in Competency Assessment 677
RELATED WORK
Stress and Maritime Navigational Safety Extreme stress at the workplace could damage employees’ well-being and decrease their performance (Mozgovoy, 2019). From the Yerkes-Dodson human performance and stress curve (Yerkes and Dodson, 1908), it shows that too little stress leads to inactive or laid back, too much stress leads to fatigue and exhaustion, extreme stress causes anxiety, panic or breakdown, while adequate stress boosts the performance which is considered as opti- mum stress. Thus, it is important to monitor stress and assess the ability to handle stress to improve maritime navigational safety.
To induce stress, stressors could be both mental (Lazarus and Folkman, 1984) and physical in personal and professional life (Anderson et al., 2002). Seafarers are usually exposed to more stressors when they are on board and reported to have a higher level of stress compared to normative groups (Parker et al., 1997). A literature review has been done in (Carotenuto et al., 2012) and shows that the causes of stress for seafarers include separation from family, loneliness on board, fatigue, multi-nationality, limited recrea- tion activity, sleep deprivation, etc. A survey from the Australian Maritime Safety Authority (AMSA) also shows that there could be differences in stress experienced by seafarers with varied occupational categories (Parker et al., 1997). For example, masters/mates were found to have higher incidences of both occasional to frequent and moderate to high-level stress than crew mem- bers. The raised stress level not only harms the well-being of seafarers but also leads to fatigue, loss of attention, low productivity and eventually accidents (Alderton et al., 2004). Other than stress during regular routine navigation, there could be “crisis routine navigation” encountered by seafarers, which is the period just before the accidents (Dominguez-Péry et al., 2021). Under such a situation, the stress level of seafarers is most likely to increase, and their performance could be affected (Sheridan, 2008). In summary, stress is one of the important factors that affect maritime navigational safety.
Stress Measurement at Workplace Stress can be measured by questionnaires on physiological biomarkers (Lupien and Seguin, 2013). For example, a N.U.T.S. questionnaire has been developed by (Centre for Studies on Human Stress) to measure stress. Using blood, urine and saliva, the hypothalamic-pituitary-adrenal (HPA) activation can be measured, which is an indicator of stress level. An interesting finding from (Stetler and Guinn, 2020) also shows that Cortisol levels from hair can reflect the stress experienced. Other biomarkers for stress measurement from the literature include heart rate (Hjortskov et al., 2004) (Mozgovoy, 2019), blood pressure (Hjortskov et al., 2004), Galvanic Skin Response (Mozgo- voy, 2019), vagal tone (Porges, 1995), Salivary alpha-amylase (Nater et al., 2005), and Motion Activity (Mozgovoy, 2019). However, not all the mea- surements are suitable for the workplace. The ideal measurement should be able to monitor the stress level of the employees accurately and timely without interrupting their primary tasks.
Table 1. Stress recognition based on emotion and workload (adapted from (Lim et al., 2018)).
Table available in the original PDF paper.
Emotion Level Workload Level Stress Level State Positive Relaxed 0 Low Neutral Relaxed 0 Low Negative Relaxed 1 Medium Low Positive Minimal 2 Moderate Low Neutral Minimal 2 Moderate Low Negative Minimal 3 Medium Positive Moderate 4 Medium High Neutral Moderate 4 Medium High Negative Moderate 5 Moderate High Positive High 6 High Neutral High 6 High Negative High 7 Very High
In recent years, Electroencephalogram (EEG)-based mental states recogni- tion has gained increasing attention due to the availability of easy-to-set-up EEG devices for healthy users. The EEG-based recognition has some unique advantages such as high temporal resolution, a reflection of inner and true feelings. Several studies have been published to use EEG to identify the emo- tion (Liu et al., 2011), workload (Lim et al., 2015), stress (Hou et al., 2015), and fatigue (Liu et al., 2020).
In this paper, we applied the EEG-based stress recognition to identify the stress level of the navigators and marine Pilots and analyzed their responses to demanding events in the simulated scenarios.
METHODOLOGY
The algorithm for stress recognition using EEG signals in (Lim et al., 2018) is adopted in our work, where stress is obtained using the combination of emotion and workload. The idea of combining emotion and workload to get stress is illustrated in Table 1. In this algorithm, Higuchi Fractal Dimension (Higuchi, 1988) and six statistical features (Picard et al., 2001) are extracted from the raw EEG signals using a sliding window of 4 seconds which moves by 1 second each time. Support Vector Machine (SVM) classifier is used in the training and classification steps (Liu and Sourina, 2014).
EXPERIMENT An experiment has been designed, and the EEG data collection is still in progress. A full-mission ANRS is used in the experiment to simulate the navigation scenarios.
Participants Currently, 13 participants have participated in the experiment, 8 of them are navigators with at least one year of independent navigational watchkeeping
EEG-Based Stress Recognition in Competency Assessment 679
experience, and the other 5 are marine Pilots with more than ten years of Pilotage experience.
Device EMOTIV EPOC X (Emotiv) is used to capture the EEG signals. The device comes with 14 channels sensing the whole brain and wireless connectivity. The sampling rate of the EMOTIV device is 128 Hz.
Scenarios For navigators, the scenario is based on navigation in the busy traffic separa- tion scheme (TSS) in Singapore Straits and manoeuvring the ship to pick up Pilot at an assigned Pilot Boarding Ground (PBG).
For marine Pilots, the Pilot navigates the ship from PBG to an assigned anchorage in the first scenario and to the berth for the second scenario, inside the Singapore Port.
Experiment Procedure The participants are briefed about the scenario, and the goal of the expe- riment. The consent forms are signed for their willingness to participate. They are given time to familiarize themselves with bridge equipment and to manoeuvre the vessel in the simulator. For every instance of data collection in ANRS, there are two participants. One of them is Officer of Watch (OOW) or Pilot, and the other acts as Helmsman. The EMOTIV device was mounted at the participants who acted as Pilot or OOW after the simulator familiariza- tion. Two calibration sessions were carried out for the participants with the EEG device. In the first calibration session, two sets of sound clips selected from the IADS database (Stevenson and James, 2008) were used to elicit posi- tive and negative emotions. In the second calibration, three levels of Stroop color test (Golden and Freshwater, 1978) were used to induce three levels of workload, from low to high. In this Stroop color test, the participants need to react to the color of the word and ignore the meaning of the word seen from the screen by pressing the corresponding numerical key. In the first level, the meaning and color of the word are congruent; in the second level, the mea- ning is different from the color of the word; in the third level, in addition to the incongruence, the participants need to react within 1 second; otherwise, it is considered as an incorrect reaction. Besides these two calibration sessions, 1 minute of eyes-open and eyes-closed EEG were also recorded, and the eyes- open EEG was labeled as neutral emotion and relaxed state for the mental workload. As a result, there are three sets of calibration data collected for emotion recognition, namely positive, neutral, and negative; there are four sets of calibration data collected for workload recognition, namely relaxed, minimal, moderate, and high, as shown in Table 1. The calibration data were used to train the SVM classifiers as described in the Methodology section. Once the calibration data had been collected, the navigation exercises were started. During the exercises, demanding events such as engine failure, close quarter situation, and another bridge equipment failure was introduced, and
Table 2. Average stress experienced by two pilot participants.
Table available in the original PDF paper.
Participant ID Scenario Average Stress
1 1 4.37 2 4.17 2 1 2.72 2 2.47
Table 3. Average stress for demanding events experienced by two pilot participants.
Table available in the original PDF paper.
Participant ID Scenario Demanding Event ID Average Stress
such events were marked in the EEG data collection to analyze the changes in stress during such events.
RESULTS
In this section, the preliminary results from two Pilots are presented. Both Pilots have been through scenarios 1 and 2 for the Pilot group. The average stress over the entire exercise for scenarios 1 and 2 are shown in Table 2, from which we can see that Participants 1 experienced a higher stress level than Participant 2 for both scenarios, which could be due to management of stress based on their work experience and ability to cope with unfamiliar situations. It is also interesting to see both participants experienced lower stress levels in the second scenario. Scenario 2 is mostly in an area with lesser traffic density and in protected waters, and the noted stress level is in alignment with expected outcomes.
The stress level for demanding events such as close quarter situation while entering anchorage area, failure of main engine or some bridge equipment during the exercises have been analyzed as well, and the results are shown in Table 3. Participant 1 was labeled with five demanding events in scenario 1 and one demanding event in scenario 2, while participant 2 was labeled with only one demanding event in scenario 1 and two demanding events in sce- nario 2. We can see that participant 1 had increasing stress from demanding events 1 to 4, and it lowered a bit for the last demanding event in scenario 1.
The bar chart illustrating the distribution of different stress levels experi- enced by the participants are plotted in Figure 1 and 2, respectively. In these
EEG-Based Stress Recognition in Competency Assessment 681
Figure 1: Distribution of different stress levels experienced by participant 1 (a) in scenario 1 (b) in scenario 2.Figure 2: Distribution of different stress levels experienced by participant 2 (a) in scenario 1 (b) in scenario 2.
figures, the X-axis shows the stress levels ranging from 0-7, corresponding to Table 1. Some levels are missing in the figures due to there being no such level experienced by the participant, e.g., in Figure 1 (b), there is no stress level 7 for participant 1 in the second scenario. The Y-axis is the percentage of the duration for a certain stress level over the entire exercise, e.g., in Figure 1 (a), participant 1 experienced stress level 8 for more than 50% time of the entire exercise. From Figure 1, we can see that participant 1 had a more rela- xed state and low stress states in scenario 2 compared to scenario 1. From Figure 2, we can see participant 2 had an almost similar pattern for stress in both scenarios, which is also consistent with the very close average stress levels for scenarios 1 and 2 shown in Table 2.
CONCLUSION
In this paper, we presented the work using EEG to assess the stress of the marine Pilots in a full mission ANRS. The preliminary analysis results show that the EEG-based stress recognition can correlate with the demanding events in the experiment and can reflect the difficulty level of the demanding
events. It also shows the varying ability of the Pilots to cope with unexpe- cted events. It also shows that Pilots with more work experience tend to have lower stress for the same scenario, and each individual reaction to the deman- ding events shows a variable response. This EEG-based stress recognition has the potential to be used in navigational competency assessment to evaluate the stress management capability of the navigators in an objective manner. In the next step, we will analyze the data from more participants and inte- grate the stress recognition to propose a navigational competency assessment framework.
REFERENCES
Alderton, T., Bloor, M., Kahveci, E., Lane, T., Sampson, H., Zhao, M., Thomas, M., Winchester, N. & Wu, B. (2004). The global seafarer: Living and working conditions in a globalized industry, International Labour Organization. Anderson, G. S., Litzenberger, R. & Plecas, D. (2002). Physical evidence of police offi-
cer stress. Policing: an international journal of police strategies & management. Baker, C. & McCafferty, D. Accident database review of human element concerns: What do the results mean for classification. Proc. Int Conf. ‘Human Factors in Ship Design and Operation, RINA Feb 2005. Citeseer. Baniela, S. I. & Ríos, J. V. (2011). Maritime safety standards and the seriousness of
shipping accidents. The Journal of Navigation, 64, 495–520. Carotenuto, A., Molino, I., Fasanaro, A. M. & Amenta, F. (2012). Psychological stress in seafarers: a review. International maritime health, 63, 188–194. Centre for Studies on Human Stress. N.U.T.S. questionnaire [Online]. Available: ht
tps://humanstress.ca/stress/nuts-questionnaire-2020/. Dominguez-Péry, C., Vuddaraju, L. N. R., Corbett-Etchevers, I. & Tassabehji, R.
(2021). Reducing maritime accidents in ships by tackling human error: a biblio- metric review and research agenda. Journal of Shipping and Trade, 6, 1–32. Emotiv. EMOTIV EPOC X [Online]. Available: https://www.emotiv.com/epoc-x/ . Golden, C. J. & Freshwater, S. M. (1978). Stroop color and word test. Hasanspahi´c, N., Vujiˇci´c, S., Franˇci´c, V. & ˇCampara, L. (2021). The Role of the Human Factor in Marine Accidents. Journal of Marine Science and Engineering, 9, 261. Higuchi, T. (1988). Approach to an irregular time series on the basis of the fractal
theory. Physica D: Nonlinear Phenomena, 31, 277–283. Hjortskov, N., Rissén, D., Blangsted, A. K., Fallentin, N., Lundberg, U. & Søgaard, K.
(2004). The effect of mental stress on heart rate variability and blood pressure during computer work. European journal of applied physiology, 92, 84–89. Hou, X., Liu, Y., Sourina, O., Tan, Y. R. E., Wang, L. & Mueller-Wittig, W. EEG based stress monitoring. 2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015. IEEE, 3110–3115. Lazarus, R. S. & Folkman, S. (1984). Stress, appraisal, and coping, Springer
publishing company. Lim, W. L., Liu, Y., Subramaniam, S. C. H., Liew, S. H. P., Krishnan, G., Sourina, O., Konovessis, D., Ang, H. E. & Wang, L.(2018). EEG-based mental workload and stress monitoring of crew members in maritime virtual simulator. Transactions on Computational Science XXXII. Springer. Lim, W. L., Sourina, O., Liu, Y. & Wang, L. EEG-based mental workload recogni-
tion related to multitasking. 2015 10th International Conference on Information, Communications and Signal Processing (ICICS), 2015. IEEE, 1–4.
EEG-Based Stress Recognition in Competency Assessment 683
Liu, Y., Lan, Z., Cui, J., Sourina, O. & Müller-Wittig, W. (2020). Inter-subject transfer learning for EEG-based mental fatigue recognition. Advanced Engineering Informatics, 46, 101157. Liu, Y. & Sourina, O. (2014). Real-time subject-dependent EEG-based emotion
recognition algorithm. Transactions on Computational Science XXIII. Springer. Liu, Y., Sourina, O. & Nguyen, M. K. (2011). Real-time EEG-based emotion recognition and its applications. Transactions on computational science XII. Springer. Lupien, S. J. & Seguin, F. (2013). How to measure stress in humans. Centre for Studies
in Human Stress. Mozgovoy, V. Stress Pattern Recognition Through Wearable Biosensors in the Workplace: Experimental Longitudinal Study on the Role of Motion Intensity. 2019 6th Swiss Conference on Data Science (SDS), 2019. IEEE, 37–45. MPA Singapore. Seafarer Happiness Index (SHI) Report – Caring for Seafarers’ Men-
tal Well-Being [Online]. Available: https://www.mpa.gov.sg/assets/srs/e-bulletins/ Issue21/case-studies/seafarers.html. Nater, U. M., Rohleder, N., Gaab, J., Berger, S., Jud, A., Kirschbaum, C. & Ehlert, U.
2005. Human salivary alpha-amylase reactivity in a psychosocial stress paradigm. International Journal of Psychophysiology, 55, 333–342. Özsever, B. & Tavacio˘glu, L. Measuring mental workload and heart rate variability of officers during different navigation conditions. Marine Science and Technology Bulletin, 306–312. Parker, T. W., Hubinger, L. M., Green, S., Sargent, L. & Boyd, B. (1997). A survey of
the health stress and fatigue of Australian seafarers. Picard, R. W., Vyzas, E. & Healey, J. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE transactions on pattern analysis and machine intelligence, 23, 1175-1191. Porges, S. W. (1995). Cardiac vagal tone: a physiological index of stress. Neuroscie-
nce & Biobehavioral Reviews, 19, 225–233. Rothblum, A. M., Wheal, D., Withington, S., Shappell, S. A., Wiegmann, D. A., Boehm, W. & Chaderjian, M. (2002). Human factors in incident investigation and analysis. Coast guard research and development center groton ct. Sheridan, T. B. (2008). Risk, human error, and system resilience: fundamental ideas.
Human factors, 50, 418–426. Smith, A. P., Allen, P. H. & Wadsworth, E. J. A comparative approach to seafarers’ fatigue. Proceedings of the International Symposium on Maritime Safety, Science and Environmental Protection, Athens, (2007). Citeseer. Soares, C. G. & Teixeira, A. (2001). Risk assessment in maritime transportation.
Reliability Engineering & System Safety, 74, 299–309. Stetler, C. A. & Guinn, V. (2020). Cumulative cortisol exposure increases during the academic term: Links to performance-related and social-evaluative stressors. Psychoneuroendocrinology, 114, 104584. Stevenson, R. A. & James, T. W. (2008). Affective auditory stimuli: Characteriza-
tion of the International Affective Digitized Sounds (IADS) by discrete emotional categories. Behavior research methods, 40, 315–321. Yerkes, R. M. & Dodson, J. D. (1908). The relation of strength of stimulus to rapidity of habit-formation. Punishment: Issues and experiments, 27–41.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-36-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional