Alexander Almer, Anna Weber, Florian Haid, Julia Tschuden, Lucas Paletta, Michael Schneeberger, Dietmar Wallner, Paul Glanz, Philip Klöckl, Dominik Eder, Gerald Bauer, Oliver Kühr, Thomas Hölzl (2023). Physiological and cognitive real-time stress analysis as a basis for optimised human-machine teaming and safe decision processes for military forces. In: Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications. AHFE Open Access, vol. 112, pp. 387–396. https://doi.org/10.54941/ahfe1004151
Introduction In recent decades, the development of autonomous cyber-physical systems for a wide range of tasks has been the focus of research activities for military organisations. Modern security forces can be seen as socio-technical systems. Only an integrated approach, in which people, organisation and technology are viewed as interlocking elements, enables the optimisation of the overall system. Soldiers are still at the center of deployed sociotechnical systems despite major innovations in the field of autonomous systems and artificial intelligence (Swiss, 2020). An efficient and coordinated interaction in a task force and an optimised human-machine teaming are essential prerequisites for a successful operation and thus also for increasing the safety of the soldiers in critical operational situations. This requires, on the one hand, optimal HMI development, but also, on the other hand, information about the mental and physical state of the soldier to provide improved decision processes and operational performance. Information on a common operational picture and the status of the technical systems used is usually available, but not the psychophysical situation of the soldier. Therefore extensive development projects have been launched for solutions of psycho-physiological monitoring, with new possibilities arising from innovative developments in the field of bio-sensor technology. The aim is to optimise human performance in the field and the interaction between man and machine with highly sophisticated mission equipment. An important success factor in complex operations is the quality of the necessary decisions (decision intelligence) in time-critical security situations, whereby the current psychophysical stress state of the person is a decisive factor. Therefore, an ongoing challenge for the military task forces is managing personnel to optimise and sustain performance, improve security while also ensuring health and wellbeing. In the course of intensive training and exercises as well as in real operational scenarios, soldiers often suffer physiological and psychological borderline stresses and injuries during physical and combat-related training. In this context efficient solutions for the physiological monitoring of soldiers based on the integration of innovative biosensor technology and specific load models considering load characteristics of different military forces will enable a targeted support.Motivation and Background The challenging military work tasks are often associated with a high degree of physical stress and require a high level of mental performance and concentration. Reduced concentration and reaction cause delayed or possibly even wrong decisions, which can have critical consequences. In this context a real-time system for physiological status monitoring (RT-PSM) offers new opportunities for military purpose with individual assessment of soldiers' performance limits. However, most commercially available health and performance sport systems do not meet the relevant military requirements. They typically lack validated methods and algorithms to derive essential information in real time and are not designed to be integrated into soldier's technological ecology (Friedl, 2018). Based on the specific requirements and the experience of the Austrian Armed Forces, an RT-PSM was developed as part of the VitalMonitor project and geared to the working conditions and multifactorial stress situations of CBRN defence personnel and light infantry forces. The main objectives were to analyze the individual stress in deployment scenarios and to achieve a targeted improvement in the individual performance level through personalized adaptive training concepts and thus to optimize the health and fitness of the individual soldier. The research project VitalMonitor therefore focuses on the development of a (I) real-time monitoring system, which analyses changes in physiological parameters from heart rate, heart rate variability, skin conductance, core body temperature, etc., (II) development of a stress model considering load characteristics of different military forces, (III) communication solution for a real-time data transfer, (IV) data management and interactive real-time visualization module to support decision processes for mission commanders to determine optimal work-rest-cycles preventing physical overstraining in trainings and missions and (V) an expert interface to visualize sensor data streams (low-level data) together with model-based analysis results (high-level data) in a graphical interface as a basis for model development, verification and optimization.This paper gives an overview of the main developments and results implemented and achieved within the VitalMonitor project. In the following, wearable sensors and their evaluation, the development of a specific load model, the real-time visualization modules and finally a conclusion and outlook will be presented.
In recent decades, the development of autonomous cyber-physical systems for a wide range of tasks has been the focus of research activities for military organisations. Modern security forces can be seen as socio-technical systems. Only an integrated approach, in which people, organisation and technology are viewed as interlocking elements, enables the optimisation of the over- all system. Soldiers are still at the center of deployed sociotechnical systems despite major innovations in the field of autonomous systems and artificial intelligence (Swiss, 2020).
An efficient and coordinated interaction in a task force and an optimised human-machine teaming are essential prerequisites for a successful operation and thus also for increasing the safety of the soldiers in critical operational situations. This does not imply equally shared work tasks or responsibilities between humans and technical systems. The use of such systems as teammates in military and safety-critical contexts introduces important questions about human control and responsibility.
Optimised operation or teaming and cooperative interaction requires, on the one hand, optimal HMI development, but also, on the other hand, information about the mental and physical state of the soldier to provide improved decision processes and operational performance. Information on a common operational picture and the status of the technical systems used is usually available, but not the psychophysical situation of the soldier. There- fore extensive development projects have been launched for solutions of psycho-physiological monitoring, with new possibilities arising from inno- vative developments in the field of bio-sensor technology. The aim is to optimise human performance in the field and the interaction between man and machine with highly sophisticated mission equipment.
An important factor for success is the quality of the necessary decisions (decision intelligence) in time-critical security situations, whereby the current psychophysical stress state of the person is a decisive factor. Therefore, an ongoing challenge for the military task forces is managing personnel to opti- mise and sustain performance, improve security while also ensuring health and wellbeing. In the course of intensive training and exercises as well as in real operational scenarios, soldiers often suffer physiological and psycho- logical borderline stresses and injuries during physical and combat-related training, with overuse injuries often occurring here. In this context, efficient solutions for the physiological monitoring of soldiers based on the integra- tion of innovative biosensor technology and specific load models considering load characteristics of different military branches will enable a targeted sup- port for education and training processes, as well as for operations in the field.
MOTIVATION AND BACKGROUND The challenging military work tasks are often associated with a high degree of physical stress and require a high level of mental performance and concen- tration. Reduced concentration and reaction cause delayed or possibly even wrong decisions, which can have critical consequences. In this context, a real-time system for physiological status monitoring (RT-PSM) offers new opportunities for military purpose with individual assessment of soldiers’ performance limits instead of generalized parameters. However, most com- mercially available health and performance sport systems do not meet the relevant military requirements. They typically lack validated methods and algorithms to derive essential information in real time and are not designed to be integrated into soldier’s technological ecology (Friedl, 2018).
Based on the specific requirements and the experience of the Austrian Armed Forces, an RT-PSM was developed as part of the VitalMonitor project
Physiological and Cognitive Real-Time Stress Analysis 389
and geared to the working conditions and multifactorial stress situations of CBRN defense personnel and light infantry forces. The main objectives were to analyze the individual stress in training and exercise scenarios and to achieve a targeted improvement in the individual performance level through personalized adaptive training concepts and thus to optimize the health, fit- ness and resilience of the individual soldier. The research project VitalMonitor therefore focuses on the development of a (I) real-time monitoring system, which analyses changes in physiological parameters from heart rate, heart rate variability, skin conductance, core body temperature, etc., (II) develop- ment of a stress model considering load characteristics of different military branches, (III) communication solution for a real-time data transfer, (IV) data management and interactive real-time visualization module to support decision processes for mission commanders to determine optimal work-rest- cycles preventing physical overstraining in trainings and missions and (V) an expert interface to visualize sensor data streams (low-level data) together with model-based analysis results (high-level data) in a graphical interface as a commanders interface for decision making.
This paper gives an overview of the main developments and results imple- mented and achieved within the VitalMonitor project. In the following, wearable sensors and their evaluation, the development of a specific load model, the real-time visualization modules and finally a conclusion and outlook will be presented.
SENSOR EVALUATION As part of this project, a concept was developed to test the sensor tech- nology under various conditions for its validity and practical suitability for use in military operations. The concept envisaged an evaluation process in four phases, with the requirements for the sensor technology increasing from phase to phase. In the first two phases, the sensor technology was tested in the laboratory on moderately trained test persons. In the following phases, the evaluation was carried out on military personnel, first during an excercise under standardized conditions and finally in near-deployment exercise sce- narios in the field. The measurement data collected by the innovative sensor technology were compared with the respective “gold standards”. Descriptive statistics as well as interference statistical analyses were used to evaluate the data.
In the first study, a smart textile solution (QUS Smart Shirt by QUS Sports, Austria) was investigated. The aim of the work was to evaluate the measure- ment accuracy of the smart shirt for determining heart rate and respiratory rate under standard climatic conditions during an incremental exercise test on an ergometer. The results showed that for both heart rate and respiratory rate quality are predominantly influenced by the optimal fit of the smart textile. The heart rate measurement in particular provides valid data. In the second study, a temperature sensor system for non-invasive measurement of the core body temperature (CORE sensor by greenTEG AG, Switzerland) was inves- tigated. The aim of the work was to evaluate the accuracy of the determined core body temperature by the temperature sensor. The study was conducted
on an ergometer under standard climatic conditions. Results show statistical deviations from the gold standard, but the practical relevance of these devia- tions can be considered low, especially when the heart rate is integrated into the algorithm.
Figure 1: Setup for sensor evaluation in standardised and heat conditions. Gold standard reference sensors with 6-lead ECG and spirometry (rightmost).
In the second phase of the sensor evaluation tests, exercises were per- formed on the treadmill under hot and humid conditions (room temperature of 34–36
C; humidity of 60-80%). The aim of the work was to evaluate the measurement accuracy of the heart rate and respiratory rate determined by the smart shirt, as well as the measurement accuracy of the temperature sen- sor under hot and humid conditions and increased upper body movement. No heat or movement related decrease in measurement accuracy was observed for either the smart shirt or the temperature sensor technology. The results coincide with the results of the testing under standard climatic conditions.
In the third phase, sensor evaluations were conducted during an annual CBRN-defence training simulation with active soldiers. The participants wore the sensor equipment underneath the CBRN protective clothing. The focus was no longer primarily on assessing the data quality and its rea- sonableness, but rather on reviewing the practical suitability and technical functionality during military-specific operations. To validate the sensor data a data verification and validation pipeline was developed (Wendelken, 2003; Jennings, 2007; Godehardt, 2018). This pipeline includes checking the sig- nal quality, looking into missing data, adherence to reasonable boundaries and adherence to the expected behaviour under various external stressors, such as physiological strain or recovery. The results show that a properly fitted smart shirt and a sufficiently long warm-up phase are essential for good conductivity of the ECG electrodes that are integrated into the smart shirt. Unfortunately, some participants experienced a partial loss of data or overall noisy ECG signals resulting in incorrect HR and HRV readings. The temperature sensor also benefits from a prolonged warm-up phase for best accuracy and signal quality. The recorded and averaged measurement values were reasonable (Table 1). The real time measurements, however, showed a slight delayed raise in core body temperature compared to the ref- erence values (BodyCAP ingestible capsule) depending on the chosen CBT algorithm.
Physiological and Cognitive Real-Time Stress Analysis 391
Table 1. Non-invasive CBT measurements with greenTEG sensor compared to Body- Cap ingestible capsule (reference).
In the last and fourth phase, the sensor setup was tested during a large international CBRN live exercise in Suffield, Alberta, Canada. The exer- cise was organised as a live-agent-training where CBRN reconnaissance and decontamination capabilities were consolidated and expanded, and new developments were tested in a multi-national environment. Practical suit- ability and technical functionality of the VitalMonitor sensor setup was tested under non-standardized conditions in a hot environment. We conclude that the functionality of the system was demonstrated successfully. A more detailed analysis of the sensor data quality is part of future work.
USABILITY EVALUATION To provide a usable and comfortable to wear sensor equipped smart shirt solution two different groups of soldiers (infantrymen, CBRN-defense) were tested in specific scenarios. The acquired information enabled further devel- opment regarding the usability and comfort of the QUS smart shirt solution in the military.
Figure 2: Smart shirt usability test performed by soldiers of the CBRN-defense (left) and infantrymen (right).
The participants completed a branch specific scenario, which was devised by infantry respectively CBRN-defense subject matter experts. Infantrymen marched for 4–6 kilometers at 4 kmh-1 with full personal and platoon gear (mean additional load 54.9 ± 7.6kg = 70 ± 14% bodyweight). Subsequently they approached the target area and attacked an urban area at 0.5 kmh-1 for 2 kilometers (mean additional load 43.5 ± 3.7kg = 55 ± 9% bodyweight respectively). CBRN-defense soldiers performed different CBRN-defense spe- cific tasks for a complete duration of sixty minutes wearing full CBRN
personal protective equipment and necessary kit (mean additional load: 28.6 ± 1.7kg = 35.0 ± 4.8%). After completing the branch specific scenario, the participants completed a questionnaire developed via Austrian Armed Forces Sports Centre. It con- tained eleven questions (e.g. wear comfort, increased sweating) which could be answered using a continuous scale from 0–10 (zero = strongly disagree to ten = strongly agree). Participants were also enabled to give written state- ment on chosen questions. The aim of this questionnaire was to find out, how soldiers evaluate the usability and comfort of the QUS smart shirt solu- tion under military-specific conditions. Means and standard deviation were obtained for age, height, weight and questionnaire answers. Data analysis was conducted using Microsoft® Excel® 2016. To compare data, Student’s unpaired t-test (IBM® SPSS® 28) was performed.
Personal information was obtained at the beginning of each testing. These data are presented as means and standard deviations in Table 2.
Table 2. Sample characteristics based on number of participants (n), age, height and weight.
Figure 3: Questionnaire answers as mean for soldiers of the CBRN-defense, infantry and combined.
The selected data presented in Figure 3 and Table 3 showed that infantry- men and CBRN-defense personnel were basically content with the donned QUS smart shirt solution. Three areas were identified as showing a ten- dency towards being problematic albeit the means of the answers were below median: poor wear comfort (mean 2.7 ± 2.8), problems with additional
Physiological and Cognitive Real-Time Stress Analysis 393
(mean 2.5 ± 3.6) gear and increased sweating (mean 4.8 ± 3.7). There were isolated written remarks about skin irritation because of the sensors in the smart shirt, increased pressure at the back of the neck because of the on board unit and increased sweating.
Table 3. Mean values for selected questionnaire answers divided by infantry, CBRNdefense and combined.
0,6 ± 1.6 Values expressed as mean ± standard deviation; scale 0–10 0 = strongly agree; 10 = strongly disagree; * measured Infantry vs. CBRN: Student’s unpaired t-test t = −3.389 (p <0.001) and Welch test t = −2.694 (p = 0.010)
In summary, the QUS smart shirt solution was deemed usable and comfort- able to wear by the participants. To increase comfort and decrease sweating, a reduced kind of QUS smart solution was used for the CBRN-defense testing. This reduced version of the shirt resembles a woman’s bra. CBRN-defense personnel still mentioned increased sweating as the main issue. Further improvements to tackle these issues are planned.
LOAD MODEL DEVELOPMENT A major goal of the VitalMonitor project was to develop a load model to assess physical stress in military training and exercise scenarios in an objec- tive way, not only based on internal physiological factors but also based on external factors. Internal factors are heart rate (HR), respiration rate (RR), core-body-temperature (CBT) whereas external factors are activity, move- ment speed, external load, ambient temperature, etc., both influencing each other. The model determines and assesses the individual physical reaction of the soldier to these internal and external stresses.
First, the focus was on defining which parameters are best suited for sur- veying the individual physical state of the soldiers for the corresponding military branches CBRN-defense, light infantry and explosive ordnance dis- posal personnel. In the course of the initial model developments, thresholds and zones were defined for each parameter based on well-known literature (Wonisch & Ledl-Kurkowski, 2017, Godehardt, 2018, Hunt et al. 2016) and knowledge from military experts. Each parameter included in the stress model was categorized into six different zones and additionally combined using a specific logic. The first model was refined so that a weighting of the individual parameters can take place depending on the respective activities. In the case of strenuous activities with higher physiological stress, more weight can be assigned to the associated parameters.
Figure 4: Load model with weighted parameters and the resulting overall score.
After several test runs in different training and exercise scenarios and the subsequent evaluation of the collected data, certain parameters turned out to be very suitable, others partly unsuitable, in order to survey the individual stress of the soldiers and to adapt the stress model individually to the different requirements. For the CBRN e.g. walking speed was negligible whereas for the light infantry it turned out to be a significant parameter. Therefore a new index “Load-Speed-Index (LSI)” based on walking speed and payload was implemented. The LSI can be used as a predictor on how long the soldier can endure this particular situation with a constant load (Drain et al., 2016). Also parameter thresholds were adapted (CBT, RR) based on these tests and empirical knowledge from military experts.
The final model for the determination of psychophysical stress includes the parameters body-core temperature (CBT), heart rate (HR), respiratory rate (RR) and the load-speed-index (LSI). The measured values are weighted to each other and combined using a predefined logic, making it adaptable to different military activities. An overall score can be determined, allowing for a quick assessment on the soldiers’ state, which is also highlighted in a cer- tain colour scheme in an interactive visualization described in the following chapter.
INTERACTIVE REAL-TIME-VISUALIZATION An important factor for success is the decision quality in time-critical secu- rity situations, taking the current psychophysical stress state of the person into account. In VitalMonitor, a RT-PSM to support time-critical decisions was developed together with domain experts of the Austrian Armed Forces. It provides real-time monitoring the physiological status of the soldiers, implementing the described individual load model. It provides multiple func- tionalities in different tabs (Figure 5). First, an overview offers a view over the physiological status of all soldiers in the field, focusing on four chosen vital parameters. Additionally the previously introduced load model provides for
Physiological and Cognitive Real-Time Stress Analysis 395
a quick assessment, by visualizing the overall score according to a six-stage traffic light system depicted as smiley faces as well as a history graph to track the individual stress over time.
Second, for a more detailed analysis the figure on the right (Figure 5) shows the vital signs in more detail. Next to the four live vital signs, also coloured in a traffic light system style, a line graph displays the trend of every single vital parameter. Below, the history of the traffic light states of each individual parameter, the LSI and the overall score of the load model are shown as a timeline. This allows to quickly identify the direction in which the physiolog- ical status is likely to develop further on a parameter as well as overall score level.
Figure 5: Real-time visualisation for physiological status monitoring (RT-PSM).
Third, there is also a map view to follow the soldier’s position in real time and a view to analyse the entire soldier’s historical data offline.
CONCLUSION AND OUTLOOK The military monitors the combat readiness and functionality of its vehi- cles, aircraft and other machinery, but rarely the operational readiness of its human resources in the field or during military training. Ignoring their impor- tance can lead to a decline in the physical and mental performance of soldiers, which can have a negative impact on the outcome of the military activity. Efficient and coordinated interaction in a task force and optimised human- machine teaming are essential prerequisites for successful operations and consequently for increasing the safety of soldiers in critical operational situ- ations. Teaming and cooperative interaction require not only optimal HMI development, but also information on the mental and physical state of the soldier in order to improve decision-making and operational performance. Therefore, VitalMonitor proposed the development of psycho-physiological monitoring with innovative developments in the field of biosensor technol- ogy with the aim of optimising human performance with highly sophisticated mission equipment.
The project has provided interesting insights into the military-specific requirements of an RT-PSM and has opened up a number of topics for further research. Research focuses for future work will include the development of
sensor specific quality parameters, as well as the possibility of adapting the load model parameters and their weighting in relation to each other depend- ing on the military branch or scenario of use. The addition of a predictive load model to optimise work-rest cycles will also be part of future work. This will require a larger dataset, also including a greater number of female participants to integrate gender differences into the load model.
ACKNOWLEDGMENT
This work was funded from the Austrian Defense Research Programme FORTE, the Austrian Research Promotion Agency (FFG) and the Austrian Federal Ministry of Finance.
REFERENCES
Drain, J., Billing, D., Neesham-Smith, D., Aisbett, B. Predicting physiological capacity of human load carriage – A review, Applied Ergonomics, Volume 52, 2016, Pages 85–94, https://doi.org/10.1016/j.apergo.2015.07.003. Friedl, K. E. (2018) Military Applications of Soldier Physiological Monitoring.
Journal of Science and Medicine in Sport 2018; 21: 1147–53. Godehardt, K. (2018). Referenzwerte gesunder Probanden für ventilatorische Parameter und das Atemäquivalent in Ruhe und unter Belastung auf dem Laufbander- gometer im Alter von 4 bis 74 Jahren (Doctoral dissertation, Ruhr-Universität Bochum). https://doi.org/10.1093/milmed/usy218. Hunt, A. P., Billing, D. C., Patterson, M. J. & Caldwell, J. N. (2016). Heat strain
during military training activities: The dilemma of balancing force protection and operational capability. Temperature, 3(2), 307–317, https://doi.org/10.1080/ 23328940.2016.1156801 Jennings, J. Heart rate*. In Encyclopedia of Stress (Second Edition), pages 274–277.
Academic Press, 2007. ISBN 978-0-12-373947-6. Swiss (2020): Langfristiger Forschungsplan (LFP) 2021-2024; Forschungskonzept armasuisse mit Forschungsschwerpunkten und prioritären Themen; Copyright armasuisse Wissenschaft und Technologie (W+T) Forschungsmanagement und Operations Research Feuerwerkerstrasse 39; CH-3602 Thun; Thun, 11. January 2020. Wendelken, S., McGrath, S. and G. Blike. A medical assessment algorithm for
automated remote triage. In Proceedings of the 25th Annual International Con- ference of the IEEE Engineering in Medicine and Biology Society, volume 4, pages 3630–3633 Vol.4, 2003. doi: 10.1109/IEMBS.2003.1280940. Wonisch, M., & Ledl-Kurkowski, R. P. (2017). Kompendium der Sportmedizin (Vol. 2). Wien: Springer.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Systems Engineering and Design (IHSED 2023): Future Trends and Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-88-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional