Lenart Motnikar, Zoja Anžur, Peter Fröhlich, Alexander Mirnig, Manfred Tscheligi (2023). Beyond Detection: Intervention Approaches in Driver State Monitoring Systems. In: Human Factors in Transportation. AHFE Open Access, vol. 95, pp. 437–447. https://doi.org/10.54941/ahfe1003829
Driver state monitoring systems (DSMS) – technologies that detect driver impairment and provide interventions – are becoming crucial components of driving safety technology. However, despite plentiful research on impairment detection, literature on corresponding interventions is limited. To gain insight into technology-based risk prevention and mitigation approaches to driver impairment, we review the state of the art in solutions targeting stress, fatigue, and cognitive load. We systematically analyze the latest literature published in two technology databases between 2017 and 2022. We first provide a general summary of reviewed studies, quantifying trends in technical, HCI, and methodological characteristics and additionally focus on user-evaluated solutions. Overall, we observe a general focus on fatigue and a prevalence of simple binary alerts. We report that only a minority of solutions are user-evaluated, although these exhibit greater diversity both in terms of the impairments they target and the methods they employ. Our findings show that the field offers extensive possibilities, although the proposed solutions are mainly in early developmental stages. Ultimately, we evaluate possible intervention approaches, identify the gaps, and provide guidelines to support future research.
Driver inattention – be it due to stress, fatigue, or cognitive load – has long been recognized as one of the highest risk factors for traffic safety. To respond to these risks, driver state monitoring systems (DSMS) have been developed over the recent decades and are expected to become crucial automotive safety components as the industry moves towards automated driving (Fredriksson et al. 2021).
DSMS are systems that (i) detect impaired and distracted driving and (ii) provide appropriate warning or action. As such, they take inputs from the driver’s physiology, behavior, or performance, to then offer risk mitigation measures. While DSMS can take the form of standalone systems, additi- onal safety benefits are expected from integrating the DSMS into existing
advanced driver-assistance systems (ADAS) so that they can adapt their inter- faces in a way that is sensitive to the driver’s physiological or mental state (Fredriksson et al. 2021). Furthermore, ADAS can take over certain functi- ons or assist the driver in performing operative actions, such as braking or corrective maneuvering.
Globally, regulations that mandate integration of DSMS for impairments like drowsiness and distractions are coming into effect (e.g., EU’s Regulation 2019/2144 or China’s Standard GB/T 39263-2020). However, commercial solutions mostly take the form of alerts, which have been shown to pro- vide some benefits in mitigating risk, though critics have noted that alerting systems alone may be limited without additional intervention strategies (Fitzharris et al., 2017).
When choosing optimal intervention strategies, it is difficult to obtain a comprehensive overview of best practices and possibilities. As the DSMS field is broad and interdisciplinary, the insights are scattered across nume- rous publications and fields. To our knowledge, a review does not yet appear to exist that systematically focuses on intervention approaches in DSMS and related fields, and summarizes the knowledge, best practices, and gaps.
The existing review papers surveying DSMS predominantly focus on dete- ction methods and are often limited to specific impairments, data sources, or computational approaches (Hecht et al. 2018, Kashevnik et al., 2021; Ramzan et al. 2019; Rastgoo et al. 2019). Among those that consider inte- rventions that accompany detection, Aghaei et al. (2016), for example, review design implications, although their contribution is detection-oriented and does not cover presentation or comparison of specific systems. There are some reviews that survey the design aspects of existing alerting systems, e.g., in terms of sensory modalities (Zuki and Sulaiman, 2016) or their use in commercial systems (Ahir and Gohokar, 2019), but these do not take systematic approaches.
On the side of more general overviews, Victor (2011) provides a conceptual framework for technology-based countermeasures to inattention. Similar reviews of countermeasures also exist for fatigue (Nazari et al. 2017) and distraction (Arnold et al. 2019) but do not focus on technological solutions per se and include behavioral techniques as well as educational and envi- ronmental interventions. Some recent studies also recognize the need for a comprehensive perspective on monitoring impaired driving (Brijs et al. 2020) but do not describe concrete solutions and mainly focus on countermeasures on an institutional and enforcement level, reaching beyond the scope of what can be implemented within DSMS.
Reviewing the existing literature, we observe a variety of publications on driver impairments and interventions clustered within specific areas, with a lack of covering approaches in responding to impairment.
To fill the identified gaps, we conduct a literature review that focuses on: (i) studies in the fields related to DSMS that include responses to impair- ment and not only detecting it.
Beyond Detection: Intervention Approaches in Driver State Monitoring Systems 439
(ii) concrete technical implementations of interventions instead of general countermeasure strategies.
We set out to survey recent academic literature in the HCI community to uncover what type of approaches are considered to mitigate driving impair- ments and assess their level of development. We are also interested in the prevalent trends in DSMS research, like which impairments they most often target and what type of methodologies and interfaces they employ. As we expected the field to primarily focus on impairment detection, we particu- larly focus on solutions that included some type of user evaluation. Finally, we use our review not only to describe the state of the art in academic research but also to inform future DSMS development.
METHODOLOGY
We set the scope of our review to three general categories of driving impair- ment – fatigue, stress, and cognitive load. This classification was informed by previous work on driver state identification (e.g., Barua 2019, Hecht et al. 2018), however, we should note that these concepts include various subcate- gories and often overlap with one another. Hence, we expected the ambiguity to be reflected in the multidisciplinary DSMS literature.
To account for this, we deliberately used broad conceptualizations to capture a wide array of potentially relevant solutions. We primarily jud- ged the categories as they were referred to by the authors of revie- wed studies, regardless of whether they fit a specific theoretical notion. With this in mind, we provide a generalized operationalization of the impairments:
Fatigue: inhibited cognitive performance characterized by low vigilance and tiredness. This does not entail distinctions based on causes (e.g., lack of stimulation or sleep), meaning that fatigue is grouped together with notions like sleepiness and drowsiness.
Stress: psychological and physiological overarousal, characterized by pressure and strain. This comprises various manifestations or causes (e.g., environmental, emotional, etc.) and includes any type of overarou- sal that could potentially worsen driving performance and be addressed by DSMS.
Cognitive (over)load: worsened ability to process (driving-related) infor- mation due to competing cognitive activity. In that sense, we understand the term as also including internal and external distraction.
Procedure Our approach is adapted from Okoli & Shabram’s (2010) guidelines for reviews of information systems research and is summarized in Figure 1 below.
The literature was searched with a query of keywords related to driving, impairment, and response, screened for relevance by two reviewers, and coded according to a predefined scheme (Table 1).
Figure 1: Schema of the review protocol.
OVERVIEW In this section, we quantify our coding results (Table 2) and provide a general overview of all papers that passed the screening stage as well as the sub- set of those that included user evaluation of the proposed solution. We then outline the most notable trends and comment on their significance for the field.
Dominance of Fatigue We first observe that most studies focus on fatigue. This is an expected trend, as, of the overviewed impairments, fatigue likely presents the greatest risk to driving safety. It also reflects recent developments in regulation that mandate DSM for fatigue, which additionally incentivize research in the field. Another likely reason is a pragmatical one, as most fatigue detection methods are realized as camera-based setups, which are relatively cost-effective and easy to implement.
That said, fatigue does not cover the entire spectrum of safety-critical driving impairments and although a slight dominance of fatigue might be justified, it points to a relative neglect of other impairment types.
Alerts as a Means to an End Overwhelmingly, the proposed solution to impairment is a simple alert tar- geting the driver. To some extent, this is reasonable, as most studies focused on impairment detection, only implementing alerts as an additional compo- nent. However, this shows an underlining belief that alerting the driver in any way is a sufficient intervention to impairment. As this is not entirely supported by previous research (Fitzharris et al. 2017), we argue that more attention should be given to design aspects of alerting systems as well as other intervention methods.
Beyond Detection: Intervention Approaches in Driver State Monitoring Systems 441
Vehicular sensor (steering wheel, lane position, speed, etc.)
Impairment detection input Observation (human or camera, including automated detection) No detection (e.g., experimental manipulation of impairment)
Was the system implemented and evaluated in a driving context? (e.g., integrated in a vehicle) – Yes / No
Driving context Simple alert (binary response to impairment detection)
Complex alert (phased, adaptive, or personalized alert response)
Type of alert or intervention Feedback (continuous or summarized feedback of driver state) Recommendation (passive recommendation of action to be taken)
Assistance (active assistance with impairment, driving, or task) Interference (e.g., autonomous takeover, engine shutdown)
Concept (description or blueprint without physical implementation) Technical maturity Low fidelity (simple implementation using beeps, blinks, or raw text)
High fidelity (implementation with high degree of integration or high technical maturity) Did the paper report any type of user evaluation of the intervention? – Yes / No (condition for further two categories) User evaluation
Simulator (any low- to high- fidelity driving simulation) Course (controlled road environment) Testing environ- ment
Road (naturalistic driving in real traffic) Objective quantitative (driving performance, physiological data, etc.)
Evaluation type Subjective quantitative (questionnaires, scales, etc.)
Qualitative (interviews, focus groups, etc.)
Table 2. Quantified overview of DSMS – results of the coding stage.
Table available in the original PDF paper.
Equation reproduced from the original PDF.Equation reproduced from the original PDF.Equation reproduced from the original PDF.Equation reproduced from the original PDF.
*Note that many papers were coded with two or more features per category.
Limited Evaluation Finally, we observe a significant lack of evaluation of the interventions. As mentioned, this is driven by the fact that most papers focus on detection and most commonly provide the response as an addition on a conceptual or prototypical level. As impairment detection represent only one side of DSMS, it would not be unreasonable to stipulate more efforts being given to the accompanying responses.
That said, among the papers that evaluated proposed solutions, we observe significant variety, which is summarized in the following section. However, most approaches were only represented by one or two studies, and the evaluation often seems to be lacking in depth. Integration is also limited by methodological diversity as well as inconsistent theoretical notions of impairments.
Beyond Detection: Intervention Approaches in Driver State Monitoring Systems 443
SYNTHESIS In this section, we synthesize the solutions that included any type of user evaluation. We first identified the types of evaluated interventions, resulting in thirteen unique approaches, which we clustered together based on their mechanism of action (Table 3).
Alert and Feedback Following the observations of the first part of our analysis and the trends in commercial DSMS, alert and feedback approaches were also the most common among the selected studies. They were intended for all types of impairment and generally offered the most discussion of design and HCI ele- ments, although to a limited extent. Alarm- and feedback-based approaches leave autonomy of action to the driver (or other persons of interest), and the main design questions that arise when it comes to their implementation is when and how to deliver them so that they are most effective in drawing attention and evoking desired action.
Relief and Assistance The next set of solutions concerns those that aim to relieve the driver of the workload required to safely operate the vehicle, for example, by assisting with
Table 3. Reviewed solutions, categorized and grouped by approach type.
Table available in the original PDF paper.
Solution type Papers in category Impairment(s) Alert and feedback
Fatigue, stress, cognitive load
Alerts Alotaibi and Asif, 2018; Lee et al. 2017; Nishigaki and Shirakata 2019; Pavlidis et al. 2021; Ibe et al. 2021; Li and Chung 2018; Tran et al. 2019; Wang et al. 2021 Driver state visualization Völkel et al. 2018; Ortoncelli et al. 2020 Fatigue, stress, cognitive load
Relief and assistance Risk augmentation Li and Ma, 2021; Wong et al. 2019 Cognitive load Task assistance Martelaro et al. 2019 Cognitive load Co-pilot assistance Tran et al. 2019 Fatigue Haptic guidance Wang et al. 2017 Fatigue
Passive mitigation Blue light Pramana and Puspasari, 2020 Fatigue Music Wang et al. 2021; Amirah and Puspasari, 2019 Fatigue Brain stimulation Li and Chung, 2018 Fatigue Social robot Hara et al. 2022 Fatigue
Active mitigation Gamification Ibe et al. 2021 Fatigue Exercise prompting Jang et al. 2017 Fatigue, stress Guided breathing Balters et al. 2018, 2019, 2020; Paredes et al. 2018; Zepf et al. 2020, 2021 Fatigue, stress
*In italicized studies, alerts were included but not presented as a primary solution.
the conduction of driving or secondary tasks, augmenting perception, or even taking over certain driving functions. Such solutions are already standard in existing ADAS, where they preform similar functions, albeit independent of driver state.
When it comes to integrating such solutions with DSMS, the addition is that ADAS actions are informed by the system’s outputs. Tran et al. (2019), for example, proposed a system where the level of interference increased with the level of fatigue. However, in their evaluation, subjects only mimicked fati- gue, and the authors did not consider the appropriateness of the interventions for the states.
Future research should determine if the proposed solutions are appropri- ate for the problems they aim to address and how they can be adapted to correspond to different levels of impairment.
Passive Mitigation The third set of interventions concerns those that help the driver alleviate impairment without them needing to keep an active role in the process. In all the surveyed studies that fell under this category, the proposed solution intended to mitigate fatigue, whether through blue light, music, or brain sti- mulation. An interesting case that can also fall under passive mitigation is that of the ‘social robot’, whose mere presence as a potential observer hel- ped preserve vigilance. While these solutions proved successful in reducing fatigue, they mostly did so for a limited time of about 30 minutes before it started increasing again. This is an important finding for this type of solution, as it gives the driver considerable time to act but also shows the limitation that they cannot be relied on as long-term solutions.
DSMS developers that consider utilizing this type of approach should also think about ways to encourage behaviors that lead to sustained mitigation or even promote habit change to avoid driving in fatigued conditions altogether.
Active Mitigation Finally, we observed several solutions that required the subjects’ active parti- cipation. These included a game to sustain alertness, an exercise-prompting infotainment system, and guided-breathing systems to alleviate stress and prevent fatigue. The haptic guidance system presented by Wang et al. (2017), which intended to assist drivers with lane keeping, also showed fatigue-mitigating effects.
As suggested in the previous section, solutions that require the users’ active involvement have the potential to yield additional benefits compared to those that are merely passive. However, utilizing such approaches brings additional problems, the main being that of potential counter-productivity. Solutions that require action from drivers present additional workload, which could shift attentional resources away from driving, negating its initially positive effect.
Indeed, the question of additional workload was raised in the evaluati- ons of the approaches in question. To address these problems, the authors
Beyond Detection: Intervention Approaches in Driver State Monitoring Systems 445
mostly suggested that the systems be active only in conditions where cognitive demands are low, for example, in long monotonous drives.
A question that follows from this is if these constraints on usability have an effect on user acceptance. While the solutions have generally received positive feedback in their initial assessments in experimental context, it is not unrea- sonable to question if this is good enough for broader adoption and everyday use. Future research should therefore put more emphasis on evaluating the solutions within the frame of user needs as well as requirements from other stakeholders.
CONCLUSION
In this review, we analyzed existing solutions in DSMS in technical acade- mic literature. Our results show that the field primarily focuses on detection, and the proposed solutions are usually simple and have low technological maturity. Studies that evaluate the solutions in user studies are diverse, both in terms of operation and methodology, although the systems are still in its infancy and require deeper evaluation. To move the field forward, researchers should aim towards greater methodological unity, use more robust definiti- ons of the studied phenomena, and choose consistent experimental protocols and outcome metrics. Another possible improvement is to evaluate systems over a prolonged period. The existing research predominantly views impair- ments as an acute problem, but other strategies can also be taken to improve safety, such as systemic and lifestyle changes.
ACKNOWLEDGMENT
This work was supported by funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement number 953426.
REFERENCES
Aghaei, A., et al. (2016) ‘Smart Driver Monitoring: When Signal Processing Meets Human Factors’, IEEE Signal Processing Magazine, 33(6), 35–48. Ahir, A. & Gohokar, V. (2019) ‘Driver Inattention Monitoring System: A Review’,
2019 International Conference on Innovative Trends and Advances in Engineering and Technology. Alotaibi, A. S. & Asif, A. (2018) ‘Wakeup: Designing Multimodal Alerts for Drowsy Drivers with Smart Phones’, 21st Saudi Computer Society National Computer Conference. Amirah, A. Y. & Puspasari, M. A. (2019) ‘Music as Countermeasure for Driving
Fatigue Using Brain Signal Indicator’, Proceedings of the 2019 5th International Conference on Industrial and Business Engineering. Arnold, L., et al. (2019) ‘Effectiveness of Distracted Driving Countermeasures: A Review of the Literature’, AAA Foundation for Traffic Safety. Balters, S., et al. (2018) ‘Breath Booster!: Exploring In-Car, Fast-Paced Breathing
Interventions to Enhance Driver Arousal State’, Proceedings of the 12th EAI International Conference on Pervasive Computing Technologies for Healthcare.
Balters, S., et al. (2020) ‘Calm Commute: Guided Slow Breathing for Daily Stress Management in Drivers’, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(1), 1–19. Balters, S., Landay, J. A. & Paredes, P. E. (2019) ‘On-road Guided Slow Brea-
thing Interventions for Car Commuters’, Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Barua, S. (2019) ‘Multivariate Data Analytics to Identify Driver’s Sleepiness, Cognitive load, and Stress’, Mälardalen University, Västerås. Brijs, T., et al. (2020) ‘i-DREAMS: an Intelligent Driver and Road Environment
Assessment and Monitoring System’, Proceedings of 8th Transport Research Arena. Fitzharris, M., et al. (2017) ‘The relative importance of real-time in-cab and external feedback in managing fatigue in real-world commercial transport operations’, Traffic Injury Prevention, 18(S1), S71–S78. Fredriksson, R., et al. (2021) ‘European NCAP Program Developments to Address
Driver Distraction, Drowsiness and Sudden Sickness’, Frontiers in Neuroergono- mics, 2(786674). Hara, K., Takemoto, A. & Nakazawa, A. (2022) ‘Drowsiness prevention using a social robot’, 31st IEEE International Conference on Robot and Human Interactive Communication. Hecht, T., et al. (2018) ‘A Review of Driver State Monitoring Systems in the Context
of Automated Driving’, Proceedings of the 20th Congress of the International Ergonomics Association. Ibe, T. (2021) ‘Development of Game-Like System Using Active Behavior Input for Wakefulness-Keeping Support in Driving’, IEEE Transactions on Intelligent Vehicles, 6(2), 323–332. Jang, S., Kim, H. & Cho, J. (2017) ‘Stretchertainment: Inducing Passive Stretching
with HUD Infotainment in Automotive’, Adjunct Proceedings of the 9th Inter- national Conference on Automotive User Interfaces and Interactive Vehicular Applications. Kashevnik, A., et al. (2021) ‘Driver Distraction Detection Methods: A Literature Review and Framework’, IEEE Access, 9, 60063–60076. Lee, B. G., et al. (2017) ‘Wearable Mobile-Based Emotional Response-Monitoring
System for Drivers’, IEEE Transactions on Human-Machine Systems, 47(5), 636–649. Li, G. & Chung, W. (2018) ‘Combined EEG-Gyroscope-tDCS Brain Machine Interface System for Early Management of Driver Drowsiness’, IEEE Transactions on Human-Machine Systems, 48(1), 50–62. Li, Z. & Ma, Y. (2021) ‘Response and Evaluation of Vehicle AR-HUD Assistant
System to Risk Cognition’, 16th International Conference on Computer Science & Education. Martelaro, N., Teevan, J. & Iqbal S. T. (2019) ‘An Exploration of Speech-Based Productivity Support in the Car’, Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Nazari, S. S. H., Moradi, A. & Rahmani, K. (2017) ‘A systematic review of the effect
of various interventions on reducing fatigue and sleepiness while driving’, Chinese Journal of Traumatology, 20(5), 249–258. Nishigaki, M. & Shirakata, T. (2019) ‘Driver attention level estimation using driver model identification’, 2019 IEEE Intelligent Transportation Systems Conference. Okoli, C. & Schabram, K. (2010) ‘A Guide to Conducting a Systematic Literature
Review of Information Systems Research’, SSRN Electronic Journal. Ortoncelli, A. R., et al. (2020) ‘Summarizing Driving Behavior to Support Driver Stress Analysis’, 15th IEEE International Conference on Automatic Face and Gesture Recognition.
Beyond Detection: Intervention Approaches in Driver State Monitoring Systems 447
Paredes, P. E., et al. (2018) ‘Just Breathe: In-Car Interventions for Guided Slow Breathing’, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(1), 1–23. Pavlidis, I., et al. (2021) ‘Biofeedback Arrests Sympathetic and Behavioral Effe-
cts in Distracted Driving’, IEEE Transactions on Affective Computing, 12(2), pp. 453–465. Politis, I., Brewster, S. & Pollick, F. (2013) ‘Evaluating multimodal driver displays of varying urgency’, Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Pramana, M. F. S. & Puspasari, M. A. (2020) ‘Evaluation on the Effects of Visual
Intervention on Emotion and Fatigue While Driving Using Brainwave Indicator’, The 6th International Conference on Industrial and Business Engineering. Ramzan, M., et al. (2019) ‘A Survey on State-of-the-Art Drowsiness Detection Techniques’, IEEE Access, 7, 61904–61919. Rastgoo, M. N., et al. (2019) ‘A Critical Review of Proactive Detection of Driver
Stress Levels Based on Multimodal Measurements’, ACM Computing Surveys, 51(5), 1–35. Tran, D., et al. (2019) ‘A Human-Vehicle Collaborative Driving Framework for Driver Assistance’, IEEE Transactions on Intelligent Transportation Systems, 20(9), 3470–3485. Völkel, S. T., et al. (2018) ‘I Drive My Car and My States Drive Me: Visualizing
Driver’s Emotional and Physical States’, Adjunct Proceedings of the 10th Inter- national Conference on Automotive User Interfaces and Interactive Vehicular Applications. Victor, T. (2011) ‘Distraction and Inattention Countermeasure Technologies’, Ergonomics in Design: The Quarterly of Human Factors Applications, 19(4), 20–22. Wang, F., et al. (2021) ‘Real-Time Detection and Early Warning of Driving Fati-
gue Based on Basic Scale Entropy’, 3rd International Conference on Artificial Intelligence and Advanced Manufacture. Wang, Z., et al. (2017) ‘The Effect of a Haptic Guidance Steering System on Fatigue- Related Driver Behavior’, IEEE Transactions on Human-Machine Systems, 47(5), 741–748. Wong, P. N. Y., et al. (2019) “‘Watch Out!”: Semi-Autonomous Vehicles Using Asser-
tive Voices to Grab Distracted Drivers’ Attention’, Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. Zepf, S. (2021) ‘Breath-Triggered Haptic and Acoustic Guides to Support Effortless Calm Breathing’, 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society. Zepf, S., et al. (2020) ‘Studying Personalized Just-in-time Auditory Breathing Guides
and Potential Safety Implications during Simulated Driving’, Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. Zuki, F. S. M. & Sulaiman, S. (2016) ‘Investigating sensory modalities in fatigue driver warning systems’, 4th International Conference on User Science and Engineering.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-71-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional