INTRODUCTION
The technologies and solutions of connected, cooperative and automa- ted mobility (CCAM) promise a broad range of benefits: increased safety, reduced environmental impacts, and inclusiveness. For these benefits to mate- rialize, CCAM’s technologies and mobility solutions must be widely adopted. A key factor for the acceptance and thus the success of CCAM is the level of
© 2022. Published by AHFE Open Access. All rights reserved. 40
Meeting User Needs in Vehicle Automation 41
subjective uncertainty that users experience when interacting with automa- ted vehicles (AVs). Subjective uncertainties occur when people cannot predict future events due to a lack of experience or information. If such uncertainties occur in the use of AVs, the formation of trust and thus the acceptance of this technology is impaired by the negative emotions associated with uncer- tainty. Consequently, the design of AVs must aim to prevent the occurrence of subjective uncertainties of their users. How such a system design must be conceived can be answered from the perspective of human factors, as taken up in the German-funded project AutoAkzept, whose full title translates to: Automation without uncertainty to increase the acceptance of automated and connected driving. In this paper, we provide an overview of AutoAk- zept’s empirical research and design work, present specific solutions aimed at improving AV users’ experience, increasing trust in automation and pro- moting acceptance of CCAM technologies, and introduce the underlying lead concept of user-focused automation. We will begin with the latter.
USER-FOCUSED AUTOMATION A lack of knowledge about AVs, which are new in their capabilities and characteristic in many ways, can trigger subjective uncertainty among users. Research shows that depending on the speed and maneuvering behavior of AVs, users may experience uncertainty in understanding, predicting, and evaluating the vehicle behavior or traffic situation (Hartwich et al., 2020). However, engaging in activities unrelated to driving, such as working in a mobile office, can also lead to subjective uncertainties about whether kinetosis is occurring, for example, or whether the time needed to reach a destination or system boundary is sufficient to complete important tasks. To reduce such subjective uncertainties in the use of AVs or automated systems in general, these systems must focus on the users and their needs. For insta- nce, recent studies point to the relevance of considering the information needs of users of AVs (Hartwich et al., 2021). To relate the needs of users and the design of AVs, we developed the approach of user-focused automation. The concept proposes to meet two basic human needs: the need to understand (NTU) and the need to be understood (NTBU). To account for the NTU, the design of automated systems must ensure that the technologies and technical functions not only do what they promise, but also what the users imagine. For example, AVs must behave predictably, even if they have never been used before. Implementing this requirement ensures that automated systems are transparent to their users, in such a way that they can easily infer functions and modes of operation and understand how a system works with mini- mal effort. To account for the NTBU, automated systems should recognize whether their users are uncertain or stressed and respond accordingly. A pre- requisite for such user-specific responses is that the systems recognize when it is appropriate to provide additional information, for example, and when it is not. Therefore, automated systems must focus on the user and be able to reco- gnize various human states as well as derive the appropriate system behavior. In order to prototype the concept of user-focused systems for the application domain of AVs and to demonstrate its effectiveness, various technical and
design solutions, concerning either the NTU or NTBU, were developed in the project.
USE CASES: SUBJECTIVE UNCERTAINTIES To this end, three use cases (UCs) of automated driving with a high level of automation (SAE level 4), all of which involved some kind of subje- ctive uncertainty, were implemented in user studies with real AVs or driving simulators:
UC1: Mobile Office. A vehicle occupant is working in an automated inter- urban vehicle, but is uncertain whether the current work can be completed in time before the AV reaches the limit of its operational design domain (e.g., leaving the highway). The system is able to distinguish this activity from other activities (e.g., relaxation), assesses the context, and adjusts the route and light setting of the interior accordingly to assist the occupant with the current activity, thus minimizing stress.
UC2: Robotaxi. While taking an inner-city ride, a passenger of a robotaxi is uncertain about the vehicle’s ability to safely navigate through complex (e.g., confusing) traffic situations. This uncertainty is detected by the system and interpreted concerning the current traffic context. The specific need to improve the occupant’s state is derived and appropriate measures to reduce uncertainty (e.g., by presenting information, or adjusting the driving style) are initiated.
UC3: Kinetosis. An AV reduces the risk of kinetosis and the associated uncertainty of whether and when it will occur by enabling the occupant to anticipate and muscularly counteract the forces that occur via peripheral perception by communicating vehicle behavior through adjustments to the interior lighting.
For all three use cases, we will present solutions developed and evaluated in the project. On the one hand, these include the adaptation of informa- tion transfer via human-machine interfaces (HMIs) as well as the creation and application of intelligent user profiles to optimize system behavior to address the NTU. On the other hand, they include the unobtrusive detection of relevant user states and activities of vehicle occupants as well as interven- tion strategies that adapt the interior, routing or driving style to address the NTBU. The results exemplify how user-focused system design can help strengthen the acceptance of CCAM technologies and systems.
OVERVIEW ON USER STUDIES UC1: Mobile OFFICE
Towards a user-adaptive vehicle interior for supporting mobile office work during CCAM (Ihme et al., 2021; Walocha, et al., accepted; Walocha, Drewitz & Ihme, accepted).
Objective: We aimed to create an interior set-up for an AV that satisfies the users’ NTBU when accomplishing activities, such as mobile office (MO) work or relaxing during automated driving. Hence, we developed methods
Meeting User Needs in Vehicle Automation 43
for assessing a user’s current activity and stress level as well as evaluated interior adaptations to support users accomplishing such activities.
Method & Main Results: A vehicle mock-up with flexible interior inclu- ding a retractable steering wheel, a revolving seat, a work space for MO, and an innovative light concept (focus light and ambient dome light) was set-up in a driving simulator. In this set-up, three studies were conducted in which participants accomplished MO work with different stress levels or could relax in their AV. In study 1 (Walocha, Drewitz & Ihme, accepted), real- time methods for assessing the activity and the stress level of the user based on body joint locations extracted from video streams and electrocardiogram were developed (29 participants). Using machine learning on the body joint locations, it was indeed possible to classify the three activities driving (classi- fication accuracy: 76%), MO work (93%) and relaxing (86%). In addition, a method to estimate the current stress level from heart rate and heart rate vari- ability could be developed and tuned. Study 2 (Ihme et al., 2021) proposes and evaluates strategies for supporting users during MO work and relaxa- tion (51 participants). It was revealed that users find it a very useful when a navigation system looks for routes with similar arrival times but longer auto- mated driving sections to extend the time that could be spend for MO work. Finally, in study 3 (Walocha et al., accepted), we evaluated whether users prefer whether the interior light shall automatically adapt to their current activity (user-adaptive) or if they prefer to be able to adapt the interior light on their own (adaptable) (13 participants). Interestingly, participants valued the hedonic qualities of the user-adaptive interior light and the pragmatic qualities of the adaptable variant. This suggests that users want to set the light according to their preferences, but appreciate an automated adaptation to their current activity and stress level. Together, this work demonstrates nicely how a combination of user assessment and a flexible interior design can be utilized to satisfy the users’ NTBU.
UC2: Robotaxi Towards a user-adaptive iHMI to address the user’s NTU in interactions with other traffic participants (Drewitz et al., 2021).
Objective. This study investigated the research question regarding the opti- mal presentation time or presentation distance of interacting other traffic participants for an automated vehicle’s adaptive internal human-machine interface (iHMI). The specific research question of this study was therefore: At what time or at what distance do the users of an AV wish to receive feedback on other road users from their iHMI to address their NTU?
Method & Main Results. To answer this research question, 106 par- ticipants in an online study were invited to contribute their information needs regarding this question directly to the iHMI interaction design as co-designers for an adaptive iHMI. The user state assessment required for user-focused systems was thus virtually simulated and replaced by the eva- luation of the study participants. The iHMI was based on the concept of a LED light-band, which highlighted other traffic participants perceived by the AV (Wilbrink, Schieben, & Oehl, 2020). As experimental factor, the type of
traffic interaction partner in the videos presented was systematically manipu- lated in a three-way gradation (male adult pedestrian vs. male adult cyclist vs. female child). All experimental participants experienced and rated all mani- festations of this factor. The dependent variable was, first, the individual distance at which participants desired system feedback (NTU) so as not to feel uncertainty in the interaction of their AV with another road user. On the other hand, the preferred time for a system feedback of the iHMI was deter- mined in more detail by means of an item, in relation to visibility of other road users as well as in relation to the braking process of the own AV. In addition, the online study also investigated the user experience (UX) for user interaction with this iHMI created in this co-design process. With regard to the study results participants expressed the highest NTU with regard to the interaction with the child followed by the cyclist and the pedestrian. Partici- pants’ subjective statements on the desired time for system feedback via the iHMI underline this finding: 68% of all test participants wanted the system to provide feedback as soon as the road user first appeared, if the road user was a child. In the case of a cyclist, 56% of all test participants also wanted system feedback when the road user was visible. The adult pedestrian, on the other hand, should only be indicated when visible by 42% of the test parti- cipants. Irrespective of the road user, 89% of the participants wanted system feedback via the iHMI even before the braking process. This iHMI received high UX ratings by the participants.
Group-specific effects of iHMIs with differing information availability on passenger experience and trust (Hartwich et al., 2021; Hollander et al., 2021). Objective. Based on passengers’ information needs previously identified (Hartwich et al., 2020), a user-adaptive iHMI concept for uncertainty redu- ction was developed. It included two iHMI versions with permanent vs. context-adaptive information availability, addressing the higher vs. lower information needs of passengers with lower vs. higher initial trust in AVs. In a driving simulator, group-specific effects of both iHMI versions on passenger experience and trust in AVs were evaluated.
Method & Main Results. 50 first-time users were divided into a lower and higher trust group based on initial trust in AVs indicated via questionnaire. All participants experienced three identical, fully automated rides along a mainly urban test track in a driving simulator in balanced order. The three rides differed in the iHMI condition presented in the center stack display (perma- nent vs. context-adaptive vs. no iHMI). The permanent iHMI provided full information (on system status, driving environment, driving maneuvers) any- time. The context-adaptive iHMI provided full information solely in complex situations (e.g. intersections, obstacles), but only system status information in non-complex situations. During the ride without iHMI, a speedometer was presented as a baseline. Passenger experience (discomfort) was assessed during driving via continuous real-time feedback, complemented by post-ride questionnaires on further aspects of passenger experience (perceived safety, understanding of driving behavior, comfort, enjoyment) and trust. Compa- red to the baseline, both iHMI versions improved trust and all aspects of passenger experience significantly. The higher trust group reported the best
Meeting User Needs in Vehicle Automation 45
passenger experience with the context-adaptive iHMI, while the lower trust group tended to a more improved passenger experience with the perma- nent iHMI. These results demonstrate the relevance of user-adaptive iHMI concepts addressing users’ individual NTU by customizing features such as information availability.
Investigating the individualization of AVs based on user characteristics (Brück, 2021).
Objective. Adapting the driving style of an AV to fit the user’s prefere- nces has the potential to reduce discomfort while driving. Therefore, we explored whether individual driving settings can be derived based on user characteristics.
Method & Main Results. We performed a driving simulator study with 25 participants to derive maneuver-based driving styles. A careful, normal, and risky driving style was defined based on the resulting driving data. These dri- ving styles were presented to users as video sequences during an online survey with 308 participants. Participants answered questions about demographic data, personality traits, technical affinity, trust in AVs, and driving styles. Afterward, different traffic scenarios were presented to evaluate the parti- cipants’ preferences with regard to the preferred driving style. The results show that the most promising factors to adjust an initial driving behavior are the manual driving style (anxious, risky, angry, and careful), technical affinity, and gender. Nevertheless, the correlations are significant but low or moderate. Therefore, they could help define the initial settings, but the user must be able to change these settings (e. g., with a graphical user interface, as described in Trende et al., 2019).
An Integrated Model for User State Detection of Subjective Discomfort in Autonomous Vehicles (Trende et al., 2020; Niermann et al., 2021).
Objective: Handing over control to an autonomous vehicle and the corre- sponding perceived loss of control can lead to an increased level of discomfort while driving. Detecting this discomfort can be used to adapt the system to reduce named discomfort.
Method & Main Results. A driving simulator study with 50 participants was conducted (see Hartwich et al., 2020). The participants wore physiologi- cal sensors to record the heart rate, and galvanic skin response. Furthermore, an eye tracker was used to record gaze and pupil dilation. Contextual data about the traffic situation was recorded by the traffic simulation software. Based on previous results described in Trende et al. (2020), an integrated model for user state detection of subjective discomfort was built. The model can achieve a precision of up to 80% by integrating physiological and contex- tual information. A SHAP analysis was performed to calculate each feature’s importance with regard to the model’s output.
Passenger uncertainty in an autonomous driving experiment. Objective. In AV experiments targeted at simulating autonomous driving in a shared space, we exposed the passenger to complex traffic situations having the potential to induce uncertainty in a passenger as to whether the AV will master the traffic situation safely. We asked the participants to indi- cate their subjective level of distress during the ride and correlated it both with physiological data and results from questionnaires regarding subjective
uncertainty and its causes taken after the experiment in an attempt to derive predictors for AV passenger uncertainty.
Method & Main Results. AV driving in a simulated shared space was undertaken on an airfield with a real AV. As the AV passenger, each of the 17 participants was subjected to seven different scenarios designed for this study. Five of the scenarios reflected complex traffic situations in which cyclists and pedestrians crossed the AV’s trajectory in close proximity. The remaining two scenarios were simple, had only little cyclist and pedestrian involvement and served as a baseline. The passenger was given a handheld regulator (simi- lar to a handheld speed controller used in a slot car game) to indicate his or her subjective level of distress in each scenario. The distress level was recorded along with physiological data (heart rate, most importantly). In order to avoid accidents, a safety driver was on board the AV, ready to take control of the vehicle at all times. For most participants, the signal from the handheld regulator increased with traffic situation complexity, signifying that they experienced a higher degree of passenger uncertainty. Small correlations were found between heart rates and signal levels from the handheld regula- tor, providing measurable evidence for the subjectively experienced distress. Participants reported that uncertainty was caused mainly by crossing pede- strians, passing cyclists, and confusing traffic situations. Here, the degree of uncertainty was higher, when the participant regarded the intentions of pede- strians and cyclists as ambiguous and their behavior as less predictable. The distance between the AV and other traffic participants played a lesser role for passenger uncertainty.
Intuitive iHMI design for increased subjective trust and acceptance of AVs (Pape et al., 2020) Objective. In order to reduce passengers’ uncertainty towards autonomous driving, we developed a display iHMI design concept informing passengers intuitively about the AV’s upcoming behavior in response to the current traffic situation. This system transparency is adapted to user needs and results in enhanced driving experience and leads to increased trust and acceptance.
Method & Main Results. The study context for the user-centered design and evaluation of the iHMI was two-pronged: at first, 20 participants rated multiple low-fidelity prototypes created by using rapid prototyping in semi- structured interviews after they had just completed multiple autonomous drives with situations eliciting uncertainty due to many other road users in a shared space. The developed alternatives differed in terms of informa- tion density, illustration type and the differentiation between (non-) critical road users and their intended trajectories. The resulting design specifica- tions of this co-design approach were finally condensed into two iHMI variants, which were evaluated in a second study (10 participants) showing the same real-life scenarios, but this time inside of the simulation environment Tronis®. Participants preferred a schematic vs. a photorealistic illustration (80%) from a third vs. first person point of view (72%). Furthermore, arrows proved to be an intuitive way of illustrating planned trajectories. There was no unified opinion on visually indicating distances (44% no indication vs. 27% showing distances vs. 22% only showing within focus area). Of the two developed iHMI alternatives, the one informing about the traffic situation
Meeting User Needs in Vehicle Automation 47
significantly outperformed the other both in terms of its effect on reducing uncertainty, a higher acceptance and higher user experience.
UC3: Kinetosis Addressing the user’s NTU with regard to the AV’s motions to prevent kinetosis (Hainich et al., 2021).
Objective: With the development and deployment of CCAM, kinetosis or so-called “motion sickness” will become an important issue to address, as passengers in AVs are predicted to be more likely affected by kinetosis than in manually controlled vehicles. Anticipatory motion cues provided via light-based iHMIs in the vehicle are a promising solution that can poten- tially alleviate kinetosis by addressing the user’s NTU. However, design and user experience issues of these iHMI solutions are still open. The- refore, we set the goal to develop an interaction strategy for the AV’s iHMI that prepares users for the AV’s directional changes und thus prevents kinetosis.
Method & Main Results. In a realistic driving study (Hainich et al., 2021) an iHMI concept that has already been tested in another context of use (Dzi- ennus et al., 2016) was applied to provide the AV’s user with transparency regarding the driving behavior of the automation. With the help of a LED light-band installed in the vehicle interior, an iHMI was realized with the aid of direction-resolved light signals. In the AV study on a standardized test track with 16 participants with medium kinetosis susceptibility, a kineto- sis reduction effect was evaluated based on the kinetosis increase during the experiment (control vs. iHMI group). Results showed that for a high kineto- sis sensitive subsample the iHMI showed a large effect, indicating that these study participants developed fewer kinetosis symptoms while driving in the AV during the iHMI condition. Thus, a kinetosis reduction effect was found within a highly sensitive subsample by presenting via the iHMI anticipatory ambient light cues on the LED light-band. The iHMI prototype was found to be effective with highly sensitive users for the NTU.
CONCLUSION
This paper describes the results and solutions of the AutoAkzept project for the design of AVs. We have presented the user-focused automation approach, which places basic human needs at the center of system design for automa- ted driving: the need to understand (NTU) and the need to be understood (NTBU). The project results from a variety of studies on the assessment of AV user’s states and activities to address the NTBU and on user-focused ada- ptation strategies to address the NTU provide two key insights. First, it is possible to assess key aspects of user states and user activities of AV users. Second, based on this, the user experience of AV users, which is central to the development of user trust and technology acceptance, can be positively influenced during the actual use by adequately adapting the system behavior. The design of AVs should take this into account.
ACKNOWLEDGMENT
The authors gratefully acknowledge the financial funding of this work by the German Federal Ministry of Transport and Digital Infrastructure under the grants 16AVF2126A, 16AVF2126B, 16AVF2126C, 16AVF2126D and 16AVF2126E.
REFERENCES
Brück, Y., Niermann, D., Trende, A., & Lüdtke, A. (2021). “Investigation of Personality Traits and Driving Styles for Individualization of Autonomous Vehicles.”, in: International Conference on Intelligent Human Systems Integration (pp. 78–83), Cham: Springer. Drewitz, U., Wilbrink, M., Oehl, M., Jipp, M., and Ihme, K. (2021). Subjektive Sich-
erheit zur Steigerung der Akzeptanz des automatisierten und vernetzten Fahrens. Forschung im Ingenieurwesen, 85, 997–1012. Dziennus, M., Kelsch, J., and Schieben, A. (2016). “Ambient light-based interaction concept for an integrative driver assistance system – a driving simulator study.”, in: de Waard, D., Brookhuis, K.A., Toffetti, A., Stuiver, A., Weikert, C., Coelho, D., Manzey, D., Ünal, A.B., Röttger, S., Merat, N. (Eds.). Proceedings of the Human Factors and Ergonomics Society Europe Chapter 2015 Annual Conference, 171–182. Hainich, R., Drewitz, U., Ihme, K., Lauermann, J., Niedling, M., and Oehl, M.
(2021). Evaluation of a Human–Machine Interface for Motion Sickness Mitiga- tion Utilizing Anticipatory Ambient Light Cues in a Realistic Automated Driving Setting. Information, 12 (4), 176. Hartwich, F., Hollander, C., Johannmeyer, D., and Krems, J.F. (2021). Improving passenger experience and trust in automated vehicles through user-adaptive HMIs: “The more the better” does not apply to everyone. Frontiers in Human Dynamics, 3:669030. Hartwich, F., Schmidt, C., Gräfing, D., and Krems, J.F. (2020). “In the passenger seat:
Differences in the perception of human vs. automated vehicle control and resulting HMI demands of users”, in: H. Krömker (Ed.): HCI in Mobility, Transport, and Automotive Systems. Automated Driving and In-Vehicle Experience Design. HCII 2020 (pp. 31-45), Cham: Springer. Hollander, C., Hartwich, F., Johannmeyer, D. and Krems, J. (2021). “Looking at HMI concepts for autonomous vehicles: System transparency effects trust, gaze beha- vior and HMI preferences of vehicle occupants.”, in: A. Huckauf, M. Baumann, M. Ernst, C. Herbert, M. Kiefer, M and Sauter (Eds.), Contributions to the 63rd Tagung Experimentell arbeitender Psychologen (112), Pabst Publ. Ihme, K., Walocha, F., Wegener, J., Rehm, J., Grolms, G., Nguyen, H.P., Niedling, M.,
Elster, D., Trende, A., Niermann, D. (2021). “Towards User-Focused Automated Vehicles Supporting Mobile Office Work”, in: Proceedings of the 7th Humanist Conference, Rhodes Island, Greece, 26–27 October 2021. Niermann, D., Trende, A., Ihme, K., Drewitz, U., Hollander, C., and Hartwich, F.
(2021). An Integrated Model for User State Detection of Subjective Discomfort in Autonomous Vehicles. Vehicles, 3(4), 764 – 777. Pape, A.-A., Cornelsen, S., Ihme, K., Oehl, M., Drewitz, U., Hartwich, F., Schroedel, F., Lüdtke, A., Schramm, M., Fäßler, V. (2020. “Empathic Assistants - methods and use cases in automated and non-automated driving”, in: M. Bargende, H.-C. Reuss and A. Wagner (Eds). 20. Internationales Stuttgarter Symposium. Proceedings, Wiesbaden: Springer.
Meeting User Needs in Vehicle Automation 49
Trende, A., Gräfing, D., and Weber, L. (2019). “Personalized user profiles for autonomous vehicles”, in: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. Trende, A., Hartwich, F., Schmidt, C., and Fränzle, M. (2020). “Improving the
detection of user uncertainty in automated overtaking maneuvers by combi- ning contextual, physiological and individualized user data”, in: International Conference on Human-Computer Interaction, 390–397. Walocha, F., Drewitz, U., and Ihme, K., Activity and Stress Estimation Based on OpenPose and Electrocardiogram for User-Focused Level-4-Vehicles. IEEE Transaction on Human-Machine Systems (Accepted). Walocha, F., Nguyen, H.P., Niedling, M., Drewitz, U., and Ihme, K. (2022). Evalu-
ation of a user-adaptive light-based interior concept for supporting mobile office work during highly automated driving. Accepted at AHFE 2022. Wilbrink, M., Schieben, A., and Oehl, M. (2020). “Reflecting the automated vehicle’s perception and intention: Light-based interaction approaches for on-board iHMI in highly automated vehicles”, in: Proceedings of the 25th International Conference on Intelligent User Interfaces Companion (IUI’20), 105–107.