Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

PersonalAIzation - Exploring concepts and guidelines for AI-driven personalization of in-car HMIs in fully automated vehicles

1 Future Everyday Department, Eindhoven University of Technology, Eindhoven 5612, AZ, The Netherlands ↗
2 Institute of Human Factors and Technology Management IAT, University of Stuttgart, BW 70569, Germany ↗
3 Fraunhofer Institute for Industrial Engineering IAO, Stuttgart, BW 70569, Germany ↗
4 Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB, Karlsruhe, BW 76131, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1002474

Abstract

The role of the driver changes to that of a passenger in autonomous cars. Thus, the vehicle interior transforms from a cockpit into a multimedia station and workspace. This work explores concepts for Artificial Intelligence (AI) to provide a personalized user experience for the passengers in the form of Contextual Personalized Shortcuts and Personalized Services in the infotainment system. The two use cases were iteratively developed based on literature research and surveys. We evaluated AI- Personalized Services and compared AI-generated to the manually configurable shortcuts. AttrakDiff (Hassenzahl et al., 2003) and Car Technology Acceptance Model (CTAM; Osswald et al., 2012) were used to evaluate UX and user acceptance. The AI-Personalized interface obtained positive scores and reactions in the user testing and shows potential. Based on the insight from the user studies and literature review, we present and human-AI interaction guidelines to build effective AI-personalized HMIs.

INTRODUCTION

Conventional human machine interfaces (HMIs) are usually a “one size-fits- all” solution sometimes with configurable options that rely on user input. Such systems have an inability to cater to differences in knowledge, style, and preferences of users. This emphasizes the need for streamlined commu- nication between future cars and its occupants (Amditis et al., 2006) and thus, personalized interfaces, wherein the HMI adapts to the behavior and preferences of the user (Langley, 1997). As the users’ needs and behaviors change, the HMI needs to adapt accordingly (Hasenjäger et al., 2017). AI

© 2022. Published by AHFE Open Access. All rights reserved. 423

in its essence, is a learning system (Winkler, 2019), that can take user data as input and build a user model for dynamic adaptation. Thus, AI driven personalization is highly suitable for HMIs, especially in automated vehicles. This work contributes to the state of the art by investigating the effect of AI- Based Personalization of Automotive HMIs in fully automated vehicles on the User Experience and User Acceptance of its occupants. We build upon related work from research on in-car HMIs and Human-AI interaction to identify and explore relevant use cases.

USE CASES FOR AI-BASED PERSONALIZED HMIS The inspiration and input from the related work was combined with surveys and studies for a systematic exploration of the research questions. We began by outlining use cases for AI-based personalization in automated vehicles. First, we identified the intended target group of AI personalized HMIs – users aged between 18-45 years – as they have been surrounded with tech- nology since a very young age and would be the first ones with access to and interest in automated vehicles (Mangelsdorf, 2015; Bolton, 2013). We then invited 6 representative users of the target group to an online brain- storming to gauge their perception of AI in car HMIs. We conducted another focus group with 6 researchers of mass personalization in the field of automo- tive human factors. The Focus group lasted 60 minutes with “Identification of use cases for AI-Personalized HMIs” as the purpose statement (Wilkin- son, 1998). Additionally, we interviewed experts in the field of research on automotive human factors on their views on status quo of AI in cars and identification of use cases for AI-Personalized HMIs. We also conducted a systematic literature review to identify activities that vehicle users currently conduct in their car and desired activities for the future: Pfleging et al. (2016) helped us to identify use cases such as media consumption, news, gaming, and fitness. Braun et al. (2020) allowed us to find use cases such as proactivity, navigation etc. for automated driving. Subsequently, we were able to identify the potential of using AI to customize the user interface through the work of Gaffar et al. (2016). Secondary research (PYMNTS.com, 2019; Zoghby et al., 2018; Feinberg et al., 2017; Segment, 2017) revealed current activities conducted with cars when not commuting, such as ordering food for takea- way, dining at drive thru, grocery shopping etc., as well as a key interest in personalized experiences. After removing duplicates, this process resulted in 30 potential use cases for AI personalization in automated vehicles. Out of 30 potential use cases, we chose 2 for a detailed study: The first use case, Contextual Personalized Shortcuts, is based on earlier research on personalized HMIs. Garzon stated that car infotainment systems had at least 3 levels of interaction before accessing most used functionalities. Garzon proposed the following solution: shortcuts to often used functio- nalities based on the context functionalities are most used in (2012). We validated the relevance of Contextual Personalized Shortcuts using Screens Studio (www.screensstudio.com), an online car HMI benchmarking tool: We analyzed the number of steps necessary to access the most frequently used functionalities (Garzon, 2012) for 5 state-of-the-art vehicles, see Figure 1. We

Personalization – Exploring Concepts and Guidelines for AI-Driven Personalization 425

Figure 1: Analysis of steps necessary to access most frequently used features in 5 state-of-the-art automotive HMIs, including analysis of utilization of shortcuts.
Original PDF Figure 1:

also assessed whether the HMIs included shortcuts, the number of shortcuts, and how the shortcuts were implemented.

The analysis revealed a continuing relevance of Contextual Personalized Shortcuts, as users of current automotive HMIs must take 3-6 steps just to access commonly used functionalities. Shortcut analysis revealed that shor- tcuts are not based on context. Thus, the shortcuts do not adapt to users’ changing needs. This makes research on Contextual Personalized Shortcuts relevant to users.

We identified the second use case, Personalized Services, based on subtasks of current in-car activities, e.g., ordering food for takeaway, grocery shop- ping, visiting restaurants, etc. Potentially relevant subtasks for Personalized Services include restaurant reservation, grocery shopping, ordering groceries for pick up, ordering food for takeaway, ordering coffee for takeaway, and ordering food in a drive through (PYMNTS.com, 2019). To choose the three most interesting out of these subtasks, we conducted an online survey with n = 60 participants. We described the subtasks and how they would work when based on AI. The three subtasks ranked most highly were: picking up food for takeaway, visiting restaurants, and grocery shopping.

DESCRIPTION OF THE USE CASES We formalized the two use cases for a detailed study and implementing prototypes for evaluation.

  • Contextual Personalized Shortcuts: The HMI should display personalized shortcuts to functions (such as making a call, playing a particular radio channel etc.) based on the users’ frequency of this activity in a particular context (day, time, and location).
  • Personalized Services: The AI-personalized HMI provides personalized recommendations for grocery ordering, dining out or ordering food for takeaway based on the users’ decisions in the past. E.g., if the user shops

for apples at a certain supermarket chain on the way home on Fridays, the HMI provides personalized recommendations for groceries and commu- nicates with the smart home to recommend missing grocery items. Upon selection, the system places the order based on estimated waiting times and navigates the car to a supermarket where the user can pick it up.

We validated user interest for both use cases with a survey (n = 30). Partici- pants stated their general interest in AI as 75% on a scale of 0% (no Interest) to 100% (Very interested). We also queried whether participants already had AI-based HMIs in their cars. 93% of the users did not. We then described the 2 use cases Contextual Personalized Shortcuts and Personalized Services, and asked participants whether they were interested to use the respective system. The survey confirmed relevance, goals, potential pain points, and motiva- tors for the use cases. Data from the survey was used to create a persona as representative of the target group for our prototype.

THE AI-BASED PERSONALIZED HMI Based on the user-centered design approach (Abras et al., 2004) we star- ted with a user journey, curating guidelines on human-AI interaction, and iteratively implementing, and refining the prototypes.

First Iteration: Paper Prototype We used the goals of our persona and use cases to write the user journey, based on the question “How would the persona use the system described in the two use cases in a perfect day in their life?”. We searched literature for guidelines on human-AI interaction as basis for the prototype, but quickly realized that we had to curate guidelines ourselves. A total of 59 guideli- nes were curated from over 15 sources, which include academic research as well as best practices from big organizations (Amershi et al., 2019; Uga, 2019; Lovejoy, 2018; Baxter, 2017; Holbrook et al., 2017; Nielsen, 2005; Shneiderman, 2004; Horvitz, 1999). Based on this we evaluated a paper pro- totype (see Figure 2a) with 2 human factors experts and gathered valuable feedback on improving the interaction flow for the user. This was used as a blueprint for the high fidelity protype built in InVision1 (see Figure 2b).

Second Iteration: High-Fidelity Prototype & Expert Evaluation The high-fidelity prototype was implemented to compare AI and non-AI shortcuts and evaluate Personalized Services. We conducted the Heuristic Evaluation with 6 experts in the field of automotive Human Factors, using Nielsen’s 10 heuristics (Nielsen, 2005) and the ISO 9241-110 (ISO, 2020). We identified 5 areas of optimization potentials to enhance our prototype: feed- back given by AI, recommendations given, understandability of user control on choices, reversibility of actions to increase user control, and aesthetics.

1www.invisionapp.com/studio

Personalization – Exploring Concepts and Guidelines for AI-Driven Personalization 427

Figure 2: Exemplary screens from paper prototype (a, left) and clickable prototype (b, right) used for expert review and user study.
Original PDF Figure 2:
Figure 3: Example for use case Personalized Services: Based on user selection history, different cuisines are recommended for a lunch break enroute. Based on user choice, available restaurants are then recommended.
Original PDF Figure 3:

Third Iteration: User Study on Refined High-Fidelity Prototype A pilot study (with 4 participants) resulted in a change of the comparison condition for the use case Contextual Personalized Shortcuts: Instead of “walking through menus step by step”, we implemented manually configu- red shortcuts as comparison condition based on the pilot study. We compared the use of AI to provide Contextual Personalized Shortcuts through the HMI helps the user perform frequent actions, impacts the user experience and the user acceptance.

We conducted remote moderated user testing with 14 users. Users had to test the prototype on a tablet device, with a driving video playing on the com- puter in front of them, to simulate the driving environment. They were given tasks based on the interaction scenario in three blocks (see Figure 4). Persona- lized Services (see Figure 3) were evaluated by themselves as a contemporary HMI did not exist.

RESULTS AND DISCUSSION

For the use case Contextual Personalized Shortcuts, the effect of Contextual Personalized Shortcuts on user experience was positive compared to manually configured shortcuts. The Contextual Personalized Shortcuts received higher

Figure 4: Procedure of user study for the two use cases including sub tasks and measurements.
Original PDF Figure 4:

ratings for 8 out of 10 qualities. It also scored better on the AttrakDiff’s port- folio of results, achieving “desirable” Pragmatic and Hedonic qualities. Users expressed a positive response towards the contextual aspect of the personali- zation, simplicity as well as transparency. They also expressed that a greater number of shortcuts as well as an ability to have both AI personalized and manually configurable shortcuts together would be a welcome addition.

For the exploration of Personalized Services, results from the CTAM & AttrakDiff questionnaires revealed that users exhibited positive user experi- ence and user acceptance, but Perceived Safety and Social Influence needed improvement. For both subtasks of Personalized Services, the concept scored well for user experience in the AttrakDiff, once again achieving “desired” Pragmatic and Hedonic qualities in the portfolio in both cases. It also sco- res well for User Acceptance on CTAM, except for the constructs of Social Influence and Perceived Safety. In the case of Personalized Services, users expressed positive feelings towards the simplicity of the concept and user flow, the detailed recommendations provided by the system, as well as the end-to-end solution. Participants expressed concerns over the fact that this system felt slightly unpredictable due to the AI’s behavior, and the fact that they felt restricted by the AI’s choices.

The opinions expressed by the users in the detailed qualitative interviews and the results from the questionnaires in the user study reveal that users are interested in AI-personalized infotainment systems in the vehicles. While the system fascinated them, the users also expressed certain concerns about the AI-personalized systems, such as being restricted by choice, and unpredictable behavior. These concerns highlight the areas of concern to be addressed when designing AI-driven HMIs for automated vehicles in the future. We also took these concerns into consideration while compiling our final list of Human-AI interaction guidelines for AI-Personalized in-car HMIs.

GUIDELINES FOR AI DRIVEN IN CAR HMI(S) Based on insights derived from the extensive surveys, user studies and lite- rature review, we propose a consolidated set of interaction design guidelines for AI based personalized in-car HMIs (see Table 1). These have been catego- rized based on the different phases before, during, and after interaction takes place with the user and the HMI.

Personalization – Exploring Concepts and Guidelines for AI-Driven Personalization 429

Table 1. Guidelines for human-AI interaction in automated vehicles, based on existing guidelines from human-machine interaction and feedback on our AI- personalized HMI.

Table available in the original PDF paper.

Guideline Definition During Interaction Timing is Critical During an important activity, the AI should not override the user’s task with a non-critical suggestion or activity. Speed is Important The AI should be able to let the user complete their tasks faster and more accurately, thus having an edge over conventional systems and contributing to improved UX. Use simple, conversational language

This enables the user to build trust in the system due to clear communication. Terms like “suggested”, “recommended” are used to keep expectations realistic. Require few steps The AI should not complicate the steps to execute current tasks or personalized ones. It may lead to high cognitive load and disrupt UX. When Wrong 6 Let the user know of resources behind the AI’s output

The user should be aware of the resources that influence the AI’s output, the data which the recommendation comes from, and/or the factors that affect personalized output. Offer an escape hatch User should always be able to decline the AI’s recommendations & suggestions; perform the same tasks at their will. This is important to ensure user control and build trust. Over Time/Long Term

Maintain a working memory of recent interactions

The system should maintain a working memory of the user’s recent interactions with the system, since this is essential for the building the user profile for personalization. Continue learning by observation

The system should be endowed with the ability to become better at working with the users by continuously learning about the user’s goals and needs, making AI-driven personalization dynamic. Update and adapt cautiously

AI should adapt to changes in users’ goals and requirements subtly, thereby avoiding disruptions in UX through sudden changes. For Personalization

The personalization should be visual and discreet

Visual feedback in the car should inform the user of personalized functions, while not affecting regular operation of the car. The Personalization should be contextually relevant

The personalized content should be relevant to the users’ current task and situation, to enable understandability of presented content and effective interaction. Providing opportunities for users to give feedback

The user should be enabled to provide feedback or have alternate choices to specify their goals, as user goals drive personalization.

ACKNOWLEDGMENT

The authors would like to thank all participants for their contribution to our work. This work was supported by a grant from the Baden- Wuerttemberg Ministry of Economic Affairs, Labour and Tourism (Project ≫KI-Fortschrittszentrum Lernende Systeme und Kognitive Robotik≪).

REFERENCES

Abras, C. et al. (2004) “User-centered design”, Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications, 37(4), pp. 445–456. Amditis, A. et al. (2006) “Communication and interaction strategies in auto-

motive adaptive interfaces”, Cognition, Technology & Work. Springer, 8(3), pp. 193–199. Amershi, S. et al. (2019) “Guidelines for human-AI interaction”, in Proceedings of the 2019 chi conference on human factors in computing systems, pp. 1–13. Baxter, K. (2017): How to Meet User Expectations for Artificial Intelligence.

Research-based recommendations to create a human-centered AI experience. Salesforce Design. Available online at https://medium.com/salesforce-ux/how -to-meet-user-expectations-for-artificial-intelligence-a51d3c82af6, checked on 2/14/2022. Bolton, R. N. et al. (2013) “Understanding Generation Y and their use of social media: a review and research agenda”, Journal of service management. Emerald Group Publishing Limited. Braun, M. et al. (2020) “What if your car would care? Exploring use cases

for affective automotive user interfaces”, in 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 1–12. Feinberg, A., Mullapudi, M., Benore, M.; Page, O. (2017): The Restaurant of the Future. Creating the next-generation customer experience. Deloitte Digital. Avai- lable online at https://www2.deloitte.com/tr/en/pages/consumer-industrial-produ cts/articles/restaurant-future-survey-technology-customer-experience.html, che- cked on 2/14/2022. Gaffar, A., Monjezi, S. (2016) “Using artificial intelligence to automatically custo-

mize modern car infotainment systems”, in Proceedings on the International Conference on Artificial Intelligence (ICAI). The Steering Committee of The World Congress in Computer Science, Computer …, bl 151. Garzon, S. R. (2012) “Intelligent in-car-infotainment systems: A contextual personalized approach”, in 2012 Eighth International Conference on Intelligent Environments. IEEE, pp. 315–318. Hasenjäger, M., Wersing, H. (2017) “Personalization in advanced driver assistance

systems and autonomous vehicles: A review”, in 2017 ieee 20th international conference on intelligent transportation systems (itsc). IEEE, pp. 1–7. Hassenzahl, M., Burmester, M., Koller, F. (2003) “AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität”, in Mensch & Computer 2003. Springer, pp. 187–196. Holbrook, J., Lovejoy, J. (2017): Human-Centered Machine Learning. 7 Steps to

stay focused on the user when designing with machine learning. Google Design. Available online at https://medium.com/google-design/human-centered-machine- learning-a770d10562cd, checked on 2/14/2022. Horvitz, E. (1999) “Principles of mixed-initiative user interfaces”, in Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp. 159–166.

Personalization – Exploring Concepts and Guidelines for AI-Driven Personalization 431

Langley, P. (1997) “Machine learning for adaptive user interfaces”, in Annual Conference on Artificial Intelligence. Springer, pp. 53–62. Lovejoy, J. (2018): The UX of AI. Using Google Clips to understand how a human-

centered design process elevates artificial intelligence. Googe Design. Available online at https://design.google/library/ux-ai/, checked on 2/14/2022. Mangelsdorf, M. (2015) Von Babyboomer bis Generation Z: der richtige Umgang mit unterschiedlichen Generationen im Unternehmen. GABAL Verlag GmbH. Nielsen, J. (2005) “Ten usability heuristics”. Available online at https://www.nngrou

p.com/articles/ten-usability-heuristics/ Osswald, S. et al. (2012) “Predicting information technology usage in the car: towards a car technology acceptance model”, in Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 51–58. Pfleging, B., Rang, M., Broy, N. (2016) “Investigating user needs for non-

driving-related activities during automated driving”, in Proceedings of the 15th international conference on mobile and ubiquitous multimedia, pp. 91–99. PYMNTS.com (2019): The Digital Drive Report 2019. The Road to $230 Billion in commerce behind the wheel. PYMNTS.com. USA. Available online at https: //www.pymnts.com/the-digital-drive/, checked on 2/13/2022. Segment (2017): The 2017 State of Personalization Report. Segment. Available online

at http://grow.segment.com/Segment-2017-Personalization-Report.pdf, checked on 2/14/2022. Shneiderman, B. (2004) “Designing for fun: how can we design user interfaces to be more fun?”, interactions. ACM New York, NY, USA, 11(5), pp. 48–50. Standard ISO 9241-110, 5/1/2020: Ergonomics of human-system interaction — Part

110: Dialogue principles. Available online at https://www.iso.org/standard/75258. html. Uga, B. (2019) “Towards Trustworthy AI: A proposed set of design guidelines for understandable, trustworthy and actionable AI”. Wilkinson, S. (1998) “Focus group methodology: a review”, International journal of

social research methodology. Taylor & Francis, 1(3), pp. 181–203. Winkler, M. (2019): Accelerating Automotive’s AI Transformation. How driving AI enterprise-wide can turbo-charge organizational value. Capgemini Research Insti- tute. Available online at https://www.capgemini.com/ca-en/wp-content/uploads/s ites/10/2019/03/Ai-in-automotive-research-report-1.pdf Zoghby, J., Tieman, S., Moino Perez, J. (2018): Making It Personal. Why brands

must move from communication to conversation for greater personalization. Pulse Check 2018. Available online at https://www.accenture.com/t20171031T 052156Z__w__/us-en/_acnmedia/PDF-34/Accenture-Pulse-Check-Dive-Key-Fin dings-Personalized-Experiences.pdf, checked on 2/12/2022.

Cover of the proceedings volume Human Factors in Transportation

Explore the complete digital volume

Human Factors in Transportation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-36-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →