Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Exploring user preferences regarding facial interface expressions of service robots across multiple age groups: A case study of the Kebbi Air robot

1 Department of Industrial Design, Tatung University, Taiwan ↗
2 The Graduate Institute of Design Science, Tatung University, Taiwan ↗
Publication year
Digital Object Identifier10.54941/ahfe1004618

Abstract

The integration of artificial intelligence (AI) and remote communication technology has enabled development and application of AI robots beyond industrial use, with such robots being applied for household use and services. In such applications, the facial expressions of robots are crucial to information exchange between humans and machines. The ability of robots to subtly change their facial expressions and respond sensitively to the emotional states of humans is a key focus in the development of service robots.This study analyses the Kebbi Air robot, exploring the preferences of users across various age groups (young people, middle-aged adults, and older adults) regarding the facial interface design styles applied to service robots (flat vs. realistic design). It also analyses how different design styles affect user recognition of robot facial expressions portraying emotions. This study developed a set of recommendations pertaining to facial expression styles for the Kebbi Air robot. The study comprised 2 phases. In the first phase, 21 older participants from Zuozhen, Tainan, Taiwan, were recruited to participate in a questionnaire survey and interview to enable assessment of their preferences regarding facial interface designs for robots. In the second phase, the survey plus interview format was repeated to compare the age-stratified data collected from four groups of participants stratified by age (i.e., older adults, middle-aged adults, prime adults and young people).The results indicate that regardless of design style, the younger participants were generally more accurate in recognising robot facial expressions than the other participants were. Furthermore, they demonstrated a higher level of emotional recognition for expressions portrayed in the realistic style, and they expressed a greater willingness to interact with robot interfaces. On the basis of this study’s findings, qualitative suggestions were proposed for various age groups; these suggestions encompassed style recommendations for robot facial expressions (e.g., eyebrows, eyes, mouth, and auxiliary symbols). Through its empirical exploration, this study provides valuable insights and recommendations for designing robot-friendly interfaces for multiple age groups.

© 2024. Published by AHFE Open Access. All rights reserved. 102

Exploring User Preferences Regarding Facial Interface Expressions 103

INTRODUCTION

In a global context, the proportion of elderly individuals is surging, leading to a gradual increase in caregiving burdens. As of 2021, the global popula- tion aged 65 and above reached 7.61 billion and is anticipated to surpass 1.6 billion by 2050 (Wilmoth et al., 2023). Taiwan is currently in the transitional phase before entering an ultra-aged society. While not yet at its peak com- pared to other countries, the elderly population is expected to exceed 21% by 2025 (Council, 2022). The imperative need to prepare for the anticipated ultra-aged society is imminent.

With the development of AI robots, their applications have expanded from traditional industries to a broader range of household and service sectors. Contributions in smart homes, home assistants, and some simplified medi- cal monitoring aspects aid in alleviating the productivity and caregiving gaps associated with the ultra-aged society. However, the use of technology some- times conflicts with the general population’s lifestyle. Given that the elderly are typically not the most adept at using interfaces, hindrances in operational experiences may diminish the confidence (Marquié & Huet, 2000) and task efficiency (Mead et al., 2000) of older individuals. Additionally, operators of different ages exhibit significantly different motivations and preferences in interacting with robots (Ellis & Marshall, 2019).

Intuitive and universally designed interface impacts the overall system usability, user satisfaction, and error prevention. This study identifies the style and patterns of facial expressions in robots and their ability to accu- rately convey emotions as critical factors influencing user preferences and task performance. It also aims to evaluate key factors in the user experience of service robots.

The Interface Design for Users of Different Ages The key factors influencing the effectiveness of user interface design are the usability generated by the design and the overall experience when using it through any type of device. Two predominant design trends in the field of user interface design are skeuomorphism and flat design (Spiliotopoulos et al., 2018). Skeuomorphic design emphasizes visually presenting a more skeuo- morphic appearance, serving as a method that fosters emotional connection and imagination (Oidekivi et al., 2021). It has also been proven to reduce cognitive burden for older users, making it a more user-friendly interface design for the elderly (Urbano et al., 2022). On the other hand, flat design emphasizes removing unnecessary decorations and elements, representing a minimalist approach that has been widely applied. This involves higher con- trast and saturation, simple color combinations, 2D illustrations, and ample white space (Johnson, 2013), allowing viewers to naturally focus on the content.

In the past, computer technology relied on graphical user interfaces (GUIs) as the primary interaction between users and computers. These GUIs were inspired by the appearance, feel, and potential behavior of physical objects in our daily lives. Currently, flat design is the prevailing trend compared to

skeuomorphic design. Although flat design is generally considered a challeng- ing proposition for older users, some studies confirm that specific metaphori- cal behaviors in user interface operation are unrelated to age (Hurtienne et al., 2010). The ability to link interface icons to real-world objects depends on past experiences. While this results in older users being more inclined to use skeuo- morphic designs that emphasize the physical appearance of the real world, an intuitive interface does not impact the ability and operational experience of older users. Previous research suggests a certain correlation between age and interface style preferences (Urbano et al., 2022). Understanding how users perceive the two design approaches in interface design, including the rec- ognizability and effectiveness of robot expressions, as well as users’ overall preferences for the two design approaches, is the focus of this study.

The Interface Design of Service Robots For service robots, gestures and facial expressions play a crucial role in exe- cuting human-like behaviors (Alcubilla Troughton, 2019) and serve as the primary non-verbal means of interaction between humans and robots, con- veying emotions and empathy. Park et al., (2021) suggest that the design of robot facial expressions should employ easily understandable simple lines. This not only prevents strong aversion caused by the uncanny valley but also helps prevent users from developing expectations that robots should exhibit facial expressions overly similar to humans. Following Mori et al. (2012) introduction of the Uncanny Valley theory, the relationship between the degree of robot realism and user preferences has gradually become a topic of discussion. Although these research findings do not entirely support the Uncanny Valley theory, it remains a noteworthy issue.

Therefore, this study uses the Kebbi Air (as shown in the Figure 1 below) model as a specific condition to explore the relationship between robot facial expressions and user preferences. The built-in features of Kebbi Air include an Automatic Speech Recognition (ASR) system, body expression movements (12 servo motors), touch detection (infrared sensors, touch sensors), and face recognition (5-megapixel recognition camera). It also supports voice wake- up and sound source localization. The robot is equipped with a 7-inch LCD screen, displaying facial features such as eyebrows, eyes, mouth, and decora- tive auxiliary expression symbols, including blush, tears, bulging veins, and other icons to aid in conveying emotions. Kebbi Air is a widely used home service robot in the Taiwan region.

Figure 1: Kebbi Air (image source: NUWA robtics).
Original PDF Figure 1:

Exploring User Preferences Regarding Facial Interface Expressions 105

Two-Phases Experiments In this study, we compiled the common elements of Paul Ekman and Robert Plutchik’s categorization of basic human emotions, which serve as six emo- tional options for users to identify the effectiveness of robot emotion. These emotions include joy, sadness, anger, disgust, surprise, and anticipation. Two styles of facial expressions were designed for each of these six emotions (as shown in the Table 1 below). To quantitatively measure users’ ability to rec- ognize robot expressions effectively, 12 robot expressions were presented to participants in a printed format for observation. Participants rated each expression on a Likert scale from 1 (strongly disagree) to 5 (strongly agree) based on their perceived emotional content. This scale was used to compare participants’ perceptions of the arousal levels of robot facial expressions in two different styles. After each item, participants were asked to provide rea- sons for their ratings, aiming to understand the ideal presentation of emotions associated with robot expressions according to participants’ perspectives.

Table 1. Robot expressions in flat design and skeuomorphic style (drawn by this study).

Table available in the original PDF paper.

Emotion Joy Sadness Anger Disgust Surprise Anticipation

Flat design Skeuomorphic design

This study consists of two phases. In the first phase, through a question- naire survey and interviews, several elderly participants were recruited in Zuo-Zhen, Tainan, Taiwan, to evaluate the preferences of elderly users for robot facial interface design and make process improvements. Twenty-one participants aged between 70 and 85 years were involved, with groups of 4 to 5 individuals taking turns. During the initial 10 minutes of the exper- iment, participants interacted with Kebbi Air, engaging in chat interactions and witnessing demonstrations of its singing and dancing capabilities. This aimed to familiarize them with the emotional meanings conveyed by robot facial expressions and body movements. A researcher explained the question- naire filling process to the team and conducted approximately 40 minutes of survey completion.

According to the results of the first experiment, the average scores of the 21 participants were all greater than 3.0 for each emotion cat- egory, with standard deviations less than 1.5, except for the emo- tion of disgust. Generally, participants believed that a skeuomorphic design style could better express the robot’s emotions compared to a flat design. Joy (MS=4.29>MF=4.18), sadness (MS=3.76>MF=3.30), anger (MS=4.05>MF=3.40), surprise (MS=4.10>MF=3.62), and antici- pation (MS=4.19>MF=3.67) were all rated higher in the skeuomorphic design, while flat design was favored in expressing the emotion of disgust (MS=3.52<MF=3.90). However, in this experiment, three participants had

hearing impairment, and most participants only understood Minnan dialect, lacking the habit of using electronic devices. Instructors had to spend a signif- icant amount of time explaining tasks and questions, resulting in suboptimal testing efficiency at the team level. Improvements were planned for the second phase, including participant recruitment, process optimization, and grouping methods.

In the second phase, using the same questionnaire and experimental frame- work, recruitment criteria required all participants to have normal vision and hearing, independent reading ability, and over one year of experience in operating electronic devices in the past three months. The Kebbi Air’s functionality demonstration continued with teams of 4–5 people, while the questionnaire survey and interviews were adjusted to one-on-one sessions to ensure accurate understanding and answering within a fixed time. In the end, a total of 42 participants were recruited for the second experiment, including 20 elderly individuals (average age = 71.0, SD = 4.29), 7 middle-aged (aver- age age = 59.5, SD = 2.56), 6 prime adults (average age = 35.0, SD = 2.00), and 9 young individuals (average age = 18.8, SD = 0.83). The answers of the four age groups of users were compared. Regarding the preference for robot facial expression styles, among elderly users, 19 preferred a skeuomorphic design interface, with only 1 choosing a flat design interface. In the middle- aged users, 6 preferred a skeuomorphic design interface, with only 1 choosing a flat design interface. Among prime adult group, 4 chose a skeuomorphic design interface, and 2 chose a flat design interface. In the young user group, 5 chose a skeuomorphic design interface, and 4 chose a flat design interface. On the other hand, this study aimed to understand whether there were differences in the effectiveness of emotion conveyed by robot expressions among elderly, middle-aged, adult, and young user groups. According to the analysis results of the ANOVA single-factor variance, the emotion recogni- tion results under different styles were not significant. Although the statistical results in this sample lacked evidence supporting the difference in the influ- ence of user age groups on the recognition of robot expressions in two design styles, the overall effectiveness of conveying emotions in the skeuomorphic style was greater than that in the flat style for all four groups. Surpris- ingly, prime adult users gave opposite results in the rating of robot expres- sions of sadness and anticipation (sadness MS=4.17<MF=4.33; anticipation MS=3.83<MF=4.33), and elderly users showed this phenomenon in the rat- ing of expressions of disgust and anticipation (disgust MS=2.75<MF=2.80; anticipation MS=3.35<MF=3.40).

If we consider scores above the average (3 points) as participants’ agree- ment that the robot effectively expressed a certain emotion, and vice versa, in the second phase of the experiment, a total of 42 participants were included in the calculation of the agreement percentage under different styles and emotions. For the skeuomorphic style, the agreement percentages for each emotion, from highest to lowest, were: joy 88%, anger 86%, surprise 76%, sadness 74%, anticipation 69%, and disgust 40%. For the flat style, the agreement percentages for each emotion were, from highest to lowest: antic- ipation 67%, sadness 62%, surprise 60%, anger 57%, joy 45%, and disgust 31%. According to the percentage statistics, regardless of the presentation

Exploring User Preferences Regarding Facial Interface Expressions 107

style, participants found it challenging to distinguish the emotion of ‘disgust’ in robot expressions, consistent with the first phase. In the interview data, key opinions from participants on expressing the emotion of disgust in robot facial expressions included: a need for a sense of disdain in the eyes, rolling eyes, and a stern gaze; in terms of the mouth, sticking out the tongue, tilting the mouth to one side, and overall, making a face or wrinkling the face could effectively express the emotion.

CONCLUSION

This study explores the preferences of users of different ages for the facial interface design styles of service robots and the extent to which different design styles affect users’ recognition of robot expressions representing emo- tions. In the first phase, the study found that elderly users are more willing to use a skeuomorphic design style of robot interface compared to a flat design style, as the former has better emotional recognition. Among the six emo- tions, disgust is more challenging to express through the interface, and these phenomena were highlighted in the second-phase experiment. Although the effectiveness of users of different ages in recognizing emotions conveyed by robot expressions could not be statistically confirmed, this study, through two phases of surveys and interviews, provides insights and references for future designs of robot expressions that are universally usable for all ages and specifically cater to the interface design for the elderly population, offering reminders and considerations in iterative design and emotional recognition.

Throughout the observation process, this study received feedback on color, hue, brightness, and saturation. Different age groups showed slight dif- ferences in perception and preferences regarding these details. Faced with future more refined interface designs, the research can evaluate the impact on user usability or user experience concerning the presentation of forward and backward colors..

REFERENCES

Alcubilla Troughton, I. (2019). Feel with Me: From Simulation Theory to Empathetic Encounters in Human-Robot Interaction Council, N. D. (2022). Projected Aging Schedule of the Population in the Republic

of China. https://www.ndc.gov.tw/Content_List.aspx?n=D527207EEEF59B9B Ellis, A., & Marshall, M. T. (2019). Can skeuomorphic design provide a better online banking user experience for older adults? Multimodal Technologies and Interaction, 3(3), 63. Hurtienne, J., Stößel, C., Sturm, C., Maus, A., Rötting, M., Langdon, P., & Clarkson,

J. (2010). Physical gestures for abstract concepts: Inclusive design with primary metaphors. Interacting with Computers, 22(6), 475–484. Johnson, S. (2013). The rise of flat design. Target Mark, 36, 8-9. Marquié, J. C., & Huet, N. (2000). Age differences in feeling-of-knowing and confidence judgments as a function of knowledge domain. Psychology and Aging, 15(3), 451. Mead, S. E., Sit, R. A., Rogers, W. A., Jamieson, B. A., & Rousseau, G. K. (2000).

Influences of general computer experience and age on library database search performance. Behaviour & Information Technology, 19(2), 107–123.

Mori, M., MacDorman, K. F., & Kageki, N. (2012). The uncanny valley [from the field]. IEEE Robotics & Automation Magazine, 19(2), 98–100. Oidekivi, M., Nolte, A., Aabloo, A., & Kruusamäe, K. (2021). Identifying emotions

from facial expression displays of robots-results from a survey study. 2021 14th International Conference on Human System Interaction (HSI), Park, U., Kim, M., Jang, Y., Lee, G., Kim, K., Kim, I.-J., & Choi, J. (2021).

Robot facial expression framework for enhancing empathy in human-robot inter- action. 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), Spiliotopoulos, K., Rigou, M., & Sirmakessis, S. (2018). A comparative study of skeuomorphic and flat design from a UX perspective. Multimodal Technologies and Interaction, 2(2), 31. Urbano, I. C. V. P., Guerreiro, J. P. V., & Nicolau, H. M. A. A. (2022). From skeuo-

morphism to flat design: age-related differences in performance and aesthetic perceptions. Behaviour & Information Technology, 41(3), 452–467. Wilmoth, J. R., Bas, D., Mukherjee, S., & Hanif, N. (2023). World social report 2023: Leaving no one behind in an ageing world. UN.

Cover of the proceedings volume Accessibility, Assistive Technology and Digital Environments

Explore the complete digital volume

Accessibility, Assistive Technology and Digital Environments

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-97-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →