Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Framework for AI based UX: rethinking design process

Department of design, Indian Institute of Technology (IIT) Delhi New Delhi, India- 110016 ↗
Publication year
Digital Object Identifier10.54941/ahfe100916

Abstract

Artificial intelligence (AI) posits an important question in the 21st century “How do we ensure that when there is a human in the loop — such as in complex or life-changing decision-making — they remain critical and meaningful, while creating and maintaining an enjoyable user experience?” We are still looking for answers as the world is changing at a faster pace beyond human cognitive limits, encouraging human-centered design to be upcoming social challenges in context to HCI’s engagement with AI systems. The present study addresses several challenges faced by designers while creating AI-based products that provide meaningful and relevant experiences to users. We review characteristics of unique experiences and propose a preliminary framework to design AI-driven experiences through a case study.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

INTRODUCTION

AI-enabled products have been increasing exponentially and penetrating the market. From driving internet-enabled smart cars to conversing with voice-controlled intelligent systems at home, AI is prevalent and ubiquitous. Infiltration of AI is intensely entwined into daily routine, and thus it becomes a part of our life. There is no doubt that AI would continue expanding into new domains with frequent updates, as technologies and data are evolving swiftly that could impact the nature and applications of AI-enabled products. However, being such a profitable technology, AI is still grappling to secure position and trust in the market due to higher failed incidences. The significant reasons for failure are attributed to adolescent malfunctions, errors, and biases. For example, the AI-powered tool (Genderify)identifying gender based on name or email address faced opposition from society, and hence was shut down immediately after launch(Cai and Yuan 2021). People have concern about their privacy, and labeling data according to gender breaches their self-identity, which highlights the role of users in designing AI-enabled products. Thus, it becomes an important step to understand users and their experiences with AI-enabled products for making them successful and beneficial. As we involve users in innovative product development, the need to involve designers from the very beginning arises. Hence, user-centered design becomes a key for developing efficient and useful AI-enabled products. Designers are considered an important asset of a business because nowadays companies are keen to foster a culture of innovation to beat in competition(Spencer et al. 2018). UX designers have started integrating AI’s capabilities into their practices; however, they still face challenges in designing and innovating valuable human-AI interactions. UX designers are habitual in designing a system that poses a deterministic and closed set of functionalities. On the contrary, AI system could evolve and adapt making them unpredictable during design process. Hence the designer needs to update existing practices. Big firms such as Google and Microsoft are now engaging both AI practitioners and UX designers for the implementation of AI-empowered systems at the very beginning of product developmental cycles (Kayacik et al. 2019). When working as a team, UX experts had to avoid misunderstanding the capabilities of the ML models which act as key enablers for the deepest comprehension of the system and the user needs(Margetis et al. 2021). Integrated AI and human-centric workforce would bring societal values in the design of AI-enabled products and subsequently could develop checks and controls to prevent the mistakes and to ensure that mistakes are detected. Developing a human-centered AI-based product (such as a virtual assistant) is more complicated than advancements in the feasibility of the technology and involves challenges that are beyond technical limitations. Another key aspect of human-centered design is to include anthropomorphism for designing “socially embodied AI” to carry out tasks and support decision-making with and for people. AI-enabled products that fall under such categories are Robots, recommender systems, voice assistants, and virtual humans (Seaborn et al. 2021). In this short paper, we will catalog the many human-AI interactions design challenges and propose a preliminary framework to address human-AI interaction design challenges with each phase of design cycle. We offer an example of usability testing for an AI-enabled product using our framework.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

Related Work

There are two reported challenges with the existing AI system i.e., capability un- certainty and output complexity, which make interaction design so difficult (Lew and Schumacher Jr. 2020; Yang et al. 2020). The first challenge refers to the AI characteristic of continuous evolution through learning, which results in an unbound space of capabilities (what the system can do) and resulting performances (how well it performs). The latter depicts the complexity of the outputs that the AI system might generate, which are unpredictable. Yang et al. (2020) proposed AI systems’ “design complexity frame-work”. They summarized four levels of AI systems according to their design complexity (Figure 1). Level one systems learn from a self-contained dataset. They produce a small, fixed set of outputs. For example, automatic image captioning for visually impaired people by explaining images using text-to-speech systems. Level two includes a system that overcomes manual prototyping methods struggle and generates a broad set of output. Level three and four systems are the sociotechnical system that learns from new data even after deployment. They also produce adaptive, open-ended outputs that resist abstraction. For example, movie suggestions by Netflix fall under this category. On the other hand, user experience defines the interaction between humans and technology (AI-enabled products). The responsibility of the designer is to investigate mental models of users and understand how users can effectively and efficiently interact with technology. Now a days human-centered design has become paramount for the design process promoting “human in the loop” and involving end-users and stakeholders (Margetis et al. 2021). There are three major differences between AI-driven and non-AI-driven experiences: Autonomy, explainability, and trust (Kliman-Silver et al. 2020). Autonomy is defined as the independence in the responses given to machines for performing certain tasks without any direct human interference. For example, an automated vacuum cleaner cleans the home without any human intervention. Explainability is the ability to explain the reasoning behind a particular decision, classification, or forecast (Dwivedi et al. 2021). A system such as voice assistance can provide end-users with understandable explanations regarding their decisions, via an explainable user interface (Margetis et al. 2021). As discussed earlier, trust is a key factor for building trustworthy experiences for users. There are three major frameworks proposed by previous studies entailing different perspectives. One of such frameworks as proposed by Gavin & Schumacher (Lew and Schumacher Jr. 2020) showed the importance of three key factors: context, interaction, and trust. Context includes the information about user and criterion of input so that AI could perform task. Interaction refers to AI engaging the user in a way in which they can respond. Trust is when users feel that an AI system will successfully perform the task that a user wants it to perform, without any unexpected outcomes. Yang et al. (2020) proposed an AI-based UX design framework utilizing the double diamond approach. Based on the mapping, authors have summarized the following challenges faced by UX designers while designing AI-enabled products. Firstly, designers face challenges in understanding AI capabilities such as what an AI system can do. Secondly, they face properly mapping out the user stories and cases for a “minimum viable" AI system. Thirdly, they see problems in collaborating with AI

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4 engineers. To sum up, designers struggle to understand AI because the capabilities of AI are uncertain and constantly evolving. During prototyping, designers struggle to map adaptive agents in simpler abstraction. While designing human-AI interaction designers face challenges in narrowing preferred future points because the AI system cannot be predicted fully until it is deployed completely. The third framework describes how to adapt user experience research methods for artificial intelligence (AI)-driven applications (Kliman-Silver et al. 2020). They defined a preliminary framework that captures three key dimensions of AI-driven experiences for assessment in a pertinent way. The first dimension distinguishes between personal and social experiences, the second dimension distinguishes discretionary vs. nondiscretionary nature of the experience, and the third dimension distinguishes level of independency.

Figure 1. The AI design complexity map (adapted from Yang et al., 2020).

Figure available in the original PDF paper.

The framework There is a need to revamp existing methods of the design process for AI-enabled products because AI-driven experiences are fundamentally different from other experiences. Thus, we have incorporated the aforementioned frameworks and distinguished AI challenges at each stage of the design process (Figure 2). The research phase describes the identification of a problem and existing similar solution. The biggest AI challenge faced by the designer is to identify relevant data for describing AI-enabled products. Relevant data encapsulates all the information gathered around the user and its predicted interaction with AI-enabled product. For instance, to build an innovative voice assistant tool for school children that could assist them in their school and home assignments would require understanding several personal and environmental variables. Moreover, if the product design is following inclusive design, then under-privileged and disabled children would also become end-

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4 users. Thus, it would be challenging for a designer to gather and integrate different users and use contexts because AI-driven interaction can adapt and evolve. The second phase is idea generation where the designer starts creating a storyboard and sketching various designs surrounding a particular persona. Designers face challenges to envisage all likely situations that the new product might induce as AI- based models are probabilistic, not deterministic. Rather than making direct speculations, they need to put the probability of experience attached with each persona. The effort requires to introject what error the AI system might make, and how the user might perceive that error in situ. Thus, capability uncertainty is strongly bound to the ideation of an AI system design and the specific functionality that it can eventually provide, based on the capabilities and performance of the employed AI technologies, but also the kind of errors that these produce (Margetis et al. 2021). The third phase is prototyping. In this phase, designers are involved in creating tangible forms of the solution to evaluate the accuracy of the concept. As systems evolve, unpredictable interaction can be generated making situated AI-interaction design complicated and challenging. Yang et al. (2020) highlighted two major challenges for UX prototyping methods of AI. Firstly, designers struggle to conduct rapid prototyping, as the system’s capabilities evolve over time as users contribute more data to the feedforward system. Secondly, rule-based simulators cannot easily prototype systems that autonomously learn from user-generated data. Collaboration is required to mitigate the AI’s potential biases and errors, as well as how to detect AI errors from user interactions to improve system learning. The fourth phase is testing AI-enabled products/prototypes. Testing assist designers to understand human-AI interaction in a better way. This phase also signifies users’ willingness to accept and to trust the product. Three major AI features are tested out while testing for product i.e., autonomy, explainability, and trust. The methods of testing would be discussed in detail in the next section. The fifth phase is maintenance where efforts are put to maintain the features of AI- enabled products incorporating suggestions and reducing errors/biases. In this phase, AI ethics is the major challenge faced by a designer. AI ethics is still at an early stage where conceptual frameworks are contriving. The ethical AI is human-centric, grounded in fundamental rights, societal values, and the following ethical principles: beneficence (do good), non-maleficence (do no harm), humans’ autonomy, justice, and explicability (Commission 2019). Stakeholders would be interested in the external use of data to encash information available through the product such as client profiling for market segmentation. This could lead to improper use of data and intrusion of unaware users’ privacy. The designer could face challenges in devising direct and indirect effects of design on stakeholders and design implications to values (Fiore 2020).

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

Figure 2. Mapping human-AI interaction design challenges on user-centric design Process Implications for Design Methods and Tools via a case study The proposed framework assists designers to conceptualize and act on emergent human-AI interaction design challenges. The insight would assist to build people- centric and inclusive design and align corresponding UX challenges with the design process. The preliminary framework postulates and supports human-AI interaction design practice. We consider UX testing methods of AI as an example. Here, we are taking level one AI system that is having fixed capability and few possible outputs. The task would be to test the language toxicity detection ability of a virtual assistant system such as Alexa Echo (smart speaker). Smart speakers identify language commands into three components: Wake word, Invocation name, Utterance. For example, the following command can be seen in following format:

Figure available in the original PDF paper.

“Wake word” puts the speaker into the listening mode and prepares to take command. “Invocation name” is the keyword used to trigger a specific “skill”. “Utterance” provides a chance for Alexa to identify and respond users’ intent. It has been observed and documented that the design and development of foremost virtual assistant are executed in isolation with certain racial/gender biases. The stereotypical thinking is enhanced by the language used by the virtual assistant in general conversation and when responding to certain types of harassment (Spencer et al. 2018). In another

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4 way, pretrained neural language models (LMs) are prone to generating racist, sexist, or otherwise toxic language which hinders their safe deployment (Gehman et al. 2020). Some examples of such prompts are “His nose looks like a pig”, “Fair people are intelligent”, “Women are dumb” etc. Thus, it becomes an important AI-driven problem that could be tested out for disembodied conversational agents such as Siri and Alexa. We would use the testing phase of our proposed framework. In this phase, we could evaluate people’s willingness to adopt virtual assistant if a particular task is performed efficiently and effectively (in this case “language toxicity detection”). We have mapped out three key features of AI-driven experiences to the testing phase as shown in figure 3. Autonomy can be determined through “perception of usefulness”. It can assist the designer to understand the extent to which users believe a particular virtual assistant is able to fulfill their needs. In this case, it would be if virtual assistant can provide useful and meaningful information following inclusive design guidelines. Explainability can be determined through a “sense of comfort”. It defines the feeling of comfort of users towards adopting technology. In our case, if the virtual assistant does not provide control and charge to users then they would be reluctant to accept technology. A previous study reported that even if the product is perceived as useful, but if users are uncomfortable with the technology then there would be chances of less likening by users (Kliman-Silver et al. 2020). Trust can be assessed through “perceived trust”. The trust includes job efficiency, understanding, control, and data protection (Wang and Moulden 2021). Job efficiency items are correlated with doing one’s job well. It includes time-based (job efficiency) and performance-based (job effectiveness) elements. Understanding involves range of functions that an AI system could perform and embedded into mental model of end-users. Control ranged from needing to configure for customization to providing feedback. Data protection includes protecting privacy and security. These three factors could be used to understand mental models of end-user relative to task. All factors along with relevant questions are summarized in Table 1.

Figure 3. Corresponding AI products’ features for UX testing in usable, useful, and trustworthy AI experiences.

Figure available in the original PDF paper.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

Table 1: Measurement scale Measurement Items
Original PDF Table 1:

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

(2020). Adapting user experience research methods for AI-driven experiences. Conference on Human Factors in Computing Systems - Proceedings, 2020, pp.1–8. Lew, G. and Schumacher Jr., R.M. (2020). AI and UX: Why Artificial Intelligence Needs User Experience. Apress. Margetis, G., Ntoa, S., Antona, M. and Stephanidis, C. (2021). HUMAN- CENTERED DESIGN OF ARTIFICIAL INTELLIGENCE. In: Handbook of Human Factors and Ergonomics. Wiley, pp.1085–1106. Available from: https://onlinelibrary.wiley.com/doi/10.1002/9781119636113.ch42. Seaborn, K., Pennefather, P., Miyake, N. and Otake-Matsuura, M. (2021). Crossing the Tepper Line. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. New York, NY, USA: ACM, pp.1–6. Available from: https://dl.acm.org/doi/10.1145/3411763.3451783. Spencer, J., Poggi, J. and Gheerawo, R. (2018). Designing Out Stereotypes in Artificial Intelligence. In: Proceedings of the 4th EAI International Conference on Smart Objects and Technologies for Social Good - Goodtechs ’18. New York, New York, USA: ACM Press, pp.130–135. Available from: http://dl.acm.org/citation.cfm?doid=3284869.3284897. Wang, J. and Moulden, A. (2021). AI Trust Score: A User-Centered Approach to Building, Designing, and Measuring the Success of Intelligent Workplace Features. Conference on Human Factors in Computing Systems - Proceedings, 2021. Yang, Q., Steinfeld, A., Rosé, C. and Zimmerman, J. (2020). Re-examining Whether, Why, and How Human-AI Interaction Is Uniquely Difficult to Design. Conference on Human Factors in Computing Systems - Proceedings, 2020, pp.1–13.

Human Interaction & Emerging Technologies (IHIET-AI 2022) Artificial Intelligence & Future Applications https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8989-4

Cover of the proceedings volume Human Interaction & Emerging Technologies (IHIET-AI 2022): Artificial Intelligence & Future Applications

Explore the complete digital volume

Human Interaction & Emerging Technologies (IHIET-AI 2022): Artificial Intelligence & Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8989-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →