Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Creating emotionally resonant User Experiences in response to ever-evolving challenges in Open Distance e-Learning Institutions.

University of South Africa, Pretoria, South Africa ↗
Publication year
Digital Object Identifier10.54941/ahfe1005626

Abstract

Advances in ever-present, social, mobile, and physical computing technologies have moved the field of Human Computer Interaction (HCI), particularly User Experience (UX), into practically all areas of human activity, including education. Higher education institutions (HEIs) face a growing need to enhance online learning opportunities that address an increasingly diverse learner community. This underscores the necessity of UX communities to propose that the development and evaluation of digital technology should not only include usability, but the broader range of user experiences, where users’ feelings, motivations, and values are given as much, if not more, attention than efficiency, effectiveness, and basic subjective satisfaction experiences which it offers. Emotion and the expression of emotion play a powerful role in the way human social interaction is shaped, and therefore can and must be exploited in HCI. An emotion lexicon is essential for describing emotions and emotion-aware experiences, facilitating the identification of emotions, and describing subsequent behavioural patterns which highlights the importance of a match between an emotion experienced, experience design and design principles. To explore the role of emotion and the possibilities it has for technology interaction, a non-formal learning initiative was implemented in an Open Distance e-Learning (ODeL) environment using a freely available social media app exploiting its capacity for inclusiveness and educational potential. Using phenomenology as a research philosophy and a methodology, emotions were identified and mapped to behavioural patterns resulting in design principles. This allowed for the creation of emotion-aware experiences and a more comprehensive understanding of mobile tutoring in a ODeL environment.

INTRODUCTION

Higher education institutions (HEIs) face a growing need to enhance online learning opportunities due to resource constraints, technological advancements, demographic changes, and demands for accessibility and equity. The increasingly diverse learner community, spanning socioeconomics, culture, ethnicity, experience level, and motivation, underscores the necessity of UX communities to propose that the

© 2024. Published by AHFE Open Access. All rights reserved. 602

Creating Emotionally Resonant User Experiences in Response 603

development and evaluation of digital technology should not only include usability, but the broader range of user experiences, where users’ feelings, motivations, and values are given as much, if not more, attention than efficiency, effectiveness, and basic subjective satisfaction experiences which it offers.

User Experience (UX), a complex and dynamic field is determined by many factors and is positioned within the greater Human Computer Interaction (HCI) framework. The advances in mobile, universal, social, and tangible computing technologies have moved HCI into practically all areas of human activity. The consequence of this is that evaluation of digital technology should not only include usability, but also the broader range of user experience, where users’ emotions, motivations, and values are given as much, if not more, attention than efficiency, effectiveness, and basic subjective satisfaction experiences which it offers. UX is the outcome of an interaction between a user and application. The role and impact of a user’s goals, expectations, needs, traits, and values on UX must be considered, the application determines and shapes the user’s experience as the user reacts to the application through their senses and the interaction defined as an action taken by users’ using an application that influences or modifies their mental and physical state. Rosalind Picard (1995, 1997) was one of the first to document the importance of emotions in human-computer interactions in her research on affective computing. Since then, research in this field has gained significant momentum as researchers have worked to understand the subtleties of emotion and its effect on our behaviours (Baveye et al., 2018; Maia and Furtado, 2018; Lambie, 2020). This research proposes a framework to be used to identify emotions experienced by a user during the interaction with a mobile tutoring application (app). Identifying emotions and mapping them to behavioral patterns contribute to a better understanding of the user, the interaction and the application which can be used to identify emotional experiences in the broader UX framework.

The Human Emotional Experience in HCI Emotions arise as reactions to important life events and reflect experiences and actions in our social lives. It is important to define emotion, in contrast to describing it, for two reasons. Firstly, it is to identify the research object; and secondly it is to make the insights obtained comparable among different disciplines (Scherer, 2005; Vornewald, Eckhardt and Krönung, 2015). As there is no consensus on how to understand emotions and therefore what a definition of emotion must include, it impacts interdisciplinary discourse and scientific co-operation. The scope of existing definitions ranges from “abstract working definitions” (Vornewald, Eckhardt and Krönung, 2015, p. 1758) to “more complex approaches”such as Scherer’s (2005) Component Process Model in which the definition is part of a theory. Sherer (2005, p. 697) defines emotion as “an episode of interrelated, synchronised changes” in a person that occur “in response to the evaluation of external or internal stimuli or events relevant to major concerns” of the person. According to Shiota and Kalat (2012) the most fundamental question to ask about emotions

604 le Roux and van Staden

concerns the relationship between the stimuli and how a person relates to events in the environment. Emotions are also known to motivate and guide an individual’s attention as well as perception (Izard, 1977). In the field of psychology, authors agree that individuals engage their emotion systems in all thoughts and actions (Jeon, 2017). This research aims to provide new ways to visualise the basic dimensions of emotion in the context of UX, allowing for the creation of a framework that can assist in the understanding of UX in mobile teaching and learning. The identification of human emotional states is difficult and complex (Picard, 2010). Therefore, to gain a better understanding of the subject, theories and models describing how emotions are experienced can be found in literature and help to convey a specific feature of human emotion and suggest approaches as to how emotions are presented and interpreted by the human mind (Sreeja and Mahalakshumi, 2017).

Early theories of emotions that have ongoing impact as a historical context focus on the physiological component of emotions and are based on an intuitive understanding of the processes involved in emotions experienced. Discreet emotions theories are described as basic (discreet) emotions and are based on the “evolutionary origin of basic emotions” found in all cultures (Ortony and Turner, 1990, p. 317). The number of these basic emotions suggested by the psychologists, range between two and eighteen and include inter alia happiness, sadness, interest, hope, frustration, boredom, and wonder. Dimensional or secondary emotion theories are based on “irreducible constituents of other emotions” (Ortony and Turner, 1990, p. 317).

Cognitive Aspects of Emotion The central construct in a cognitive understanding of emotion is appraisal. Appraisal theories claim that emotions are elicited by evaluations (appraisals) of events and situations. The central belief shared by appraisal emotion scholars is that, without an antecedent cognitive appraisal of the event, emotions do not occur and that the appraisal, not the event itself, causes the emotion. Emotion is also a process and if the appraisal changes, even if the situation does not, the emotion will change.

Magda Arnold’s (1960, 1970) Appraisal theory assumes that an emotion is produced because of the interaction between an appraisal, brain activity and arousal. She proposed that an emotional sequence begins by an initial appraisal which arouses the appropriate physiological reactions and the emotional experience itself. Richard Lazarus expanded on Arnold’s Appraisal theory to develop a structural model that distinguish between primary (the meaning or significance of an event) and secondary (ability to cope with the consequences of the event) appraisals (Lazarus, 1991). The Component Process theory (Scherer, 2005) asserts that emotions are created due to the interaction between many different cognitive and physiological components. According to his multi-level sequential check model, Stimulus Evaluation Checks (SECs) motivate appraisal development and emotional expression. Desmet (2003) proposed a multi-layered model of product emotions presented by four main parameters in the eliciting process of

Creating Emotionally Resonant User Experiences in Response 605

emotions: 1) appraisal, 2) concern, 3) product, and 4) emotion. The first three, as well as the interactions between them, determine whether a product elicits an emotion, and if so, which emotion is evoked. Desmet’s appraisal types and Scherer’s SECs are summarised in Table 1.

Table 1. Comparison of Scherer’s SECs and Desmet’s product emotions.

Table available in the original PDF paper.

Scherer (SECs) Desmet Emotions Novelty check Surprise Intrinsic pleasantness check Aesthetic Goal/need significance check Instrumental Coping potential check Interest Norm/self-compatibility check Social

Conceptualised Framework Following the investigation of these cognitive appraisal theories and models, a framework to identify emotions experienced was proposed as depicted in Figure 1. The first component of the framework is that of concern, or in UX terms, usability. If the initial appraisal leads to the user experiencing a feeling of added value, it will evoke positive emotions which will motivate the user to continue. The second component, association, will allow the user to either enhance or diminish the initial feeling of value. If the emotion is enhanced, an association between the user and application will evolve and will lead to engagement of the user with the application and the evolving of positive emotions. The opposite is also true. If the initial appraisal leads to the user experiencing a feeling of little to no value, negative emotions will be evoked which will demotivate the user and the user will have a diminished will to continue with the application. The final component is that of interaction or engagement. After the association the user will or will not start to collaborate with the application.

Figure 1: Abstracted framework to identify an emotion.

Figure available in the original PDF paper.

Methodology and Brief Implementation Overview As the aim of this research was to identify and analyse emotions experienced during online mobile tutoring sessions, the philosophical worldview of interpretivism was adopted. Phenomenology as both a research philosophy and a methodology were employed. Examining the variables related to participants’ emotional experiences using mobile tutoring in the ODeL

606 le Roux and van Staden

context and the contribution of these variables towards addressing the social, emotional, and functional needs of the participants, demanded a research approach that addressed qualitative methods.

To explore the role of emotion and the possibilities it has for technology interaction, a non-formal learning opportunity was implemented in a ODeL institution. Fourth-level students served as tutors for first-level students in a programming course using a tutor-tutee driven mobile app. The mobile app was implemented using a freely available social media platform e.g. Facebook, WhatsApp, etc. Social media has shown a capacity for inclusiveness as it allows for the sharing of knowledge, dialogue and conversations addressing diversity and inclusion. Additionally, social media also shown advantages in terms of relevance, appropriateness, flexibility, and participation highlighting its educational potential. In terms of the quality management, social media was found to be responsive to the needs of the learning community due to participatory design followed before and during the tutoring process. This research uses the situated learning framework of a Community of Practice (COP) as a theoretical lens through which the social and emotional context of the participants could be understood highlighting shared interest in social media (domain), mobile learning needs (practice) and reliance on shared knowledge and support (community).

Data Collection Data was collected through recordkeeping journals kept by the tutors while engaging in mobile tutoring. The tutors were encouraged to describe each experience in terms of situations (who, what, when, and where), feelings (a single feeling, e.g., mad, frustrated, etc), and thoughts (a detailed description of the feeling). As English is not the first language for several of the participants, a list of adjectives was provided to help them to describe their experiences. Several emails were sent hence and forth between the researcher and participants to address and ensure continuity. The steps consisted of a planning and preparation phase, a pre-study brief, logging period and post-study follow up.

Data Analysis The data analysis process was advised by Braun and Clarke’s (2013) Thematic Analysis as well as the Phases of Thematic Content Analysis in ATLAS.ti by Friese, Soratto and Pires (2018). To get familiarised with the data, observational techniques by Ryan and Bernard’s (2003) were followed. The techniques included repetition, local terms, metaphors and analogies, natural shifts, similarities and differences and casual and conditional relations. The thematic content that emerged from this study provided potentially important information in relation to the development and implementation of mobile tutoring applications themes.

Creating Emotionally Resonant User Experiences in Response 607

Figure 2: Major themes and subthemes identified from the experience journal.
Original PDF Figure 2:

In ATLAS.ti, ‘groundedness’ and ‘density’ are code qualities that show the researcher how often they are used, and how they relate to one another and to other elements within an ATLAS.ti project. By looking at the groundedness and density factors, the researcher had certainty of identifying a relevant theme. The higher the groundedness and density values as determined by ATLAS.ti, the higher the impact of the theme. Figure 2 shows a summary of the four themes that emerged from the data and the sub-themes within each of those. Using the abstracted UX framework proposed earlier, where usability or concerns allow for associations and results in engagement or not, concerns have been identified based on theme rankings in groundedness and density. The top four concerns identified with the associated rankings were tutoring method (169), student participation (154), tutoring content (148) and tutoring sessions (119). For example, for the concern student participation, several positive and negative emotions were experienced by the tutor. Table 2 presents the results for the concern identified as student participation. The emotion experienced is presented in the final column.

Table 2. Positive emotions associated with the concern: ‘student participation’.

Table available in the original PDF paper.

# Concern Association Engagement Emotion

EXCITED

[P03] Student participation

upcoming sessions

student seemed devoted and optimistic

APPRECIATIVE

[P06] Student participation

participant had responded to participate

finally, the sessions could commerce

HAPPY

[P12] Student participation

agreeable to the times proposed

student seems eager to be part of the study

HOPEFUL

[P17] Student participation

Student’s curiosity

session is going to be exciting and full of positive energy

(Continued)

608 le Roux and van Staden

Table 2. Continued # Concern Association Engagement Emotion

Table available in the original PDF paper.

ENCOURAGED

[P18] Student participation

student seems to be very excited

DISCOURAGED

[P05] Student participation

student was not responding

I thought it was not going to work

ANXIOUS

[P06] Student participation

time was elapsing

not getting feedback from the student

NERVOUS

[P12] Student participation

student has not yet responded

noting that hours later

MISERABLE

[P21] Student participation

student was not responding

ANXIOUS

[P22] Student participation

still have not received any feedback

not sure if I should send a reminder

The 206 emotions experienced by the tutors was first categorised as positive and negative and then mapped to the eight basic emotions devised by Plutchik (1980). He proposed three languages to describe emotional states; in everyday language, descriptive language based on behavioural observation, or functional language based on the adaptive function of whatever is happening. For example, if a user experience joy, the resulting behavior will be processing and therefore want to explore more - reproduction. If a negative emotion is experienced, for example, anger, the user behaves in an attacking mindset, the resulting behaviour will be that of destruction and no engagement will result. This is depicted in Table 3.

Table 3. Emotion words according to Plutchik (1980).

Table available in the original PDF paper.

Behavioural Language

Functional Language

Emotions Number of times emotion was reported

Orientation

Positive Joy 92 Processing Reproduction Acceptance (trust) 23 Bonding Incorporation Anticipation 16 Examining Exploration Surprise 7 Stopping, Freezing Reintegration Negative Sadness 35 Crying for help

Protection

Fear 31 Withdrawing, escaping Anger 2 Attacking Destruction Disgust 0 Vomiting Rejection

Creating Emotionally Resonant User Experiences in Response 609

DISCUSSION

Participation in the mobile tutoring process required tutors to capture their immediate emotion first, after which they could elaborate on the event using their thoughts and feelings. Emotions were grouped as positive and negative and the use of the emotion and derived behavioural pattern allowed for a contribution towards a more comprehensive understanding of emotion- aware UX by identifying and analysing emotions Joy was reported with words like ‘confident, delighted, excited, happy, feeling good’ and the basic behavioural pattern associated with joy is reproductive learning. Acceptance was reported with words like ‘comfortable, pleased’ and is associated with incorporation. If acceptance or trust is experienced, the behavioural pattern of incorporation will ensue. Anticipation was reported with other adjectival words, such as ‘creative, enthusiastic hopeful, inspired, and optimistic’. Anticipation is also associated with exploration achieved though the presentation of content via a relatively new and stimulating way to work (social media). The emotion of surprise was experienced in the behavioural patterns of orientation through words like ‘amazed, impressed, and respected’. The presentation of a novel object, in this case the mobile tutoring app, required the participants to orientate themselves to a new way of teaching and learning.

Sadness expressed through words such as ‘unhappy, aggrieved and struggle’ was brought about by feelings of loss and isolation. According to Feidakis (2016) the design of online applications that are emotion-aware and allow for social inclusiveness will address these feelings of loss and isolation and will allow for the re-integration of the tutoring process. Fear was reported as an emotion with words like ‘anxious, confused and helpless’ and is stimulated by events that feel threatening and is associated with the behavioural pattern of protection. Any interference with the tutoring process, such as communication, connectivity or engagement triggered emotions of fear. Anger or being annoyed is associated with destruction. According to D’Mello et al. (2014) anger is found to be prominent during learning because this emotion is associated with confusion and the inability to resolve it (Taub et al., 2021). Although confusion can be beneficial in learning as it can also foster engagement that leads to deeper enquiry and understanding (Arguel et al., 2017). However, not being able to resolve any confusion will most likely lead to anger. Avoiding such negative learning experiences, therefore, is especially important in mobile learning environments as learner engagement cannot be closely monitored. There were no expressions of disgust which would have led to the rejection of the mobile tutoring app and the subsequent abandonment.

CONCLUSION

The evidence presented in this study shows the powerful role that emotion and emotion expression play in mobile apps as well as the amount of information conveyed through emotions during mobile sessions. The description of emotion-aware experiences is an important step towards experience design. Basic emotions and subsequent behavioural patterns

610 le Roux and van Staden

highlight the importance of a match between an emotion experienced and design principles. Using emotion-aware experiences gives the opportunity to integrate dispersed knowledge about experiences and find ways to integrate them. Furthermore, mobile tutoring using social media as a non-formal learning opportunity is an instructional methodology that is consistent with most teaching and learning goals in that it provides the opportunity for students in ODeL to become active participants in their own learning. Ultimately, this research contributes to a more comprehensive and nuanced understanding of the processes involved in developing and embedding inclusion and accessibility within instructional practices in ODeL institutions.

REFERENCES

Arguel, A. et al., (2017) ‘Inside Out: Detecting Learners’ Confusion to Improve Interactive Digital Learning Environments’, Journal of Educational Computing Research, 55(4), pp. 526–551. Available at: https://doi.org/10.1177/ 0735633116674732. Arnold, M. B. (1960) Emotions and Personality. New York: Colombia University Press. Arnold, M. B. (1970) ‘Perennial problems in the field of emotion’, in M. B. Arnold

(ed.) Feelings and Emotions. New York: Academic Press, pp. 169–185. Baveye, Y. et al., (2018) ‘Affective Video Content Analysis: A Multidisciplinary Insight’, IEEE Transactions on Affective Computing, 9(4), pp. 396–409. Available at: https://doi.org/10.1109/TAFFC.2017.2661284. Braun, V. and Clarke, V. (2013) Successful Qualitative Research: A Practical Guide

for Beginners. London: Sage. Desmet, P. M. A. (2003) ‘a Multilayered Model of Product Emotions’, The Design Journal, 6(2), pp. 1–13. D’Mello, S. K. et al. (2014) ‘Confusion can be beneficial for learning’, Learning and

Instruction, 29, pp. 153–170. Available at: https://doi.org/10.1016/j.learninstruc .2012.05.003. Feidakis, M. (2016) ‘A Review of Emotion-Aware Systems for e-Learning in Virtual Environments’, Formative Assessment, Learning Data Analytics and Gamification: In ICT Education, (March), pp. 217–242. Available at: https: //doi.org/10.1016/B978-0-12-803637-2.00011-7. Friese, S., Soratto, J. and Pires, D. (2018) Carrying out a computer-aided thematic

content analysis with ATLAS.ti, MMG Working Paper 18-02. 18. Göttingen, Germany. Izard, C. E. (1977) Human Emotions. New York: Plenum Press. Jeon, M. (2017) ‘Emotions and Affect in Human Factors and Human-Computer Interaction: Taxonomy, Theories, Approaches, and Methods’, in Emotions and Affect in Human Factors and Human-Computer Interaction. Elsevier Inc., pp. 3–26. Available at: https://doi.org/10.1016/B978-0-12-801851-4/00001-X. Lambie, J. A. (2020) ‘New Ideas in Psychology The demanding world of emotion:

A Gestalt approach to emotion experience’, New Ideas in Psychology, 56(March 2019), pp. 3–13. Available at: https://doi.org/https://doi.org/10.1016/j.newideap sych.2019.100751. Lazarus, R. S. (1991) Emotions and Adaption. New York: Oxford University Press.

Creating Emotionally Resonant User Experiences in Response 611

Maia, C. L. B. and Furtado, E. S. (2018) ‘Using psychophysiological measures to estimate dimensions of emotion in hedonic experiences’, Computers and Electrical Engineering, 71(July), pp. 431–439. Available at: https://doi.org/10.1016/j.comp eleceng.2018.07.048. Ortony, A. and Turner, T. (1990) ‘What’ s Basic About Basic Emotions?’, Psychological Review, 97(3), pp. 315–331. Picard, R. W. (1995) Affective Computing. Massachusetts: MIT Press Massachusetts. Picard, R. W. (1997) Affective Computing. Edited by MIT. Massachusetts. Picard, R. W. (2010) ‘Emotion research by the People, for the People’, Emotion Review, 2(3), pp. 250–254. Available at: https://doi.org/10.1177/ 1754073910364256. Plutchik, R. (1980) Emotion: A Psycho-evolutionary Synthesis. Ryan, G. W. and Bernard, H. R. (2003) ‘Techniques to Identify Themes’, Field

Methods, 15(1), pp. 85–109. Available at: https://doi.org/10.1177/1525822X 02239569. Scherer, K. R. (2005) ‘What are emotions? And how can they be measured?’, Social Science Information, 44(4), pp. 695–729. Available at: https://doi.org/10.1177/ 0539018405058216. Shiota, M. and Kalat, J. (2012) Emotion. 2nd edn. Wadsworth: Cengage Learning. Sreeja, P. and Mahalakshumi, G. (2017) ‘Emotion Models: A Review’, International

Journal of Control Theory and Applications, 10(8), pp. 651–657. Taub, M. et al., (2021) ‘How are students’ emotions related to the accuracy of cognitive and metacognitive processes during learning with an intelligent tutoring system?’, Learning and Instruction, 72(May 2019), p. 101200. Available at: https://doi.org/10.1016/j.learninstruc.2019.04.001. Vornewald, K., Eckhardt, A. and Krönung, J. (2015) ‘Emotions in Information

Systems Research – A Five Component View’, in Proceedings der 12. Internationalen Tagung Wirtschaftsinformatik (WI 2015). Osnabrückitaly cornona, pp. 1756–1770.

Cover of the proceedings volume Human Factors in Design, Engineering, and Computing

Explore the complete digital volume

Human Factors in Design, Engineering, and Computing

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-35-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →