Yanick Trindade, Francisco Rebelo, Paulo Noriega (2022). Measuring emotion, interaction, and cultural outcomes after a VR game: The case of São Tomé. In: Ergonomics In Design. AHFE Open Access, vol. 47, pp. 439–447. https://doi.org/10.54941/ahfe1001968
The development of strategies based on the preservation, innovation, and dissemination of cultural aspects of the São Tomé and Príncipe (STP), represents an essential condition for the improvement and dynamization of the culture and creativity sector. Nowadays, due to low cost and technological advances, virtual Reality (VR) devices can provide an immersive experience to share cultural aspects. The goal of this study is to analyze (1) if the memory of the cultural and natural aspects remains after the end of VR experience, (2) the emotional reaction after the experience, and (3) interaction problems reported by participants. The number of participants in this study was 10. To measure what remains related to cultural and natural aspects of STP, the Think-aloud method was used to collect information. This protocol was applied after the participant finished the VR experience (retrospective think-aloud). This allowed knowing what participants value at a specific moment. Questions related to cultural aspects were directly asked to participants through a Questionnaire with questions about paintings/artists and creole language (QPCL) After three weeks, the QPCL was applied again, to verify if the cultural/natural aspect was recalled after this period. We also apply the Self-Assessment Manikin (SAM) tool to analyze the emotional reaction. Regarding the results using think-aloud, we structure the answers into 4 categories: (1) Interaction in VR; (2) Culture; (3) Nature, and (4) The desire to be there. Concerning Interaction in VR, the majority do not report any interaction problems (7 participants). Regarding the cultural and natural aspects, the natural aspect related to the green environment was the most reported aspect (5 participants). The desire to be there, was reported by 4 participants. We did not ask if they want to visit São Tomé or any other aspect categorized (think-aloud), they just verbalized what they were thinking at the time (after finishing the VR experience). The data collected using SAM reveals that the pleasure was positive, and arousal was considerable (Pleasure Average = 7.4; Pleasure Standard Deviation = 1.3 / Arousal average = 6.8; Arousal Standard Deviation = 2.1). Three weeks later, we applied the SAM and QPCL questionnaire again, and the results were still close to the values initially reported. In conclusion, the think-aloud and QPCL methods were helpful tools to understand what participants most value during a VR game experience (related to cultural, natural, and interaction aspects), and the results related to emotional reaction (SAM questionnaire), show that the experience provided a high level of pleasure.
The development of communication and interaction strategies that promote cultural fruition, promoting intercultural understanding and an ethic of global citizenship (United Nations, 2015), can enhance empathy between cultures. The Virtual Reality (VR) device can immerse users in environments (Murray, 1997; Ijsselsteijn, 2004; Zahorik, 1998) and situations they have never been in, and this can promote empathy. VR technology can be a promi- sing tool to promote tourism (Reis et al. 2016) and cultural heritage. The use of VR for cultural promotion purposes can be a way to promote STP culture. The basis of the culture of STP, is linked to the reciprocal acculturation betw- een Europeans from Portugal and Africans from south of the Sahara (Seibert, 2002), as well as the environment and their adaptation to it. These influences (between Africans and Europeans, but also between Africans from different origins and cultures), gave rise to the emergence of different languages and cultural aspects such as clothing, handicrafts, theatre, dance, music, gastro- nomy, rituals, housing, and property among other identity traits. The islands of STP are characterized by great linguistic diversity (Hagemeijer, 2009). In São Tomé Island (the bigger), in addition to Portuguese (official language), there are two predominant autochthonous languages, Forro (Hagemeijer, 2009) and Anguené (Hagemeijer, 2009). To propose an interactive experience that promotes the cultural aspects mentioned above, we use the Human- Centered Design approach, applying human factors and usability techniques and knowledge, focusing on users, their needs, and requirements (ISO 9241- 210, 2010; ISO 13407, 1999), and the UCD4VR framework (Trindade et al. 2021) to accomplish this purpose. A key component in this approach, is the User Experience (UX). Knowing users is fundamental to creating a good expe- rience. UX focuses on understanding the use or anticipated use of a product, system, or service through user perception and response (ISO 13407, 1999). Another key component is the user engagement. In the relationship betw- een user and resource, user engagement is the existing emotional, cognitive, and behavioral link, at a time or over time (Atffield et al. 2011). To col- lect data regarding the user experience, the think-aloud method (Leow and Morgan-Short, 2004), and SAM questionnaire (Bradley and Lang, 1994), can be helpful to access rich and varied data.
Goals In this study, the goals are to analyze if and which (1) cultural and natural aspects inside the VR game experience (related to STP) remain after the end of it, (2) What kind of emotional reaction are notified, and (3) interaction issue reported by participants during the VR game experience.
METHOD
Participants Ten participants (10) took part in this study, 9 Portuguese and 1 Spanish speakers. The age group was between 20-24 years old (6), 25-28 years old (2) and over 32 years old (2). Of these, 6 were male and 4 were female. None of
Measuring Emotion, Interaction, and Cultural Outcomes After a VR Game 441
Figure 1: Sequence of VR game experience moments.
then traveled to STP, and just one knows more about the country due to the fact she previously did research works about STP in architectural fields.
Game Apparatus In this study, a 3D virtual environment was created. In this environment, participants interacted with the VR game experience SOIA. All 3D game objects were created in Autodesk Maya®. The UI, some images, and texture were created in Inkscape®. All this visual content was later attached to the UNITY® game engine. To perform inside this VR world, participants used the VR device HTC VIVE® (with 1 Head-Mounted Display (HMD), 2 con- trollers, and 2 movement sensors). The virtual experience has 5 moments: (1) Naming and Logos; (2) Tutorial; (3) Hostel entrance, (4) Art hallway, and the (5) Bedroom. The interaction inside this VR environment was made using: (1) VR tools from ergoUX Lab, (2) VRTK tool (by Sysdia Solutions Ltd), and Steam VR®. An ASUS® computer (Intel® Core™i7 processors, 16 GB RAM, and NVIDIA® GeForce® GTX 1080 graphics) was used to run the VR game experience.
VR Environment -Training room: After logos, the tutorial room is revealed (fade off). In this training room, the player must: (1) Move the head and rotate the body to see the virtual environment around; (2) Move in the virtual world using telepor- ting, know areas which it can move or not using the controller, as well avoid
colliding with objects and walls in the real world; (3) Manipulate objects in the virtual world using the controller.
-Portal: After completing the tutorial (but still in the training room), the user must unlock the portal that will allow him to go to the entrance of a hostel located in São Tomé Island. To accomplish that, he must say the keywords. But the keywords will have to be said in the STP Creole language (creole forro). When the user says the keywords correctly, the portal will open, and he/she is teleported to the hotel entrance.
-Hotel entrance: At the hotel entrance, the user Will receive information that helps to contextualize, as well the challenge he/she will have to ove- rcome is revealed. But to overcome it, he will have to complete tasks and sub-challenges.
-Art Hallway: Inside the hostel, the first task will be to photograph one of the four paintings that are in the art hallway. After that, he/she needs to find the game map. For this, the user will have to go to the suite room he/she will be staying in.
-Suite room: In the bedroom, the user needs to find the golden key to open the balcony door. In this balcony area, he will find the game map. With the map in his/her hands, the user will see that the next task he needs to overcome, is to find a vinyl music disc and put the disc on the turntable. But before he gets close to the turntable, a message appears, saying that this is a demo version, and the experience ends.
Measures To assess and analyze user experience, the following tools were applied: Think-aloud (retrospective): Retrospective verbalization is frequently used immediately after some tasks take place. It can be in specific break moments, or immediately after the end of the entire task (Leow and Morgan-Short, 2004). A questionnaire with questions about paintings/artist and creole lan- guage (QPCL). Additionally, we use the tool Self-Assessment Manikin (SAM): A non-verbal pictorial assessment technique to measure pleasure and arousal (Bradley and Lang, 1994).
Procedure All participants filled out a consent form and were informed of the goals of the study. The experiment was performed in a room using a desktop com- puter and VR device. Each participant completes the VR gaming experience in approximately 25 minutes. The Think-aloud protocol and QPCL were applied first simultaneously (after the end of the VR game), and the SAM was applied in another room through an online form (google form). Three weeks later, we applied the QPCL again, to see if they still remember.
RESULTS AND DISCUSSION
Each participant took approximately 7 minutes to report their experience through the think-aloud method and the QPPCL. To organize the data colle- cted using the think-aloud method, we structure the answers in 3 categories: (1) Interaction in VR; (2) Culture; (3) Nature, and (4) The desire to be there.
Measuring Emotion, Interaction, and Cultural Outcomes After a VR Game 443
Interaction in VR refers to the fact that the participant reports some intera- ction problems or if he/she faces no problem at all. Culture and Nature are related to the desire to know or visit São Tomé. Culture refers to the fact that the participant verbalizes/values, or not, aspects related to local architecture (exterior and interior), colors, and memory about writers, painters and cre- ole language. Nature refers to the fact that the participant verbalizes/values, or not, aspects related to the natural aspect like the beauty of green nature, natural lighting, and the beach.
-Interaction on VR: Three participants reported problems when intera- cting in the virtual environment. One mentioned that he had focus problems because he used glasses and this issue affected the experience (could not see clearly). The same participant refers also that he faced some problems with picking objects. Another mentioned that the teleport could have a little transi- tion because the first time it seems a little abrupt. Also, one participant refers that she forgot some command controls later.
-Culture: At the hotel entrance, four participants mentioned something related to the cultural aspects. Three of them reported about the architectu- ral aspects (that think is specific to the place because it is something that they have never seen before, and one said that the houses at the entrance of the hotel reminded her of São Tomé and, in particular, of the houses on the cocoa and coffee plantations, as in the past she had done academic work (archite- cture) about houses inside the plantations (in São Tomé). In the suite room, three reported that they liked the decoration style, materials, paintings on the wall, and African vibrant colors (curtains, bed sheets, etc.). Also, we found that three participants (all female) reported that they saw a little book on the table. This is a little handbook of a Santomean writer.
-Nature: Regarding the natural environment, at the entrance of the hotel, four participants reported that the luxuriant green environment was very beautiful, gave a relaxing feeling. Of them, two mentioned that it gave the feeling as they were on vacation, and one said that she had never seen so much green around her, a place with unique characteristics, and another one said that she saw the sea far away. In the art hallway, one participant mentioned that the sun and the trees outside were very inviting. In the suite room, three participants verbalized that the sea view from the balcony was stunning.
-Desire to be there: At the entrance of the hotel, one participant mentioned that he wanted to be there on vacation. In the art hallway, one participant mentioned that she saw the lounger on the outside, she really felt like sitting and resting there. In the suite room, two mentioned that the place gave a relaxing feeling, and a desire to be there. One of them reported that he did not know the place, but this experience gave him the desire to be there.
Using the QPCL (see Figure 2), we found that: Inside the virtual environ- ment (tutorial room), three participants correctly verbalized the keyword in creole forro language “Désa-mo lentlá” (that means “let me in”, in English), and the portal opened. However, no one remembers the keywords in creole forro when we asked after the end of the VR experience. We also asked if they remembered the meaning of the phrase in Portuguese, and five answered cor- rectly. Inside the hotel, in the art hallway, when asked about the paintings within this hallway, five answered correctly (choose 4 paintings from a list
Figure 2: QPCL: First time.
of 6 options, two of which were not in this hallway). We also asked if they remember the painter’s name from the painting they’ve chosen as a favorite (take a picture, and the painter’s information would appear in a hologram), and five remember and five don’t.
Globally, 4 participants only describe the actions/order of events, while the other 6 participants value more the cultural and natural aspects inside the VR environment. Of these, 4 verbalize the desire to go to the place (physically) after the VR experience.
Regarding the data collected using SAM (see Figure 3), and on a scale from 1 to 9, the most scored value for pleasure, was 7 (Pleasure Average = 7.4, Ple- asure Standard Deviation = 1.3). As for arousal, the results show that their moods are closer to excited than calm, however, we found more dispersion of results compared to pleasure (Arousal average = 6.8; Arousal Standard Deviation = 2.1). As we found in previous research (Trindade et al. 2018), sometimes, participants face some struggles trying to establish the differe- nce between calm and excited, because some of them feel calm but at the same time very excited. Being in a VR environment for the first-time trig- ger’s arousal. Also, the landscapes and environments with elements of nature give a “calming feeling”, so this ambiguity can be related to this. After three weeks, we apply the two methods previously used again (see Figure 3). What we try to analyze, was if the participants still remembered some cultural aspe- cts of the VR game experience and if the results related to SAM are still the same or close. Regarding cultural aspects, in particular about the keywords to open the portal (in creole forro language), 83% of participants answered correctly. But this time, we gave three options (2 wrong and 1 correctly). The fact that they don’t have the confidence to try speaking in another langu- age for the first time, may justify the fact that no one even tried to answer the first time. Also having options make the process easy once in terms of memory evocation, recognition is easy than recall (Eysenck & Keane, 2010). One thing that can be added to help what remain, is the fact that after the end of the VR experience, we talk with participants, explaining that the phrase in Creole forro (to open the portal) had some similarities with Portuguese, as Creole forro is a language that has influences from Portuguese and from the African languages of former slaves (Hagemeijer, 2009) (mainly in the period
Measuring Emotion, Interaction, and Cultural Outcomes After a VR Game 445
Figure 3: QPCL and SAM: Three weeks later.
of the first settlement of the São Tomé Island). We also asked again about the painting that was in the art hallway. The majority still remember, despite a little decrease. Regarding pleasure and arousal, the results are still close to the previous one (Pleasure Average = 7.5; Arousal average = 7.0), and with less dispersion.
CONCLUSION
Being part of research work to develop communication and interaction stra- tegies to promote the cultural and natural heritage of STP, the present study sought to analyze the participant feedback after the VR game experience. Concerning the goal, the results were rich and varied. The retrospective verbalization of participants after the VR game experience through the think- aloud method reveals that some participants only describe the actions and the order of events (4 participants), others value more the cultural and natural
aspects inside VR environment (6 participants) and verbalize the desire to go to the place after the VR experience (4 participants/the same ones who value the cultural and natural aspects). However, it is necessary to emphasize that we did not ask the participant if they want to visit São Tomé after the VR game experience, they just verbalized what they were thinking at the time. Another point that we highlight, is the fact that only three participants mana- ged to open the portal with the keywords in creole language correctly inside the VR environment. The fact that is a phrase and not just one word, and the majority don’t have the confidence to speak in another language completely different (shame to say something wrong) may justify these results. Maybe starting with just one word, and then increasing the difficulty to a phrase, can be a better approach in the next studies. Also, if the user can have some- thing that he/she can have to remind or read and see the word or phrase as many times as he/she wants (like a paper note that he/she can access clicking in a button) can be useful. About SAM data, the results regarding the plea- sure were high (pleasure average = 7.4 for the first time, and 7.5 three weeks later), and arousal was considerable (arousal average = 6.8 for the first time, and 7.0 three weeks later). In short, a greater number of participants will allow us to have a greater variety of personalities, that is, people who are naturally more extroverted will be able to report more details of the experie- nce, while introverts will be more succinct in their description/verbalization. And perhaps it is pertinent to differentiate right from the beginning as a way of characterizing the participant and comparing the answers and seeing the report of the two groups/personalities.
ACKNOWLEDGMENT
This work is financed by national funds through FCT - Fundação para a Ciência e a Tecnologia, I.P., under the Strategic Project with the reference UIDB/04008/2020 and ITI -LARSyS, (Project - UIDB/50009/2020).
REFERENCES
Atffield, Simon. Kazay, Gabriella. Lama, Mounia and Piwowarski, Benjamin. (2011).
Towards a science of user engagement. Bradley, Margaret and Lang, Peter. (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Therapy Exp. Psychiatry 25. Eysenck, M. W., & Keane, M. T. (2010). Cognitive Psychology: A Student’s
Handbook. 6th. N. York: Psychology Press Ltd. Hagemeijer, Tjerk. (2009) As línguas de S. Tomé e Príncipe. Revista de Crioulos de Base Lexical Portuguesa e Espanhola, Vol. 1.1. Lisboa: ACBLPE - Faculdade de Letras da Universidade de Lisboa. ISO 13407 (1999). Human-Centred design processes for interactive systems. ISO 9241-210 (2010). Ergonomics of human–system interaction — Part 210:
Human-Centred de-sign for interactive systems. Ijsselsteijn, Wijnand (2004). Presence in Depth: Technische Universiteit Eindhoven. Leow, Ronald and Morgan-Short, Kara. (2004) “TO THINK ALOUD OR NOT TO THINK ALOUD: The Issue of Reactivity”, in SLA Research Methodology. Studies in Second Language Acquisition, 26 (1), pp 35–57.
Measuring Emotion, Interaction, and Cultural Outcomes After a VR Game 447
Murray, Janet. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace. The MIT Press, Cambridge. Reis, Lara., Duarte, Emília., Rebelo, Francisco. (2016) “Older Workers and Vir-
tual Environments: Usability Evaluation of a Prototype for Safety Sign Research”. Soares, M. (Ed.), Rebelo, F. (Ed.). Ergonomics in Design. Boca Raton: CRC Press, 279–295. Seibert, Gerhard. (2002). Camaradas, Clientes e Compadres: Colonialismo, Socialismo e Democratização em São Tomé e Príncipe. 2nd edn. Vega Editora, Lisboa. Trindade, Yanick. Rebelo Francisco and Noriega Paulo. (2021) “Research Games:
A Model to Support the Development of Educational Game Using Virtual Reality Platforms”. In: Rebelo F. (eds) Advances in Ergonomics in Design. AHFE 2021. Lecture Notes in Networks and Systems, vol 261. Springer, Cham. Trindade, Yanick. Rebelo Francisco and Noriega Paulo. (2018) “Tourism and Virtual Reality: User Experience Evalua-tion of a Virtual Environment Prototype”. In: Marcus A., Wang W. (eds) Design, User Expe-rience, and Usability: Users, Contexts and Case Studies. DUXU 2018. Lecture Notes in Computer Science, vol 10920. Springer, Cham. United Nations. (2015). General Assembly, Transforming our world: the 2030
Agenda for Sustainable Development, A/70/L.1. Zahorik, Pavel and Jenison, Rick. (1998). Presence as being-in-the-world. Presence: Teleoperators and Virtual Environments.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Ergonomics In Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-23-0
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional