Naoki Takahashi, Yuri Hamada, Hiroko Shoji (2022). Analysis of an actors’ emotions and audience's impression of facial expression. In: Affective and Pleasurable Design. AHFE Open Access, vol. 41, pp. 10–13. https://doi.org/10.54941/ahfe1001774
1. IntroductionNon-verbal information is very important in all communication. According to Mehrabian’s study, the facial has 55%, the vocal has 38% and the verbal has 8% of information people receive. Technologies of computer-mediated communication promoted communication that does not need face-to-face (e.g. e-mail). Recently, however, a video communication system that enables us to have conversations looking at each other’s faces is spreading. Understanding impressions conveyed by facial expressions is getting more important again.Our purpose is to examine whether a facial expression can convey his/her emotion to a person. In the experiment, subjects evaluated their impression of images of facial expression. We compared the evaluation with another evaluation by the actor who create the facial expressions.2. MethodsIn the experiment, we made images of the actor’s face corresponding to some emotional keyword and showed subjects as audiences. The actor is one female volunteer in her 20s. Audiences are fifteen male and fifteen female volunteers in their 20s. Fifteen audiences were acquainted with the actor and the rest of them looked at her for the first time in this experiment.Actors instructed to create facial expressions of eight emotions, “surprising,” “frustrating,” “exciting,” “guarding,” “relaxing,” “angry,” “fear,” and “boring.” Stimulus images were bust shots (photographs of the upper body than her bust) of the actor creating facial expression. After the photography, she was instructed to evaluate her own emotions in the images by two Likert scales of eleven points from unpleasure (0) to pleasure (10) and from deactivated (0) to activated (10). Similarly, audiences evaluated her emotion after looking at images on the same two scales. All questionnaires were formed by Google form and conducted via the online survey.In analysis, we assumed that differences of evaluations of the actor and audiences are indicating gaps of emotions the actor expressed and the audiences felt. we examined the significance of the difference using a two-sided t-test (significance level = 0.05) to investigate the degree of the gap.3. Results and discussionEvaluations by actor’s self (N=1) and by audiences (averages of N=30) are generally similar, but there are significant differences (p<0.05) in frustrating, guarding, relaxing, angry, comfortable, fear, and boring in valence and all emotions in arousal. These results show that the actor’s emotion conveyed to audiences roughly, but the degree of the actor’s emotion was not impressed on audiences accurately.We assumed emotional plane which consists of two axes of valence and arousal using Russel’s circumplex model as a reference and calculated distances of the actor’s emotion point and audiences’ impression point on the plane to compare the difference by sex and acquaintance. Male audiences could evaluate relatively close to the actor’s emotion with female audiences, but a significant difference among sex (p>0.05) was not found in any images. On the other hand, acquainted audiences could evaluate relatively close to the actor’s emotion with unacquainted audiences, and there are significant differences among acquaintance in frustration (p=0.029), angry (p=0.029), and comfortable (p=0.040).
Nonverbal information is an important component of all types of communi- cation. According to Mehrabian’s study, the face has 55% of the information that people receive, while the voice has 38%, and the verbal content has 8%. Computer-mediated communication technology promotes communica- tion that does not need face-to-face communication (e.g., e-mail). Recently, however, a video communication system that enables face-to-face conversa- tions via a screen is on the rise, and understanding the impressions conveyed by facial expressions is becoming increasingly important.
It is well known that facial expressions as a medium of emotion are univer- sal (Ekman & Friesen, 1971). Fullwood et al. (2007) reported that impression formation via video communication makes people each other as perceive less likable and intelligent. Recently, automatic recognition of emotions from facial expressions has been developed using image processing and advanced machine learning technologies (Yang et al., 2018). However, Sato et al. (2019) illustrated that Ekman’s theory is not suitable for the Japanese population. That is, Ekman’s theory is roughly universal, but a gap between the general concept and each case may exist. Therefore, it is necessary to consider facial expressions as a medium of emotion in various situations.
Our purpose was to examine whether facial expressions can accurately convey emotions to another person. First, we prepared bust shots of an actor
Analysis of an Actor’s Emotions and Audience’s Impression of Facial Expressions 11
Figure 1: Examples of stimulus images. The left is excited, and the right is bored. All photographs are shown without black bars in the actual experiment.
showing facial expressions of instructed emotional keywords. The actor eva- luated her own emotions using a Likert scale for valence and arousal. Second, other participants as audience evaluated the actor’s emotion by looking at the photographs using the same scales. Finally, we compared the evaluations of the actor and audience to investigate how accurately facial expressions convey emotions. Our study used emotion words as evaluation terms and compared evaluations between audiences acquainted and unacquainted with the actor.
METHODS
In the experiment, we created images of the actor’s face corresponding to an emotional keyword and showed them to the participants (audience). The actor was a female volunteer in her 20s. The participants were 15 male and 15 female volunteers in their 20s. Out of these, 15 were acquainted with the actor, and the rest saw the actor for the first time through this experiment.
The actor was instructed to create facial expressions for eight emotions: “surprised,” “frustrated,” “excited,” “guarded,” “relaxed,” “angry,” “fear,” and “bored.” Stimulus images were bust shots (photographs of the upper body) of the actor creating facial expressions corresponding to the emotions. Examples of the images are shown in Figure 1. After taking the photos, the actor was instructed to evaluate her own emotions in the images using two 10-point Likert scales ranging from displeasure (0) to pleasure (10), and from deactivated (0) to activated (10). Similarly, the audience evaluated the emo- tions after looking at images on the same two scales. All questionnaires were created using Google Forms and were administered via an online survey.
In the analysis, we assumed that differences in evaluations of the actor and audience indicate gaps in emotions that the actor expressed and the audience felt. We examined the significance of the difference using a two-sided t-test (significance level = 0.05) to investigate the degree of the gap.
RESULTS
Figure 2 shows that while the evaluations by the actor herself (N = 1) and by audience (N = 30) were generally similar, there were significant differences in valence and arousal (p<0.05) for frustrated, guarded, relaxed, angry, fear,
Equation reproduced from the original PDF.Figure 2: Valence and arousal of each emotion evaluated by actor (white bars) and audi- ence (black bars and error bars indicating averages and standard deviations). Abbre- viated words on horizontal axis are “surprised,” “frustrated,” “excited,” “guarded,” “relaxed,” “angry,” “fear,” and “bored” from left to right.
Figure available in the original PDF paper.
Table 1. Distances in ratings between actor and audience. Meanings of abbreviated words on the header row are same as in Figure 2.
and bored. These results show that the actor’s emotions were conveyed to the audience, but the degree of the emotions was not communicated accurately.
We assumed an emotional plane that consisted of two axes of valence and arousal using Russel’s circumplex model (Russell and Barrett, 1999) as a reference and calculated the distances of the actor’s emotion point and the audience’s impression point on the plane to compare the difference by sex and acquaintance. The calculated distances are listed in Table 1. The average distance was 2.93 in the audience overall, 2.71 in males, 3.14 in females, 2.75 in acquainted, and 3.10 in unacquainted audience members. Male audience members could evaluate the actor’s emotion relatively more accurately than female ones, but a significant difference by sex (p>0.05) was
Analysis of an Actor’s Emotions and Audience’s Impression of Facial Expressions 13
not found for any images. In contrast, acquainted audience members could evaluate the actor’s emotion more accurately than unacquainted ones, and there were significant differences in evaluating frustration (p = 0.029) and anger (p = 0.029).
CONCLUSION
We explored whether people can convey their emotions accurately through facial expressions. We hypothesized that evaluations of the actor and audience would be similar if the emotions were conveyed precisely. Significant differences were found in valence for frustrated, guarded, relaxed, angry, fear, and bored; and in arousal for all emotions. This suggests incorrect communi- cation of emotions. Distances between the actor’s emotion and the audience’s perception were larger in audience unacquainted with the actor than in those who were acquainted. The results suggest that conveying emotion via facial expressions is not reliable, especially with unfamiliar people. In the new era of video communication, caution should be exercised. Additionally, the dif- ference may be more significant in communication with foreign people. To communicate emotions correctly, we need to construct a model explaining the differences, and support communication on the basis of the model. However, the effect of emotion conveying we showed is limited because we employed female actor for the experiment. Although we showed that the effect does not depend on audience’s sex in this study, it is possible that actor’s sex can affect the accuracy of emotion conveying. An effect of actor’s sex should be discussed in our future work.
ACKNOWLEDGMENT
We would like to thank Mr. Nakagome and Ms. Kawaguchi (who were our initial research partners until Mr. Nakagome graduated from Chuo Univer- sity) for data acquisition and summarizing the experimental data. This work was partially supported by the Collaboration Research Fund of the Institute of Science & Engineering, Chuo University.
REFERENCES
Ekman, Paul and Friesen, Wallace V. (1971) Constants across cultures in the face and emotion. Journal of Personality and Social Psychology Volume 17, No. 2. Fullwood, Chris. (2007) The effect of mediation on impression formation: A compa-
rison of face-to-face and video-mediated conditions. Applied Ergonomics Volume 38, No. 3. Russell, James and Barrett, Lisa Feldman. (1999) Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology Volume 76, No. 5. Sato, Wataru, Hyniewwska, Sylwia, Minemoto, Kazusa and Yoshikawa, Sakiko.
(2019) Facial expressions of basic emotions in Japanese laypeople. Frontiers in Psychology Volume 10, No. 259. Yang, D., Alsadoon, Abeer, Prasad, P.W.C., Singh, A.K., and Elchouemi, A. (2018) An emotion recognition model based on facial recognition in virtual learning environ- ment. Proceedings of the 6th International Conference on Smart Computing and Communications, Kurukshetra, India.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Affective and Pleasurable Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-17-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional