INTRODUCTION
Personality determines a person’s beliefs, likes, dislikes, thoughts, behavior, and how they act in various situations. Social network posts reflect the user’s personality. Personality prediction using social media posts has been done a lot, but now robots are also in this field and much work has already been done. Social robots come in a variety of shapes and sizes that are used in human-robot interactions. Science fiction has led us to imagine a future in which robots assist us in the work. In the realm of rehabilitation, these huma- noid robots are well-known, although designers are cautious to avoid the Uncanny valley idea (Pandey and Gelin, 2018; Henschel, Laban and Cross, 2021), (Mara, Appel and Gnambs, 2022). However, it is widely assumed that humanoid robots should imitate humans so, researchers are utilizing robots in personality prediction work. Recognizing human-like features is the definition of a social or humanoid robot (Fox and Gambino, 2021). Numerous personality theories are used to predict personality. Allport’s’ Trait
© 2022. Published by AHFE Open Access. All rights reserved. 53
theory, Cattell’s 16 Factor personality, Eysenck’s three Dimensions of perso- nality, and Myers-Brigg’s type Indicator (MBTI) (Matz, Chan and Kosinski, 2016) are among the most well-known. The Big-Five is the most famous and commonly used with five primary traits Extraversion, Openness, Conscien- tiousness, Agreeableness, Neuroticism. Each of these has several sub-traits. Costa and McCrae developed the NEO-Five factor inventory (NEO-FFI), a 60 items variant of NEO-FFI (1992). International Personality Item Pool Big- Five marker Scales (IPIP 50) contains 100 items presented by Goldberg (1992) after many years briefer version 20 items developed by Donnellan, Oswald, Baird, and Lucas (2006). The Big-Five Inventory (BFI) 44 items were designed by john, Donahue, Kentle (1991) after many years 10 item version was deve- loped in German & English by Rammstedt and John (2007). The rest of the paper is organized as, section II which comprises personality traits from the big-five model with sub-traits, questionnaires with how they aid in data colle- ction, and the role of a psychologist in all of this. Section III delves into how human emotions or facial expression, paralanguage, head pose, and body movement are extracted and linked traits. Section IV digs into the specifics of robots engaged in personality studies.
PERSONALITY TRAITS & QUESTIONNAIRES Personality theories contend that individual human features may be utilized to predict human emotion, cognition, and behaviors. A trait is described as “a dimension of personality used to classify persons based on the extent to which they display a certain feature” (Anzalone et al., 2017). These characteristics are regarded as the basic pillars of personality.
Traits The Big-Five model traits are prominent both in social science research study and human-robot interaction. To support these attributes OCEAN acronym is utilized, which stands for Openness, Conscientiousness, extraversion, agre- eableness, and neuroticism (Bhin, Lim and Choi, 2019), (Shah and Modi, 2021), (Müller and Richert, 2018). Each trait as well as facets derived from McCrae & Costa (2006) and utilized in many studies, with each trait inverse. Each trait has a severity scale ranging from low to high.
Questionnaire Questionnaires based on Liker Scales are utilized for personality evalu- ation (Anzalone et al., 2017), (Shen, Elibol and Chong, 2021). These surveys allow humans to self-judgment of personality (Anzalone et al., 2017), (Salam et al., 2016), (Huang et al., 2021). In most circumstances, the psychologist provides judgment, except questionnaires. Psychologists pro- vide judgment (cue validity & utilization) linked with the Lens model (Breil et al., 2021) and in our opinion features accuracy is dependent on it, Although ranking which cue is more important in which trait is tough. As previously noted, many surveys can be utilized, but NEO-FFI, IPIP, and BFI are linked to the Big-Five paradigm. Everyone is preoccupied with their jobs. Long surveys take time to complete, either a one-person survey (participant self-esteem) or
Personality Prediction in Human-Robot-Interaction (HRI) 55
when participants are needed to score other groups. These surveys are used for data gathering (features), and data is labeled by consulting a psychologist.
FEATURES EXTRACTION & TRAITS According to psychopathology, face expression and body movement have valuable applications in the study of emotions, depression, and personality. These behavior human show in different situations with speech or without it. Research on non-verbal behavior exposed different channels of judgment. Observers sometimes observe a silent film, live user behavior, typescript, or voice. Non-verbal communication that includes bodily motions, posture, gestures, and facial expressions is referred to as kinesics (Saunderson and Nejat, 2019). Humans are quite adept at deciphering body cues particu- larly when it comes to personality judgments. Some cues are common in personality studies.
Head Pose In interactions, humans often utilize the head posture to convey their thou- ghts whether they want to converse or not, and they agree or disagree. Glancing during a talk at the interaction partner indicates attentiveness but gazing elsewhere indicates apathy and anxiousness (Shen, Elibol and Chong, 2021), (Saunderson and Nejat, 2019). The gaze is connected to the head pose because measuring gaze with a robot camera requires high-resolution pictu- res and increases the computational cost. Head movement and eye contact are both favorably associated with extroverts because social people employ these indicators during the conversation. Head pose is linked to neuroti- cism as a result of the unstable emotions a person moves his head often to avoid eye contact. Both are favorably associated with openness because the individuals are open to new experiences and want to connect more. To display submissive conduct, an agreeable person avoids eye contact and low- ers the head. An organized and disciplined individual is conscientious, eye contact with head motion is used to convey confidence and disapproval. The roll, pitch, and yaw angles were used to predict head posture to deter- mine the gaze score instead of utilizing eyes (Shen, Elibol and Chong, 2021), (Zafar, Paplu and Berns, 2018) and the gap between two adjacent frames was estimated it’s referred to as Manhattan distance. Another estimating appro- ach is the direction magnitude pattern (DMP) (Zafar, Paplu and Berns, 2018), which calculates the direction and magnitude of each pixel concerning its nei- ghbor’s resultant force. ROI was recognized using the OpenCV version of the Viola and Jones (Viola and Jones, 2004), Haar cascade technique, and then the Intraface library is used to compute head angles (Salam et al., 2016). Hough transformed (Anitta and others, 2021) was also be utilized to extract head posture.
Body Movement In studies, arm gestures and body movements were assessed and even the tiniest change in the body was measured. Waving, folded arms, pointing, and other hand or arm actions as do postures like thinking posture, erect
posture, and crouching posture send messages. Except for head motions, all movements are considered body motions. Neuroticism has a negative relation with some bodily movements due to unstable emotions, but closed arms and thinking posture are positively related. Extroversions demonstrate bodily movements since they are emotionally expressive. Openness includes those individuals that display curiosity in new experiences, as seen by their open stance or self-assured posture. To demonstrate their agreeableness, agree people don’t display much body movements, instead they opt for an open sta- nce. Conscientious person is cautious and competent, they display an open posture or closed-arm posture with a change in proximity to express their opi- nions. Body pose is determined (Salam et al., 2016; Zafar, Paplu and Berns, 2018) by recognizing skeletal joints of the body, if it reaches a certain thre- shold then movement occurs otherwise nothing happens. The body motion can be extracted using two photographs. Joint angles were calculated after comparing the original and warped images, images suggesting that the skele- ton has been rotated in the reference to the initial skeleton (Shen, Elibol and Chong, 2021).
Facial Expression/Emotions Facial expressions are any facial muscle action, such as smiling and yawning, as well as expressions with the eyes and brows such as winking, scowling, and so on. The study of facial expression focus on emotions (Ekman and Friesen, 1974). People appraise effective estates based on indicators, such as drooping eyebrows indicating anger. A smile is a symbol of happiness, friendliness, and positive emotion for extroverts. Neurotic people experience negative emoti- ons such as fear, rage, and anxiety as a result of their unstable emotions. Agreeableness is associated with laughter and positive social contact since it is connected to friendliness, compassion, and warmth. Conscientious people laugh for some reason, and they have regulated smirk. Negative emotions have a weak association with conscientiousness. Openness to experience is positively associated with the laugh these individuals want to connect and smiling is a form of communication. Basic emotions were distinguished using convolution neural networks (Zafar, Paplu and Berns, 2018). Five techniques for emotion recognition were compared (Kartali et al., 2018), using happi- ness, sadness, anger, and fear. The deep learning technique AlexNet CNN for prediction, FER-CNN for extraction features, Affdex CNN, and convo- lution neural network (CNN) compared to differentiate six basic emotions. SVM and MLP ANN classifiers were employed with HOG features extra- ction, utilizing facial landmarks supplied by OpenFace for detection. The Affdex SDK and convolution neural network (CNN) were then compared to differentiate six basic emotions (Lopez-Rincon, 2019). A review of deep learning for emotion identification is presented in (Abdullah et al., 2021).
Paralanguage Paralanguage gives speech rate, voice break, frequency, pitch variance, ampli- tude, and variation in amplitude (Breil et al., 2021). Individuals rely on their voices to make an impression on others and these cues convey emotions. The
Personality Prediction in Human-Robot-Interaction (HRI) 57
big-five factors are related to communication (Sims, 2017). Extroverts are good communicators and use language signals to express intentions. A neuro- tic person has unstable emotions and refuses to communicate. Agreeableness indicates generosity, sympathy, etc. whereas aggressiveness is the converse. Pleasant people don’t adopt aggressive speech to defend their point of view. The creativity, and complexity of one’s thoughts are measured by openness, these people are expressive, witty, and verbally proficient. Goal-oriented and self-efficient persons are conscientious. These folks attain their objectives through aggressive communication. After classifying features with k-nearest neighbor (KNN) and Support vector machine (SVM), Linear domain frequ- ency and Mel Frequency Cepstral Coefficients were collected (Jothilakshmi, Sangeetha and Brindha, 2017). The speech feature extractor OpenSMILE was used to extract acoustic features from audio data (Mawalim et al., 2019), which were then combined with head motion and communication abilities as well. The classification techniques employed were SVM, random forecast, Naïve Bayes, and decision tree algorithms. Pitch and vitality of the voice, Mel- frequency cepstral coefficient was extracted (Pandey and Gelin, 2018), and auto-correlation function was employed to monitor pitch before calculating the average of short-term energy.
SOCIAL ROBOTS Human-Robot-Interaction (HRI) is an exceedingly new topic and has gained a lot of interest. Robots are rapidly being created for global applicati- ons such as eldercare, rehabilitation, robot-assisted treatment, and many more. The purpose of social robots is to engage with people utilizing both verbal and non-verbal cues. Many social robots feature anthropomorphic (humanoid) or animal-like appearances. Many social robots have been deve- loped over the years such as Geminoid, Pepper, Nao, Roman, KASPAR, Aibo, Paro, kismet, Keepon, and others (Breazeal, Dautenhahn and Kanda, 2016). NAO and Pepper have commonly used robots. Emotions (Lopez- Rincon, 2019), Autism (Ali et al., 2019), tutoring (Kanero et al., 2021), therapies (Jiménez et al., 2019), tourism (Park, 2020), and a variety of other researches have employed NAO. Pepper operates as a teacher at home (Tanaka et al., 2015), in shopping malls for amusement (Aaltonen et al., 2017), exhibiting emotions (Tuyen, Jeong and Chong, 2018), tourism guide (Park, 2020), and many more sectors, just as the NAO. Many robots are utilized in personality research. NAO was employed (Salam et al., 2016), and participants have a cooperative engagement with each other and NAO. Using the iCub robot (Anzalone et al., 2017) single extraversion trait was measured. A ROBIN humanoid robot was used (Zafar, Paplu and Berns, 2018), for the analyze personality traits in various circumstances. Using pep- per (Shen, Elibol and Chong, 2021) features extraction was done, and the robot was interacting with a person at the time. RoBoHoN (Huang et al., 2021) was used to measure personality by recognizing verbal features. Aside from that, iCat, PeopleBot, Meka are employed in numerous personality research.
CONCLUSION
Personalities understanding is vital in HRI. The Big-five is the most used model. Extraversion is the most quantifiable trait. Openness and agree- ableness were the least examined traits. All interactions were done in a controlled setting. Each study used a survey as a baseline, and psycho- logists were asked to classify the observed traits. NEO and IPIP was the most used survey. Skeletonization is often used for extracting body moti- ons while deep learning is extensively used for emotion or facial expression identification.
Features pairing can improve the accuracy of traits. People node heads with remarks to show disapproval. When people interact, they show open posture with staring and zero-degree head stance. When people are angry, they turned heads to avoid eye contact, show close arms, facial expressions change, and shout. Face expression is often linked with speech as compared to other cues. Based on cues it is easy to tell if someone is joyful, sad, furious, scared, or thrilled.
REFERENCES
Aaltonen, I. et al. (2017) ‘Hello Pepper, may I tickle you? Children’s and adults’ responses to an entertainment robot at a shopping mall’, in Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, pp. 53–54. Abdullah, S. M. S. A. et al. (2021) ‘Multimodal emotion recognition using deep
learning’, Journal of Applied Science and Technology Trends, 2(02), pp. 52–58. Ali, S. et al. (2019) ‘An adaptive multi-robot therapy for improving joint attention and imitation of ASD children’, IEEE Access, 7, pp. 81808–81825. Anitta, D. and others (2021) ‘Human head pose estimation based on HF method’,
Microprocessors and Microsystems, 82, p. 103802. Anzalone, S. M. et al. (2017) ‘Automated prediction of extraversion during human--humanoid interaction’, International Journal of Social Robotics, 9(3), pp. 385–399. Bhin, H., Lim, Y. and Choi, J. (2019) ‘Recognition of personality traits using word
vector from reflective context’, in 2019 7th International Conference on Robot Intelligence Technology and Applications (RiTA), pp. 89–92. Breazeal, C., Dautenhahn, K. and Kanda, T. (2016) ‘Social robotics’, Springer handbook of robotics, pp. 1935–1972. Breil, S. M. et al. (2021) ‘Contributions of nonverbal cues to the accurate judgment
of personality traits’, in The Oxford handbook of accurate personality judgment. Oxford University Press. Ekman, P. and Friesen, W. V (1974) ‘Nonverbal behavior and psychopathology’, The psychology of depression: Contemporary theory and research, pp. 3–31. Fox, J. and Gambino, A. (2021) ‘Relationship Development with Humanoid
Social Robots: Applying Interpersonal Theories to Human--Robot Interaction’, Cyberpsychology, Behavior, and Social Networking, 24(5), pp. 294–299. Henschel, A., Laban, G. and Cross, E. S. (2021) ‘What Makes a Robot Social? A Review of Social Robots from Science Fiction to a Home or Hospital Near You’, Current Robotics Reports, pp. 1–11.
Personality Prediction in Human-Robot-Interaction (HRI) 59
Huang, T.-R. et al. (2021) ‘Asynchronously Embedding Psychological Test Questions into Human--Robot Conversations for User Profiling’, International Journal of Social Robotics, 13(6), pp. 1359–1368. Jiménez, M. et al. (2019) ‘Recognition of Colors through Use of a Humanoid Nao
Robot in Therapies for Children with Down Syndrome in a Smart City’, Research in Computing Science, 148(6). Jothilakshmi, S., Sangeetha, J. and Brindha, R. (2017) ‘Speech based automatic personality perception using spectral features’, International Journal of Speech Technology, 20(1), pp. 43–50. Kanero, J. et al. (2021) ‘Are Tutor Robots for Everyone? The Influence of Attitudes,
Anxiety, and Personality on Robot-Led Language Learning’, International Journal of Social Robotics, pp. 1–16. Kartali, A. et al. (2018) ‘Real-time Algorithms for Facial Emotion Recognition: A Comparison of Different Approaches’, in 2018 14th Symposium on Neural Networks and Applications (NEUREL), pp. 1–4. Lopez-Rincon, A. (2019) ‘Emotion recognition using facial expressions in chil-
dren using the NAO Robot’, in 2019 International Conference on Electronics, Communications and Computers (CONIELECOMP), pp. 146–153. Mara, M., Appel, M. and Gnambs, T. (2022) ‘Human-Like Robots and the Uncanny Valley’, Zeitschrift für Psychologie. Matz, S., Chan, Y. W. F. and Kosinski, M. (2016) ‘Models of personality’, in Emotions
and Personality in Personalized Services. Springer, pp. 35–54. Mawalim, C. O. et al. (2019) ‘Multimodal bigfive personality trait analysis using communication skill indices and multiple discussion types dataset’, in Internatio- nal Conference on Human-Computer Interaction, pp. 370–383. Müller, S. L. and Richert, A. (2018) ‘The Big-Five Personality Dimensions and Attitu-
des to-wards Robots: A Cross Sectional Study’, in Proceedings of the 11th Perva- sive Technologies Related to Assistive Environments Conference, pp. 405–408. Pandey, A. K. and Gelin, R. (2018) ‘A mass-produced sociable humanoid robot: Pepper: The first machine of its kind’, IEEE Robotics & Automation Magazine, 25(3), pp. 40–48. Park, S. (2020) ‘Multifaceted trust in tourism service robots’, Annals of Tourism
Research, 81, p. 102888. Salam, H. et al. (2016) ‘Fully automatic analysis of engagement and its relationship to personality in human-robot interactions’, IEEE Access, 5, pp. 705–721. Saunderson, S. and Nejat, G. (2019) ‘How robots influence humans: A survey
of nonverbal communication in social human--robot interaction’, International Journal of Social Robotics, 11(4), pp. 575–608. Shah, V. and Modi, S. (2021) ‘Comparative Analysis of Psychometric Prediction System’, in 2021 Smart Technologies, Communication and Robotics (STCR), pp. 1–5. Shen, Z., Elibol, A. and Chong, N. Y. (2021) ‘Multi-modal feature fusion for better
understanding of human personality traits in social human--robot interaction’, Robotics and Autonomous Systems, 146, p. 103874. Sims, C. M. (2017) ‘Do the big-five personality traits predict empathic listening and assertive communication?’, International journal of listening, 31(3), pp. 163–188. Tanaka, F. et al. (2015) ‘Pepper learns together with children: Development of an
educational application’, in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pp. 270–275.
Tuyen, N. T. V., Jeong, S. and Chong, N. Y. (2018) ‘Emotional bodily expressions for culturally competent robots through long term human-robot interaction’, in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2008–2013. Viola, P. and Jones, M. J. (2004) ‘Robust real-time face detection’, International
journal of computer vision, 57(2), pp. 137–154. Zafar, Z., Paplu, S. H. and Berns, K. (2018) ‘Automatic assessment of human personality traits: a step towards intelligent human-robot interaction’, in 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pp. 1–9.