Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Interactive Gesture of Exhibition Hall Mobile Follow Service Robot’

School of Mechanical Engineering, Southeast University, Nanjing, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1001109

Abstract

In order to make gesture-based interaction more natural for users, this research aims to find suitable gestures for interaction between users and exhibition hall service robots. The research process is divided into two stages. The first stage conducts demand analysis and task definition to analyze user needs and tasks in different scenarios during the visit of the exhibition hall, thus to define the task set of the exhibition hall service robot gesture interaction. The second stage conducts two experiments. Experiment one carries out gesture inspiration. Participants are invited to design gestures for different tasks in each scene,therefore obtain the user-defined gesture sets. Experiment two conducts gesture evaluation tests. Experts are invited to select three candidate gestures for each task. In order to collect user preferences for candidate gestures, participants are asked to watch candidate gesture’s demonstration videos and score candidate gestures in terms of ease of use, comfort, matching and memorability The candidate gestures are sorted according to their scores. The most ideal gestures for human-computer interaction between exhibition hall service robots and users are obtained in different scenarios and tasks.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

INTRODUCTION

With the advancement of industrialization and the information age, service robots are becoming more and more common, therefore saving manpower and material resources. At the same time, multi-modal natural interaction is a new way of communication and interaction between robots and users through multiple channels such as vision, hearing, gestures, and somatosensory actions, thus forming an interaction similar to the natural interaction experience between humans. As the main part of somatosensory interaction, somatosensory gesture interaction is more inclined to control the system through the upper limbs or hands. There are still many problems from the user experience. It is necessary to define the suitable interactive gesture set for the interaction between the user and the service robot to improve user experience and interaction efficiency.

RELATED RESEARCH Bor Gregorcic[1]designed interactive gestures for home smart devices based on the design principles of nature, difference, simplicity and efficiency. Through experiments, the recognition time, accuracy, error rate and user time of the designed gestures were studied. Patricia Pons[2] pointed out the characteristics and principles of the current gesture interaction design of smart terminal devices. The design should follow the principles of convenience, feedback, low cost and innovation, which can be used as a reference for this research. Based on leap motion, Wenmin Li [3] studied the interactive information, gesture interaction intentions and interactive behaviors of smart terminal devices to design interactive gestures. Qin Ken[4] researched and proposed a method of gesture interaction design for wearable devices based on context models, and guided the innovation of gesture interaction design for wearable devices. Wu Huiyue[5] proposed a model for gesture interaction evaluation in an immersive environment. The evaluation model defined two objective quantitative evaluation indicators, gesture trajectory data and experimental task data, and collected data from users who interacting with gestures in an immersive environment, forming an evaluation system consistent with subjective evaluation. Rautaray[6] determined the usability evaluation index of gesture operation. Research the performance of different gestures of typical operations through laboratory experiments, and analyze the causes in combination with the cognitive and exercise characteristics of middle- aged and elderly users, so as to design the most suitable gestures for the elderly.

The current research of interactive gestures on smart terminals lacks a user-centered design method and design process, and does not combine user designing with designer designing. Therefore, this article is based on user designing and designer designing to study the interactive gestures between users and service robots in exhibition halls.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

REQUIREMENTS ANALYSIS AND TASK DEFINITION

DEMAND ANALYSIS Analyze the tasks that users interacting with the robots, so as to explore the gestures of the exhibition hall service robots under different tasks in the below experience map.

Figure 1. Get common gestures from the user experience map.
Original PDF Figure 1.

DEMAND ANALYSIS According to user's possible gestures in the user experience map(figure 1) during visiting the exhibition, the 8 commonly used gestures in the following list are finally defined after removing some gestures with low probability of using.The lists are as follows.

Table 1: Interactive gesture tasks NO Task name NO Task name
Original PDF Table 1:

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

REQUIREMENTS ANALYSIS AND TASK DEFINITION EXPERIMENT ONE: GESTURE STIMULATION

Experiment Participants:This experiment recruited 6 middle-aged subjects (3 males, 3 females), aged from 30 to 40 years old, and 6 young subjects (3 males, 3 females), aged 20- 29 years. Recruited subjects are healthy, physically clear, and able to understand the process and requirements of the experiment clearly.

Laboratory Equipment: Equipment required for the experiment: a normal working laptop, a mobile phone for taking photos and videos and recording the experiment process and a classification statistics table that records the basic conditions of the subjects and the classification results of the experiment Experiment Steps:The order of the 12 subjects participating in the experiment was disrupted. Participants had to browse the 8 gesture tasks before designing. According to the list of the gesture tasks, the subjects designed two gestures that were reasonable for each task. Take photos of the gestures designed by the Participants. At the same time, the subjects were required to express their own ideas during the design process. After deduplicating and deleting the non-standard gestures, a user-defined gesture set is obtained.

Results:For these 8 gesture tasks, a total of (8*2*12) 384 experimental gestures were obtained through Experiment one. And a set of 174 user-defined gestures were obtained after deduplicating and deleting the non-standard gestures.

EXPERIMENT TWO: GESTURE ASSESSMENT Experiment Steps: Because there are many candidate gestures, a two-round selecting method is adopted. In the first round, experts exclude obviously unreasonable gestures based on their own experience and theoretical knowledge.Then they score the remaining gestures.

The gesture set after the first round of selection by experts.

Figure 2. The gesture set after the first round of selection by experts.
Original PDF Figure 2.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Table 2: Interactive gesture tasks
Original PDF Table 2:
Figure 3. The selected candidate gestures set by experts.
Original PDF Figure 3.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Rating Scale: Invite 15 participants to use the subjective evaluation method to score the selected 24 candidate gestures(figure 3). The scoring system is 5 points. Very satisfied is 5 points, relatively satisfied is 4 points, generally 3 points, less satisfied is 2 points, and very dissatisfied is 1 point. The rating scale is as follows.

Table 3: Candidate Gesture Rating Scale
Original PDF Table 3:

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

RESULTS AND ANALYSIS Through questionnaires and subjective evaluation scales, the most ideal gestures for each task are obtained, which are 1-2, 2-2, 3-1, 4-3, 5-1, 6-1, 7-3, 8 -3.

Figure 4. The The most ideal gesture set under 8 tasks
Original PDF Figure 4.

THE CONCLUSION In this study, the most ideal gestures for each task were obtained through experiment one and experiment two. These gestures have high scores in terms of ease of execution and memorability.

In this study, the best gestures for each task of the user when visiting the exhibition hall were obtained. In actual use, the efficiency of these gestures needs to be further explored. There are some limitations of the experiments.Experts use empirical knowledge to select gestures in the first round of selecting. There are existing subjective factors which may cause deviations in experimental results. At the same time, in the actual exhibition hall scene, whether these gestures will affect the user's experience remains to be further explored. Future research may focus on evaluating the use efficiency of gestures for various tasks and exploring the factors which may affect the efficiency of gesture applications, so as to provide more accurate and scientific theoretical basis for the gesture research of exhibition hall service robots.

REFERENCES

Bor Gregorcic, Gorazd Planinsic, Eugenia Etkina Doing science by waving hands: Talk, symbiotic gesture, and interaction with digital content as resources in student inquiry[J] Physical Review Physics Education Research, 2017, 13(2)

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Patricia Pons, Javier Jaen Interactive spaces for children: gesture elicitation for controlling ground mini-robots[J] Journal of Ambient Intelligence and Humanized Computing, 2019(prepublish) Wenmin Li,Xueyi Wan,Yanwei Shi,Nailang Yao,Ci Wang,Zaifeng Gao. Depth and Breadth

of Pie Menus for Mid-air Gesture Interaction[J]. International Journal of Human– Computer Interaction,2020,37(2). Qin Ken,Chen Chen,Pu Xianjie,Tang Qian,He Wencong,Liu Yike,Zeng Qixuan,Liu Guanlin,Guo Hengyu,Hu Chenguo. Magnetic Array Assisted Triboelectric Nanogenerator Sensor for Real-Time Gesture Interaction[J]. Nano-Micro Letters,2021,13(1). Wu Huiyue, Wang Yu, Qiu Jiali et al. User-defined gesture interaction for immersive VR

shopping applications[J] Behaviour & Information Technology, 2019, 38(7) Rautaray S S, Agrawal A. Vision based hand gesture recognition for human computer interaction: a survey[J]. Artificial Intelligence Review, 2015, 43(1): 1-54.

Human Systems Engineering and Design (IHSED 2021) https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8987-0

Cover of the proceedings volume Human Systems Engineering and Design (IHSED2021): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED2021): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-7923-8987-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →