Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Design of toddler care equipment based on facial emotion recognition technology

School of Arts and Design, Shenzhen University, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1001982

Abstract

Emotional neglect for children is the most important form of early life maltreatment (ELM), and the damage it brings to society is incalculable (Müller, 2019). At the same time, the increasing pressure has left some parents with no time and energy to care for their children, leading to emotional neglect in increasing prevalence. These facts remind us of the need to pay attention to the emotional needs of children. Therefore, products that alleviate the phenomenon of children's emotional neglect are worthy of research. It is easy to understand that artificial intelligence is booming in the field of emotion recognition through literature research, and it will have great potential in the future. This issue will explore the application of facial emotion recognition in children's life situations through research on the current situation of emotional neglect. And try to apply facial emotion recognition to monitoring equipment for the toddler to reduce parenting pressure for parents and satisfy the emotional needs of children.Keywords: Toddlers, Facial Emotion Recognition, AI, Emotion Neglect

INTRODUCTION

There are two main ways of parenting in China. In some families, parents raise children; in other families, children are raised by grandparents. Inter- generational parenting has become mainstream in the city (Hu, 2022). At the same time, with the development of the social economy and the popu- larization of advanced educational concepts, the vast majority of families are willing to invest more energy and expenses in the growth of their chil- dren (Wang, 2021; Ying, 2016). However, that was not the case, the elderly are anenergia, and the parents are very busy, leading to their children’s typical emotional neglect. In recent years, with the development of the Inter- net of Things, nursing equipment has also begun to develop in intelligence, humanization, and digitization.

(1) Emotional neglect in children.

Early stressful experiences such as early life maltreatment (ELM) can negati- vely impact their later social lives (Müller, 2019), and the most prevalent form of ELM is emotional neglect in childhood. Emotional neglect is now preva- lent, and the proportion of children suffering from it increases (Strathearn, 2011; Witt et al., 2017).

© 2022. Published by AHFE Open Access. All rights reserved. 549

550 Li and Luo

(2) Facial emotion recognition.

Facial emotion recognition is a branch of emotion recognition techno- logy. With the continuous optimization of Convolutional Neural Networks (CNN), CNN-based facial emotion recognition has a more accurate reco- gnition rate than before. Recent research has demonstrated the feasibility of facial emotion recognition in virtual life situations (Martínez et al., 2021). The current research also points out the problems with emotion recogni- tion, focusing mainly on facial capture and ethical challenges (Hagendorff, 2020).

(3) Toddler’s care equipment.

There are two main types of toddler’s care equipment on the market now: health data detection equipment and camera monitoring equipment (Zhong, 2019). Among them, camera surveillance products are top-rated in China, which means that surveillance of infants and toddlers in China is not una- cceptable. There are some devices have used emotion recognition technology to analyze the images of infants’ and toddlers’ sleeping postures and moni- tor their crying (Khan, 2021). Therefore, facial emotion recognition is an efficient method and may bring better results.

This project aims to relieve the pressure of parents and grandparents to take care of children. It tries to solve a series of problems by using facial emo- tion recognition technology by designing monitoring equipment for toddlers to reduce caregivers’ neglect of children’s emotions.

METHODS

The research of this project is mainly divided into three stages. Firstly, use questionnaire analysis and literature research to understand the impact of emotional neglect on the current marriageable group and use the AHP analy- sis to study which situations in the family scene are more likely to cause emotional neglect (see Figure 1). Secondly, investigate the actual usage situ- ations of facial recognition and take pictures of children’s daily scenes at a specified angle through the interview. Finally, by using Baidu AI face emo- tion detection to detect the toddler’s emotions in the previous scene, and at the same time, emotional judgments are made manually. Then, comparing two sets of data to verify the feasibility of facial emotion recognition.

Questionnaire design: This questionnaire was divided into three parts. The first part was the survey of the current situation of emotional neglect in the marriageable group. This stage aimed to understand whether the participants know emotional neglect and whether they experienced emotional neglect in childhood. The second part was designed to investigate the high-incidence situations of emotional neglect. In this part, 12 common high-incidence situ- ations of emotional neglect in our lives will be set up as questions, 6 of which are about mothers and 6 about fathers. The situations were compi- led from interviews with 40 parents of kindergarten children, and then the participants were asked to score the possibility of emotional neglect in these situations through a total of seven options corresponding to a score of 1-7.

Design of Toddler Care Equipment Based on Facial Emotion Recognition Technology 551

Figure 1: Hierarchy of the project example.
Original PDF Figure 1:

The third part investigated the impact of childhood emotional neglect on their current lives. This part evaluated the impact of emotional neglect from current parental intimacy and attitudes toward parenting.

Participants: The participants are all the marriageable groups in China, and their age group is between 20-27 years old. A total of 110 questionnaires were collected, 43 for males and 67 for females.

Observations and interviews: Research on specific children, face-to-face inquiry and observation are more effective (Merter et al., 2018). In order to obtain a more realistic children’s environment, we chose to use observation and video records, and asked Parents pretend to ignore the children to observe and photograph the children’s emotional performance.

Participants: 4 households with infants and toddlers aged 0-3. There are two two-child family and the other are one-child family.

RESULT

Finally 102 questionnaires were collected, including 40 male questionnaires and 62 female questionnaires, and the age group was between 20-27 years old. Among the participants, 72 were mainly cared for by their parents when they were children; 36 (64.29%) were cared for by their grandparents; 4 (32.14%) were cared for by other relatives. When asked if they had suffered emotional neglect in childhood, 82 people said they had suffered emotional neglect from their parents, accounting for 80.39%, of which 70 believed that emotional neglect in childhood still affects their current life.

We asked participants to rate a total of 12 family situations that are prone to emotional neglect from three aspects: work, entertainment, and family conflicts. Moreover, use the analytic hierarchy process (AHP) to calculate the weight of these situations to get the main form of family emotional neglect (see Table 4).

From the table, the priority vectors of three types of scenes of work, enter- tainment, and family conflicts can be obtained, of which the priority vector of ‘family conflict’ is the highest of 0.344. The weight calculation for 12 situ- ations shows that fathers are most likely to ignore children’s emotions when

552 Li and Luo

Table 1. Pair-wise comparison matrix for work.

Table available in the original PDF paper.

Situation of work M.E. F.E. F.H M.H Priority vector M.E 1 0.960 0.884 1.135 0.247 F.E 1.042 1 0.921 1.182 0.257 F.H 1.131 1.086 1 1.284 0.279 M.H 0.881 0.846 0.779 1 0.217

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.
Table 2. Pair-wise comparison matrix for work.

Table available in the original PDF paper.

Situation of rest M.P F.P F.M M.M Priority vector M.P 1 0.960 0.884 1.135 0.247 F.P 1.042 1 0.921 1.182 0.257 F.M 1.131 1.086 1 1.284 0.279 M.M 0.881 0.846 0.779 1 0.217

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.
Table 3. Pair-wise comparison matrix for work.

Table available in the original PDF paper.

Situation of Conflict M.D F.D C.B C.B.P Priority vector M.D 1 0.960 0.884 1.135 0.247 F.D 1.042 1 0.921 1.182 0.257 C.B 1.131 1.086 1 1.284 0.279 C.B.P 0.881 0.846 0.779 1 0.217

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

doing hospitality work. In calculating the weight of the father situation and the mother situation, the father weight is 51.14%, and the mother weight is 48.86%, which shows that fathers are more likely to neglect children’s emotions.

Participants who had experienced emotional neglect had worse relationsh- ips with their parents, as assessed by the participant’s relationship with their parents, with an average score of 5.18 (range 1–7, the higher the score, the stronger the dependency), the score for those who did not suffer from emotio- nal neglect was 5.43, the score for participants who suffered from emotional neglect in their dependence on their father was 4.5. The score for partici- pants who did not suffer from emotional neglect was 5.4. In assessing the participants’ parenting attitudes, the participants who experienced emotio- nal neglect showed significant resistance and lack of confidence. 22.32% of the participants who had suffered emotional neglect held a negative attitude towards being a qualified parent, while only 8.33% of the participants who had not suffered from emotional neglect held a negative attitude.

In observing and interviewing 4 families, we asked parents to simulate emotional neglect situations. When parents focus on chatting with us and ignore their children, children tend to show relatively mild emotions (such as pleading, waiting, physical interference) first. If there is still no response from parents, Children of different personalities have different action strategies. The lively children used stronger emotional expressions such as screaming

Design of Toddler Care Equipment Based on Facial Emotion Recognition Technology 553

Table 4. Pair-wise comparison matrix for work.

Table available in the original PDF paper.

Criteria Local Weights

Form of Emotion Neglect Local Weights

Global Weights 0.330 Mother. Essential work (M.E) 0.247 0.081

Situation of work

0.344 Mother’s distress (M.D) 0.243 0.084

Father. Essential work (F.E) 0.257 0.085 Father. Hospitality Work (F.H) 0.279 0.092 Mother. Hospitality Work (M.H) 0.217 0.072 Situation of rest 0.326 Mother. Personal recreation (M.P) 0.259 0.084 Father. Personal recreation (F.P) 0.238 0.078 Father. Multiplayer game (F.M) 0.258 0.084 Mother. Multiplayer game (M.M) 0.245 0.080 Situation of Conflict

0.247 0.088

Father’s distress (F.D) 0.256 0.085 Conflict between parents (C.B) 0.254 0.088 Conflict between parents and other (C.B.P) Total: 1.0000

Figure 2: Comparison of AI and human judgment.
Original PDF Figure 2:

and hugging their parents, while the less lively children chose to leave. Accor- ding to interviews, these simulated emotional neglect situations are prevalent in daily life.

A total of 73 photos were collected during the interview. By calling the face recognition API interface of Baidu Inc, the facial emotions of the faces on the 73 photos were judged, and the data was collected and organized (see Figure 2).

In the manual comparison stage, 4 participants were recruited to make manual emotional judgments for 73 photos, and the data were collected and organized. By comparing the results of manual judgment and AI judgment, there are 11 of them have apparent differences (the judgment of AI is different from the judgment of the 4 participants), accounting for 15%, which is quite different from the accuracy rate advertised by Baidu Inc. Through the analysis of these 11 photos, these photos generally have problems such as partially occluded faces and side faces.

Through the above research and analysis, the project can summarize the current problems in applying facial emotion recognition to toddler monitoring.

554 Li and Luo

  • Emotional neglect is a chronic problem that occurs at almost any point in childhood. Nevertheless, current child monitoring devices are age-specific, making emotional monitoring unsustainable for long periods.
  • Children who suffer from emotional neglect use different action strategies because of their different personalities. Facial emotion recognition will be difficult to sample due to the different action strategies of the toddler, so the mode of camera monitoring equipment may be difficult to apply.
  • At present, the feedback of facial recognition results is mainly qualitative, which is difficult to express the emotions of children visually.
  • At present, the accuracy of facial emotion recognition face in real scenes is not high, mainly because of the problem of photo quality. How to capture suitable child face images is an important issue.

DISCUSSION

The above research and literature research show that facial emotion reco- gnition is difficult to support in real situations at this stage independently. The main problem is that it is difficult to collect suitable samples and cur- rent algorithms are still imperfect. This makes it is difficult to guarantee the accuracy of facial emotion recognition in actual situations, which requires the introduction of more variables to assist.

  • Adopt multimodal affect analysis framework of the emotion recognition to improve the accuracy of emotion recognition. This means that more emotion variables need to be introduced to assist judgment, including facial emotion recognition, voice emotion recognition, and text emotion recognition.
  • Adopt the mode of a mobile monitor or the mode of combination of mobile monitors and fixed monitors.
  • Reflect the changes of children’s emotions to parents in a more visual way. Use a continuous expression for the changes of emotion.
  • At this stage, the design of facial emotion recognition products can still only be based on conceptual design.
  • At this stage, the design of facial emotion recognition products can still only be based on conceptual design.

We made a design prototype of a toddler monitoring system combined with the above design suggestions. The primary purpose of this design is to capture appropriate children’s facial images to improve facial emotion reco- gnition accuracy. The design will adopt the mode of combination of mobile monitors and fixed monitors. We made a design prototype of an infant moni- toring system combined with the above design suggestions. The primary purpose of this design is to capture appropriate children’s facial images to improve facial emotion recognition accuracy. The design will adopt the mode of combination of mobile monitors and fixed monitors (see Figure 3).

Regarding the mobile monitor, GROOVE X, Inc.’s LOVOT is very suita- ble. As an electronic pet, it is equipped with ten or more CPU cores, 20 or more MCUs, and 50 or more sensors, which can scan the home environment and interact with the owner, such as following the owner. Moreover, neural

Design of Toddler Care Equipment Based on Facial Emotion Recognition Technology 555

Figure 3: Multimodal toddler monitoring system.
Original PDF Figure 3:
Figure 4: The effect of monitor.
Original PDF Figure 4:

networks have been applied to analyze faces to enhance interactions with the owner (Dinesen et al., 2021). Furthermore, LOVOT focuses on the all-age market, which also provides a reference for extending the life cycle of infant equipment. However, the mobility of LOVOT is limited, which means it is insufficient to monitor toddlers when it is applied to real families. It needs fixed monitors assistance.

Regarding the fixed monitor. In order to improve the accuracy of facial emotion recognition, the monitor was designed to rotate at a certain angle to search for the toddler’s face automatically. At the same time, it is equip- ped with a voice emotion recognition function to assist judgment (see Figure 4).

By processing, comparing, and comprehensively analyzing the emotion data collected by the two devices, emotion recognition accuracy can be improved because more data samples are collected in this way. After data analysis, a continuous emotional curve will be used to express the children’s emotional state on that day. Children’s emotions will be reported to parents on a daily basis to solve the lag of traditional monitoring equipment, and parents can locate to the point of video timeline more quickly when parents want to check the monitoring.

556 Li and Luo

CONCLUSION AND FUTURE WORK There are two contributions of this issue. First, it investigates the specific situations of emotional neglect in Chinese family. And to use the analytic hierarchy process (AHP) to obtain the high-incidence situations of emotional neglect and assign weights. Secondly, the project explored the application of emotion recognition in real situation (the accuracy of judgment is not high in the real scene). Thought the above analysis, we concluded some problems and give opinions from the perspective of product design Based on these opinions, an toddler monitoring system was offered.

The issue also has the following limitation. First of all, the number of interview samples is insufficient, which leads to the one-sided results of the interviews. More comprehensive interviews will be conducted in the future to complete the program. Second, the sample classification was not compre- hensive enough. In the future, more in-depth investigations will be conducted on the influence of the number of children and the personality of children on emotional neglect. Third, we used Baidu facial emotion recognition, which is a black-box recognition system, which makes debugging very difficult. Finally, the design orientation of LOVOT is not a product for toddlers, which leads to the fact that although the working mode of LOVOT is desirable, it needs to improve for toddler. In the future, we will use LOVOT as a reference to carry out more in-depth design of toddler monitoring equipment.

REFERENCES

Cohen, I., Sebe, N., Gozman, F. G., Cirelo, M. C., & Huang, T. S. (2003, June). Learning Bayesian network classifiers for facial expression recognition both labeled and unlabeled data. In 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. (Vol. 1, pp. I-I). IEEE. Dinesen, B., & og Teknologi, S. (2021). Evaluering af anvendelse af LOVOT-en social

robot til ældre med demenssygdom. Hagendorff, T. (2020). The ethics of AI ethics: An evaluation of guidelines. Minds and Machines, 30(1), 99–120. Hu Hui. (2022). Study on the bearers and development trend of grandparent caring

in rural China.Journal of Chongqing Technology and Business University (Social Science Edition), 1–16. Khan, T. (2021). An Intelligent Baby Monitor with Automatic Sleeping Posture Detection and Notification. AI, 2(2), 290–306. Müller, L. E., Bertsch, K., Bülau, K., Herpertz, S. C., & Buchheim, A. (2019).

Emotional neglect in childhood shapes social dysfunctioning in adults by influ- encing the oxytocin and the attachment system: Results from a population-based study. International Journal of Psychophysiology, 136, 73–80. Martínez, A., Belmonte, L. M., García, A. S., Fernández-Caballero, A., & Morales, R.

(2021). Facial Emotion Recognition from an Unmanned Flying Social Robot for Home Care of Dependent People. Electronics, 10(7), 868. Merter, S., & Hasırcı, D. (2018). A participatory product design process with children with autism spectrum disorder. CoDesign, 14(3), 170–187. Strathearn, L. (2011). Maternal neglect: oxytocin, dopamine and the neurobiology

of attachment. Journal of neuroendocrinology, 23(11), 1054-1065. Wang Na. (2021). The Current Situation and Countermeasures of 0-3 Years Old Infant Education. Contemporary Family education, 27–28.

Design of Toddler Care Equipment Based on Facial Emotion Recognition Technology 557

Witt, A., Brown, R. C., Plener, P. L., Brähler, E., & Fegert, J. M. (2017). Child maltreatment in Germany: prevalence rates in the general population. Child and adolescent psychiatry and mental health, 11(1), 1–9. Ying Di. (2016). Analysis of consumer market purchasing behavior in maternal and

child industry. China Collective Economy (36), 17–18. Zhong Chenxinyi. (2019). Research on Service Design of Infant Intelligent Accompanying System-Taking the Baby-eye Intelligent Accompanying Series as an Example

Cover of the proceedings volume Ergonomics In Design

Explore the complete digital volume

Ergonomics In Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-23-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →