Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Body Movement Support System for Prevent Disability and Promote Progress

1 Japan Advanced Institute of Science and Technology, Nomi, Ishikawa 9231211, Japan ↗
2 Karada-no-Mente, Matsumoto, Nagano 3900816, Japan ↗
3 National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, 3058568, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1004354

Abstract

For knowledge transmission, it is important to construct structured knowledge that clearly describes the knowledge. The purpose of this study is to propose a method of structuring instructional knowledge through the approach of conveying ideal body movements, and to develop a system using the knowledge. To achieve the purpose, we structured the knowledge of ideal motions through interviews and constructed computer-readable instructional knowledge. Then, we created a transmission system to utilize the knowledge and instruct the motions, and a veteran instructor confirmed the feedback from the system. From the results, we found that knowledge structured by interviews can be computer readable and incorporated into the system. The results also showed that new knowledge can be extracted by using the proposed method. The results suggested that the proposed method can clarify the instructor's knowledge and share instructional techniques with others.

1. INSTRUCTION

Due to the rapid aging of the population, Japan’s population will be less than 100 million by 2053, and the percentage of elderly people aged 65 and over will exceed 40% (Uto, 2018). Therefore, people are expected to make exercise a habit in order to maintain and improve their health. How- ever, many exercises require complex physical movements, and it is not easy to make them a habit. Therefore, there is a growing need for instructional methods that promote the improvement of physical movements and prevent disabilities (Yoshida, 2018).

To promote progress and prevent injury in previous studies, physical move- ment instructors such as sports trainers, physical therapists, and coaches of various disciplines have assessed the condition of their clients, including pos- ture, movement, and motor imagery, and have provided voice and muscle and fascia interventions to address these issues (Murakami, 2012)(Maruyama, 2004). Instructors of physical movements in these interventions are often

© 2023. Published by AHFE Open Access. All rights reserved. 16

Body Movement Support System for Prevent Disability and Promote Progress 17

unable to clearly describe their teaching methods because they often teach each complex physical movement sensitively. Therefore, it is difficult to pass on instructors’ knowledge and train instructors.

In order to transfer knowledge, structured knowledge that clearly describes knowledge has been developed. One of the methods for structuring knowl- edge with physical actions is a form of computer readability called CHARM (Ijuin, 2022). Computer readability is the readability of a text on a computer (Nakanishi, 2014). In a previous study (Yoshida, 2022), knowledge of ideal nursing care actions was structured, and data for actions in the knowledge were linked. Then, a method for extracting new knowledge from the knowl- edge and improving the knowledge is proposed. However, this previous study aims at structuring the knowledge of ideal actions, and does not aim at con- structing the knowledge of guidance to approach the ideal actions. Moreover, the development of a system that incorporates the constructed knowledge has not yet been achieved.

Therefore, this study proposes a method of structuring instructional knowledge to bring physical movements closer to the ideal. In addition, we develop a system that incorporates this knowledge and implement instruction. In the proposed method, ideal movements are first structured through interviews, and then computer-readable instructional knowledge is added to the knowledge. Then, the instructor conducts instruction using the constructed knowledge and the instructor’s instruction, and the results are compared to analyse the causes of the differences. By introduc- ing initiatives to add newly discovered knowledge and deficiencies in the existing structured knowledge found by the instructor, a cycle of knowl- edge acquisition and utilization that leads to higher quality instruction is realized.

To validate the proposed method, this paper takes basic physical move- ments in conditioning as an example. Specifically, we structure the knowl- edge used for conditioning by instructors of physical actions and focus on instructional knowledge using language. Conditioning is defined here as the conditioning of the body to a desired state toward a certain goal. With the cooperation of physical movement instructors, ideal movements and instruc- tional knowledge structuring are conducted for basic physical movements based on the proposed method. A system is then constructed based on these structurizations, and the practical feasibility of the proposed system in the field is discussed. The purpose of this paper is to develop a system that incorporates the structured knowledge.

Section 2 describes related research, Section 3 proposes a new method for structuring instructional knowledge, Section 4 presents an experimental method using basic physical movements to verify the effectiveness of the pro- posed method, Section 5 presents the results, Section 6 discusses the results, and Section 7 summarizes and discusses future prospects.

2. PREVIOUS RESEARCH

This chapter reviews related research in the field of knowledge engineering and clarifies the position of this proposal. This research proposes a method

for structuring instructional knowledge in order to make physical actions ideal. Section 2.1 describes the latest video analysis applications. In this study, we propose a method of structuring knowledge by fusing data. Therefore, Section 2.2 describes research on knowledge and data fusion. In addition, Section 2.3 investigates research on knowledge transfer for instructors of physical movements, which is the goal of this study.

2.1 About the Latest Video Analysis Applications

Until now, condition ascertainment and personal training have been based on the intuition and experience of the instructor, and thus there were differences due to experience, skill, and knowledge. In order to make the quality of testing and evaluation uniform, there are many applica- tions that provide mechanical assistance for condition monitoring and coaching.

SportipPro, developed by a venture company from the University of Tsukuba, analyzes with high precision images taken by a smartphone cam- era of posture and joint range of motion, automatically creates training menus based on the results, and also performs before-and-after checks. In addition, Performance AI, developed jointly with SportipPro, analyzes the movements of more than 5 million exercise data and more than 2,000 aca- demic paper data per year, and evaluates every movement and posture. Furthermore, for the purpose of supporting sports education, an application has been developed that provides feedback on professional coaching knowl- edge when a 10-second video shot with a smartphone is registered in the application.

In the field of knowledge engineering, there are systems that support instructors in learning physical movements by structuring their knowledge using motion capture (Kanda, 2012), and to support the practice of tradi- tional Japanese dance, a system that uses openpose as a skeletal estimation AI to provide feedback on the differences between the practitioner’s movements and model movements has been proposed. A system that provides feedback on the differences between the practitioner’s movements and model move- ments using openpose as a skeletal estimation AI has also been proposed to support traditional Japanese dance practice (Kondo, 2022). Furthermore, a system that derives health advice according to acquired physical data based on domain ontology has also been proposed (Izumi, 2008). However, this system collects common instructional content from multiple physical activity instructors and does not target the construction of instructional knowledge, which is more difficult to formalize.

2.2 Knowledge and Data Fusion

The key issue is how to represent the knowledge acquired by interview- ing the instructor. Tamura et al. (Tamura, 1988) have developed a system that enables the representation of knowledge by aggregating the expertise of image processing experts. In addition, a framework for describing human behavior models has been proposed based on the structure of functional ontology proposed by Koromura (Kitamura, 2002), which aims to support

Body Movement Support System for Prevent Disability and Promote Progress 19

the description and management of functional design knowledge (Nishimura, 2011). This model, called CHARM, explicitly describes the purpose of an action.

Using this CHARM, Ijuin et al. (Ijuin, 2022) have successfully constructed an exhaustive tacit knowledge by structuring a hybrid of work procedure knowledge and goal-oriented knowledge to express the important knowledge that caregivers value in the nursing care field or that should be conveyed in the education of new employees. Yoshida et al. (Yoshida, 2022) have also succeeded in elaborating work procedure knowledge by presenting data for structuring knowledge in the independent support care of the elderly. However, these previous studies aimed at structuring ideal movements and did not propose instructional methods or techniques for conveying these movements.

2.3 Technology Transfer Model

Takeda et al. (Takeda, 2010) used motion capture and, using a practical Japanese dance class as an example, stated that a loop occurs in the pro- cess of learners’ understanding, in which they improve their actions based on feedback from the teacher, and then improve their actions based on the next feedback. In addition, the SECI model proposed by Nonaka (Nonaka, 2020) is the basis of knowledge management, which is used to obtain new discoveries by turning tacit knowledge into formal knowledge through the processes of collaboration, representation, linkage, and internalization. The SECI model is the basis of knowledge management, and it is important to have the know-how to circulate it at a deep level. Therefore, this research is also positioned as a system that specifically supports the SECI model, and we believe that the SECI model can be circulated efficiently even if the know-how and experience are limited.

3. PROPOSED METHOD FOR STRUCTURING INSTRUCTIONAL KNOWLEDGE

In this chapter, we propose a method for constructing instructor knowledge of physical movements. The proposed method consists of four steps as shown in Figure 1: x Knowledge structuring of ideal movements is conducted through interviews, y instructional knowledge that is computationally readable is assigned to the knowledge, z actual instruction is conducted, and { efforts to add newly discovered knowledge are introduced. The method then z pro- vides actual instruction and { introduces initiatives to add newly discovered knowledge, thereby realizing a cycle of knowledge acquisition and utiliza- tion that leads to higher quality instruction. Note that x in the proposed method has been proposed in an existing study (Nakanishi, 2014). The pro- posed method is a method that includes not only x but also y instructional knowledge construction, z instructional implementation, and (4) knowledge improvement.

Figure 1: Schematic diagram of the proposed method.
Original PDF Figure 1:

3.1 Construction of Ideal Movement Knowledge

In order to construct knowledge of ideal movements, the following three steps were taken as in the previous study (Nakanishi, 2014). The following steps are conducted by interviewing the instructors regarding the target physical movements.

x-1: To construct the ideal movement knowledge x-2: Collect movement data and improve the ideal movement knowledge based on the analysis results.

x-3: Assign movement data In x-1, the ideal movement knowledge is constructed by interviewing the instructor about the ideal movement. In this study, the ideal movement knowledge is structured using the structured notation shown in Figure 2 and produced using Microsoft PowerPoint. The structuring method is called CHARM, which describes the implementation of lower-layer nodes in order to achieve higher-layer nodes. Actions are basically described by nouns and verbs, with only one verb per node. In Figure 2, in order to achieve the top- level action A, it is necessary to perform actions B and C. In order to achieve action B, it is necessary to perform actions b1 and b2 in sequence. By placing constraints on the way knowledge is described in this way, knowledge can be aggregated for the purpose of this paper.

Figure 2: Structured notation of ideal movement.
Original PDF Figure 2:

Body Movement Support System for Prevent Disability and Promote Progress 21 In x-2, the ideal movement knowledge is improved by recording move- ment data using a motion capture system or other physical movement measurement device and conducting interviews based on the results of the analysis. This is because there may be differences between the words of the instructor and the actual movements. For example, even if the instructor says, “Put the weight on the inside of the left foot,” the actual movement recorded by the measurement device may be more correct if the instructor says, “Put the weight on the center of the left foot. Thus, there are many cases in which the instructor’s words differ from the actual measured data. Therefore, in this step, knowledge that cannot be obtained only from the interviews in x-1 can be collected and improved.

In step x-3, the recorded motion data is added to the ideal motion knowl- edge. As shown by the yellow square in Figure 2, image data is added to the upper left corner of the node using a link function such as PowerPoint. As shown by the gray square, the analysis results of motion capture data and numerical data such as joint angles are added to the lower right corner of the node. This makes the structured knowledge clearer.

3.2 Add Instruct Knowledge

The following three steps are implemented to add instructional knowledge to the knowledge of ideal behavior constructed in x.

y-1: Organize the problem behavior y-2: Extract instructional knowledge y-3: Add instructional knowledge to the knowledge of the ideal behavior In y-1, typical problem behaviors that deviate from the ideal behavior are interviewed and organized. For example, problematic movements that devi- ate from the ideal movement of “standing correctly” may include arching the back, hunching the head back, and one-sided center of gravity. By inter- viewing instructors, typical problematic motions that deviate from the ideal motion are extracted and organized.

In y-2, the instructor’s knowledge of each of the problematic movements identified in y-1 is extracted by interviewing the instructor. For exam- ple, in the aforementioned problem behavior of back bending, multiple images are prepared with the degree of bending changed by several degrees, and instructional knowledge is extracted for each of them. This makes it possible to provide guidance according to the degree of back bending of the client.

In y-3, guidance knowledge is added to the knowledge of the ideal move- ment constructed in x. The instructional knowledge is added in accordance with the structured notation shown in Figure 3. This notation is based on the CHARM described in Figure 2, where instructional knowledge is added to the lower part of the ideal behavior to be performed by the client, and the knowledge is described as a noun + verb. The yellow square in the upper left corner of each node describes the subject of the action, and the green square in the upper left corner of the node of instructional knowledge describes the specific conditions under which the instruction is given. These conditions are described so that they correspond to the instructional knowledge in y-2.

Figure 3: Structured notation of instructional knowledge.
Original PDF Figure 3:

3.3 Conduct Instructions

Guidance is provided for the behavior of several clients by the following steps. This enables the discovery of knowledge for dealing with new cases that did not come to mind during the interviews.

z-1: Implementation of guidance based on the structured knowledge of guidance constructed.

z-2: Implementation of guidance by the instructor z-3: Collection of the results of the two types of instruction In z-1, instruction is conducted using the structured knowledge of instruc- tion that has been constructed. Since the structured knowledge of instruction in z-2 is computer readable, a system for instruction with built-in knowl- edge is constructed. This system has functions such as motion capture to measure the client’s body movements and skeletal detection using images, and it also analyzes joint angles and other information, compares them with the conditions in the instructional knowledge given in y, and outputs the instructional knowledge that matches those conditions. The system and the instructor film the client’s movements so that they can provide instruction for the same movements. The system provides instruction on this video. This sys- tem was developed in order to objectively compare the knowledge structured in the initial interviews with the actual contents of the instruction.

In z-2, the instructor provides guidance to the client’s movements that were filmed in z-1.

In z-3, the contents of the guidance by the system and the contents of the guidance by the instructor are collected. The contents of each are compared, and differences are found.

3.4 Improve Knowledge

The following three steps are implemented to improve knowledge through interviews.

{-1: Analyze the cause of the difference between the two instructional results.

Body Movement Support System for Prevent Disability and Promote Progress 23 {-2: Improvement of knowledge of ideal behavior {-3: Improvement of instructional knowledge In step {-1, the causes of the differences found in step z-3 are analyzed by interviewing the instructors. The fact that the results of the system and the instructor’s instruction differ indicates that we have extracted knowledge that was not discussed in the interviews. There are a variety of possible situ- ations in which the instructor’s knowledge of physical movement may differ, and physical movement is extremely complex. Therefore, it is not possible to extract comprehensive instructional knowledge at the first interview. There- fore, by examining various types of clients, new instructional knowledge is added. In addition, by analyzing the causes of differences in instructional results, the instructor’s metacognition is increased, leading to improvements in knowledge.

Let us describe an example in which new instructional knowledge is extracted by looking at the client’s movements that were not anticipated dur- ing the interview. For example, in y, the instructor’s instructional knowledge was to “extend the knees” when the knees are bent, but in z, the instructor may instruct “pull up the trunk” when the knees are bent in the same way. When we asked the instructor the reason for this, he responded that he would instruct “pull up the trunk” when the knees are bent and the trunk is tilted. In other words, this is an example where detailed conditions that were not obtained in the interview are discovered by instructing a new client.

In {-2, based on the results of the analysis in {-1, the knowledge of the ideal movement is improved. This is a re-implementation of x. For example, the client is newly examined, and the knowledge of the ideal movement of standing correctly is newly added to the knowledge of stretching the trunk up and down.

In {-3, the knowledge of instruction is also improved based on the results of the analysis in {-1. This is a re-implementation of y. For example, in the case of the example in {-1, new conditions are added for the case where the trunk is not tilted and the case where the trunk is tilted.

4. EXPERIMENTS TO VERIFY THE PROPOSED METHOD

This chapter describes an experiment to verify the instructional knowledge structuring method proposed in Chapter 3. The experiment used basic phys- ical movements in conditioning as an example. The author Yamamoto, a physical movement instructor, extracted the balance of the left and right muscles and the client’s characteristics based on information such as body wobble, leaning, and muscle tension when the client performed three types of basic physical movements (Murakami, 2012). Specifically, there are three types of exercises: “one-leg standing,” in which one leg is raised, “squatting,” in which the knee is repeatedly flexed and extended from an upright position, and “lunge,” in which the leg is widely opened back and forth. The instruc- tors provide precise guidance based on the conditions they have identified, which leads to the prevention of injury and the promotion of progress. The proposed method was tested on the knowledge of instructors of these physi- cal movements. Specifically, among the three types of basic body movements,

the one-legged standing posture, in which the axis is particularly important, was taken up. Interviews were conducted with instructors in order to struc- ture their knowledge of ideal movements and instruction. Table 1 shows the number of interviews, dates, time required, and knowledge covered. Inter- views began in September 2022 and were conducted five times until March 2023.

Table 1. Number of hearings, dates, time taken, subject knowledge.

Table available in the original PDF paper.

No. date time embodiment 1 2022.9.30 1h 50m online 2 2023.1.19 0h 50m online 3 2023.3.27 1h 0m offline 4 2023.3.38 1h 0m offline 5 2023.3.29 0h 40m online

4.1 Construction of Ideal Movement Knowledge

In x-1, knowledge construction of the ideal movement was conducted based on the procedures and respective precautions for the client to correctly per- form the one-legged standing posture based on interviews with instructors. A PowerPoint presentation was used for structuring.

In x-2, the one-legged standing posture was recorded using motion cap- ture. One adult male instructor participated in this recording, and one inertial sensor motion capture system was used at 240 Hz. 17 sensors were attached to the head, thorax, upper arm, forearm, hand, pelvis, thigh, lower leg, and foot, wearing a special suit. Figure 4 shows the one-legged standing posture performed by the instructor. Then, to form a dedicated rigid-body linkage model, the subject assumed the specified pose and was calibrated by walking for 15 seconds. The experiment was recorded from the front with an external webcam and from the side with a smartphone camera. The software provided with the camera was used to create animations of the movements.

Figure 4: One-leg standing performed by the instructor.
Original PDF Figure 4:

Body Movement Support System for Prevent Disability and Promote Progress 25

Data analysis was performed focusing on a part of the structured knowl- edge that had been constructed. This allowed us to find differences between the instructor’s statements and the actual recorded data, and to improve the knowledge of the ideal movements.

In x-3, data was added to the structured knowledge that had been partially improved by the motion analysis using the link function of PowerPoint. In this experiment, two types of data were attached to the ideal motion knowledge: numerical data indicating conditions such as angle information and image data.

4.2 Add Instruct Knowledge

In y-1, interviews were conducted with instructors to extract and organize typical problematic movements in the one-legged standing posture.

In y-2, we presented images demonstrating typical problematic move- ments to extract instructional knowledge, explained the situation, and interviewed instructors about their instructional knowledge to improve the situation. Specifically, three types of typical problematic movements were targeted: left-right tilt of the trunk, front-back tilt of the trunk, and flex- ion of the knee joint of the supporting leg. Figure 5 shows an example of a demonstration of a problematic movement related to the left-right tilt of the trunk.

In y-3, the knowledge of the ideal movement constructed in x was added to the instructional knowledge extracted in y-2.

Figure 5: Example demonstrating problematic movements related to left-right tilt of the trunk.
Original PDF Figure 5:

4.3 Conduct Instructions

Three male university students in their 20s performed the one-legged standing posture as collaborators for the experiment. They stood on one leg three times in the front-facing position and three times in the side-facing position,

in the position shown in Figure 6. The video was recorded by an external web camera at 1280p at 30 fps.

Figure 6: Camera and client positioning.
Original PDF Figure 6:

In z-1, guidance was provided by a system with built-in guidance knowl- edge. This system captured the movements of the Klein doll using an external web camera, performed skeletal detection and analysis, and output the instructor’s knowledge in real time by matching it with the structured knowl- edge of the instruction. KAPAO (William, 2022) was used for skeletal detection; KAPAO performs posture estimation by simultaneously detecting and fusing human pose objects and key point objects.

When using this system, the camera is placed 220 cm from the client and 120 cm from the ground, as shown in Figure 6. This is because KAPAO performs two-dimensional skeletal detection, so the positional relationship between the camera and the client must be fixed. In fact, a slight change in the position or angle of the camera may result in skeletal detection results that differ by several to several tens of degrees.

Figure 7 shows the interface of the developed guidance system. The left side of the screen is a video of an ideal movement, and the right side is a video of a movement performed by the client. The ideal movement side shows the actual joint angle (in degrees), and the client side shows the joint angle difference from the ideal movement (in degrees). This joint angle difference is obtained as “joint angle of ideal movement - joint angle of client. Based on the instructor’s knowledge, a threshold value is set for motion instruction when the joint angle difference exceeds a specific value. When the threshold is exceeded, the joint angle difference on the client side changes to red.

For example, in Figure 7, the client’s right knee joint angle is displayed in red, meaning that the client’s posture has exceeded the set threshold value. The system was used to input the videos of the three participants in the experiment, which had been taken in advance, and to conduct the instruction.

In z-2, the instructor provided guidance. The videos recorded in z-1 were sent to the instructor, and explanations of the situation of the three collab- orators’ one-legged standing movements and the contents of the instruction

Body Movement Support System for Prevent Disability and Promote Progress 27 to improve the situation were collected. In z-3, the results of the instruction by the system and by the instructor were collected and compared.

Figure 7: Interface of instruct system.
Original PDF Figure 7:

4.4 Improve Knowledge

In {-1, we analyzed the causes of the discrepancy between the system and the instructor’s instructional results. Comparing the results of the instruc- tion collected in z-3, we found that the instruction provided by the system and the instructor differed greatly. Therefore, we conducted two 1-hour, 40-minute interviews with the instructors to analyze the causes of the discrepancies.

In {-2, based on the results of the analysis in {-1, the knowledge of the ideal movement and the knowledge of the instructor were improved.

5. RESULT

5.1 Construction of Ideal Movement Knowledge

In x-1, the instructors were interviewed and knowledge structuring of the ideal one-legged standing position was conducted. The total number of struc- tured knowledge of the ideal motion was 16 nodes, and the results are shown in Figure 8.

In x-2, the knowledge of the ideal motion was improved based on the results of the analysis of the data recorded by motion capture. As a result, one node was improved. In the knowledge structuring based on the prelimi- nary interview, the instructor stated that “it is important that the axes from the head to the heel are connected in a straight line” at the moment when the one-legged stance movement is determined, as shown by the red line in

Figure 9. However, actual data analysis revealed that they were not com- pletely connected. Figure 10 shows the change in the head when the right foot is raised. The figure is a floor (x-y plane) projection, and the z-axis rep- resents the height direction of the test subject. According to the structured knowledge obtained from the preliminary interviews, the green line repre- senting the position of the head should intersect the red line representing the position of the left foot, but we confirmed that there was a difference of approximately 4 cm in the x-axis direction. From this result, the knowledge that “the axis from the head to the heel is connected to a straight line” indi- cated by the light yellow balloon in Figure 8 was improved to “the head is brought close to the extension of the axis of the heel” as shown by the green circle.

Figure available in the original PDF paper.

In x-3, the joint information of the ideal motion obtained from the data analysis was assigned to the four nodes (gray squares in Figure 8). In addition, image data was added to each of the four nodes using the link function of PowerPoint. Clicking on the yellow button displays the model image.

Figure 8: Structured knowledge of ideal one-leg standing.
Original PDF Figure 8:
Figure 9: Image of the axis of the instructor at the hearing. The axis from head to heel is connected in a straight line.
Original PDF Figure 9:

Body Movement Support System for Prevent Disability and Promote Progress 29

Figure 10: Head changes when the right leg is raised. Floor (xy-plane) projection, z-axis is in the direction of height.
Original PDF Figure 10:

5.2 Add Instruct Knowledge

In y-1, we summarized the typical problematic movements in the one-legged standing posture. The results showed that there were three types of problem- atic movements in the one-legged standing posture: tilting of the trunk from side to side, bending of the back, and bending of the knee of the supporting leg.

In y-2, we extracted instructional knowledge from interviews with instruc- tors regarding the three types of problematic movements obtained in y-1. Table 2 shows the explanation of the situation and the instructional knowl- edge to improve the situation for the photographs in Figure 5 in which the left-right tilt of the torso was changed little by little. Each number corre- sponds to a series of images in Figure 5, with the second row representing the situation in the image, the third row representing the tilt of the trunk (in degrees), and the fourth row representing the knowledge needed to solve the problem.

In y-3, based on the results of y-2, knowledge to teach the correct way to raise and lower one leg was assigned to the knowledge of the ideal action, and the results are shown in Figure 11. 3 types of problematic actions were used as conditions, and the corresponding instructional knowledge was assigned to them. In this experiment, 14 new nodes of instructional knowledge were added.

5.3 Conduct Instructions

In z-1, the system with built-in structured knowledge of instruction was used to provide instruction to three clients in the one-legged standing posi- tion. Table 3 shows the instructional knowledge output from the system and the actual instructional knowledge by the instructors. The three collabora- tors were designated a, b, and c. There are two types of knowledge output:

the situation described to evaluate the client’s behavior, and the instructional knowledge to improve it. For example, the postural evaluation for the one- legged standing posture performed by a was “the support from the pelvis down looks good” as the situation, and the instructional knowledge “be con- scious of extending the left shoulder blade” was output in order to improve the problem.

Table 2. Results of the extraction of instructional knowledge through interviews (as an example, the left-right tilt of the torso).

Table available in the original PDF paper.

instruction message

image number

situation inclination of the trunk

−10 keep your body straight

x It seems that the muscles related to the left iliopsoas and hip joints are weak. The area around the shoulder blade seems to be stable.

−3 Extend your right shoulder blade upwards

y The muscles related to the left iliopsoas muscle and hip joint seem to be weak. The shoulder blade area seems to be stable. The center of gravity is outside.

z It feels good, but there is a feeling that the right leg is externally rotated

2 Remember to stretch your upper body

0 Let’s have an image of lifting from the base of the hip joint. Also, make sure that the right trunk is not crushed. { The left trunk is crushed by the center of gravity of the outer leg. Trunk is out during hip extension | I have a problem with my right shoulder blade.

12 Be conscious of stretching the left shoulder blade

4 Make sure your trunk doesn’t collapse when you lift your leg. Also, lift your shoulder blades } The support from the pelvis down looks good, and the relationship on the right side is good

Figure 11: Structured knowledge to teach how to correctly raise and lower one leg.
Original PDF Figure 11:

In z-2, the instructor described the situation of the three participants in the one-legged standing posture and provided guidance to improve it. The

Body Movement Support System for Prevent Disability and Promote Progress 31

videos were shot from the front and the side, and were sent without joint angle information.

In z-3, the results of instruction by the system and by the instructor were collected and compared. As shown in Table 3, the content of the instruction by the collaborator c was consistent with that of the system.

Table 3. Instruct by system and instructor.

Table available in the original PDF paper.

5.4 Improve Knowledge

In {-1, we analyzed the differences between the system and the instructor’s instruction. As a result, new instructional knowledge was extracted by look- ing at the client’s movement, which was not assumed during the interview. The system output the instructional knowledge of “extend the trunk” when the client’s trunk tilted outward by 4

  • and the client had a lateral center of gravity, but the instructor actually instructed the client to “extend the knees a little more. When we interviewed the instructor about the reason for the discrepancy between the system and the instructor’s instruction, we found that the instructor instructed “extend the knees a little more” when the trunk was tilted outward by 4
  • to 7
  • and the knees were bent by 17
  • or more. As shown in Figure 12, instructors analyzed the situation from multiple per- spectives and provided guidance accordingly. For example, they identify the internal characteristics of “external center of gravity” and “slightly external center of gravity” based on the external characteristic of the trunk leaning outward. The thresholds of these internal characteristics were determined through interviews. The “outward center of gravity” is when the trunk is tilted outward by 8
  • or more, and the “slightly outward center of gravity” is when the trunk is tilted outward by 4
  • to 7
  • . Furthermore, it was found that the instruction “Hold the image of stretching the trunk straight up” was out- put by combining the two internal characteristics of “outer center of gravity” and “backward center of gravity”.

In {-2, it was assumed that the knowledge of the ideal action would be improved based on the results analyzed in {-1, but no improvement of the ideal action was observed in this experiment.

In {-3, the instructional knowledge was improved based on the results of the analysis in {-1. Figure 13 shows the new improved instructional knowledge based on the results of the analysis. The six instructional knowl- edge shown in orange in the figure were improved/added. The total number of nodes in the structured knowledge is 33, and the number of nodes

in the instructional knowledge is 17. Compared to the knowledge before improvement, the number of instructional knowledge nodes increased by 3.

Figure 12: Causes and effects of instructional knowledge.
Original PDF Figure 12:
Figure 13: Structured knowledge of new instruction.
Original PDF Figure 13:

6. CONSIDERATION

The results of Section 5.3 enabled us to actually build instructional knowl- edge into the system, and we conducted instruction for the client’s one-legged standing posture. The results show that the proposed method can be operated under specific conditions, although there is room for further study regarding its general computer readability.

In addition, new knowledge was extracted by comparing the results of the system and the instructor’s instruction, and six instructional knowledge items were improved/added. We will continue to verify whether it is possi- ble to further structure and add to the instructor’s knowledge in a highly computer-readable form by continuing the cycle of knowledge acquisition and utilization conducted in the validation experiments.

In the verification experiment of the proposed method, specific angle infor- mation, which was not included in the previous study (Yoshida, 2022), was

Body Movement Support System for Prevent Disability and Promote Progress 33 also added with respect to the ideal movement knowledge in x. However, the ideal criteria may differ depending on the client’s situation, goals, and history, such as whether the client is an athlete with strong upward mobility or an older person aiming to prevent disabilities. In addition, it is also possible that the client’s instructional knowledge is based not only on the client’s situa- tion, but also on how he or she tends to receive the kind of instruction that is given. We believe that the proposed method will build up knowledge of how to respond to such detailed situations through repeated implementation of steps x through {.

In addition, the instructor instructs in the same way as in Fault Tree Anal- ysis (FTA)(Suzuki, 1988), a top-down analysis method that searches for the causes of undesirable events, such as the “outward tilt of the trunk” and “outward center of gravity” as shown in the external characteristics on the left side of Figure 11. The students are also instructed to remove the causes of such events. Building knowledge of causal relationships that fail is also considered to be useful in clarifying knowledge.

7. CONCLUSION

In this paper, we proposed a method for structuring instructional knowledge. A system with built-in instructional knowledge was developed. Specifically, we took up basic physical conditioning movements and demonstrated that the proposed method can add instructional knowledge to the knowledge of ideal movements and further improve it. The structured instructional knowl- edge was computer-readable and could be incorporated into the proposed system to provide instruction. Three clients used the system to examine the results of the instruction, which revealed additions and improvements to the instructional knowledge.

We believe that this method not only clarifies the instructor’s knowledge, but also makes it possible to pass on and share teaching skills with others. In addition, the system can assist the instructor by incorporating the knowledge gained from the instructor, thereby improving the instructor’s performance.

In the future, we would like to investigate whether the proposed system can be applied to exercise movements in other sports, daily life movements such as eating and bathing, work movements at manufacturing sites, and production movements in traditional crafts. In addition, we would like to verify the effectiveness of the proposed system not for individual instructors but for multiple instructors.

ACKNOWLEDGMENT

Part of this work was supported in part by the New Energy and Industrial Technology Development Organization (NEDO) JPNP18002. A part of this work was supported by JST Moonshot-type R&D project JPMJMS2237_5.

I am grateful to Assistant Professor Shun Kurokawa of the Japan Advanced Institute of Science and Technology and Professor Emeritus Ryuichi Oka of the University of Aizu for their support in writing this paper. We also thank

Mr. Naoki Koyama of the Human Extension Research Center, National Insti- tute of Advanced Industrial Science and Technology (AIST) for his guidance in motion measurement and data analysis, and three graduate students for their help in conducting preliminary measurements. We would like to express our gratitude to them.

REFERENCES

AI posture and motion analysis application from the University of Tsukuba- SportipPro: https://lp.sportip.ai/ AI X Personal Training Gym “Performance AI” to “Visualize” Your Body: https:

//prtimes.jp/main/html/rd/p/000000022.000035944.html Ijuin, Koki. Kobayakawa, Maiko. Ino, Nami. Takuichi, Nishimura. (2022). Methodology of structuring knowledge by discussing purpose of each action in procedure- based knowledge graph, Information Processing Society of Japan Volume 63, No. 1, pp. 104–115. Izumi, Satoru. Kato, Yasushi. Takahashi, Kaoru. Suganuma, Takuo. Shiratori, Norio.

(2008). Proposal of an ontology-based health support system and its evaluation, Information Processing Society of Japan Volume 49, No. 2, pp. 822–837. Kanda, Kenichi. Yamagata, Keisuke. Omoto, Yoshimasa. Nishida, Toyoaki. (2012).

Development the physical operation acquisition supporting system using expert’s knowledge structure, Highlights of The Japanese Society for Artificial Intelligence. Kitamura, Yoshinobu. Mizoguchi Riichiro. (2002). A framework for systematization of functional knowledge based on ontological engineering, Information Processing Society of Japan Volume 17, No. 1, pp. 61–72. Kondo, Shogo. Tsukada, Koji. (2022). Proposal for a system to support the practice

of Onikenbai, interaction symposium 2022. Maruyama, Hitoshi. (2004). Trends in Rehabilitation for the Elderly, the society of Physival Therapy Science Volume 19, No. 3, pp. 163–167. Murakami, Narumichi. Kodama, Yuzi. Aoki, Hironari. (2012). aggressive conserva-

tive therapy, Kobundo. Nakanishi, Hidehiko. (2014). From human readability to machine readability, Information Processing Society of Japan Volume 57, No. 3, pp. 149–156. Nishimura, Satoshi. Kitamura, Yoshinobu. Sasazima, Munehiko. Wiloiamson,

Akiko. Kinoshita, Chikako. Hattori, Kanetoshi. Mizoguchi, Riichiro. (). CHARM: convicing human action rationalized model, Highlights of The Japanese Society for Artificial Intelligence. Nonaka, Ikujiro. Takeuchi, Hirotaka. (2020). knowledge creation company, Toyo Keizai Shinposha. Performance AI: https://lp1.performance-ai.com/a SpoGasha – sports education support: https://spogasha.com/ Suzuki, Junjiro. (1998). FMEA and FTA Implementation Methods - Reliability and

Safety Analysis and Assessment, Nikkagiren Shuppansya. Takeda, Yoko. Marushige, Mieko. (). Qualitative study of effects of feedback loops supported by information technology, Information Processing Society of Japan research paper. Tamura, Hideyuki. Sato, Hiroaki. Sakagami, Katsuhiko. Kubo, Humio. (1988).

DIA - Expert System and Its Knowledge Representation Scheme, Information Processing Society of Japan Volume 29, No. 2, pp. 199–208. Uto, Masaaki. (2018). Examining environmental risks from the viewpoint of depoputation and infrastructure issues, Journal of the National Institute of Public Health Volume 67, No. 3, pp. 306–312.

Body Movement Support System for Prevent Disability and Promote Progress 35

William, McNally. Kanav, Vats. Alexander, Wong. John, McPhee. (2022). Rethinking Keypoint Representations: Modeling Keypoints and Poses as Objects for Multi- Person Human Pose Estimation, arXiv: 2111.08557v4. Yasuyuki, Yoshida. Ino, Nami. Nishino, Takashi. Saito, Takaya. Nishimura,

Takuichi. (2022). Practice and analysis of knowledge-data integration in self- reliance support for elderly using remote technology, Information Processing Society of Japan Volume 63, No. 1, pp. 116–128. Yoshida, Yasuyuki. Zilu, Liang. Nishimura, Satoshi. Konosu, Hisae. Nagao, Tomoka. Nishimura, Takuichi. (2018). Quality Evaluation For Sports Coaching -Evaluate Trunk Torsion by Mobile Terminal, Information Processing Society of Japan Volume 59, No. 2, pp. 591–601.

Cover of the proceedings volume Emerging Technologies in Healthcare and Medicine

Explore the complete digital volume

Emerging Technologies in Healthcare and Medicine

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-92-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →