Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Combining Motion Capture with Vibrotactile Feedback for Real-Time Posture Correction

University of Antwerp, Department Product Development, Faculty of Design Sciences, Antwerp, Belgium ↗
Publication year
Digital Object Identifier10.54941/ahfe1001906

Abstract

Container lashers are at a significant risk of developing musculoskeletal diseases (MSDs) when working at port facilities. Repetitive strain injuries (RSIs) to the back, shoulders, wrists, and hands, in particular, are widespread. This work investigates the ability of a closed-loop vibrotactile motion guidance (VMG) system to teach an ergonomics-focused approach. The taught technique was developed for tensioning and loosening turnbuckles, an important step in container lashing. During five sessions, two groups, each with three participants, were observed. Participants' initial ability was tested in a baseline session. During this session, participants only receive auditory feedback. A VMG device is used to instruct the experimental group during the next three sessions. Traditional auditory feedback is used to teach the control group. Finally, neither group will wear the VMG device during the follow-up session. The findings of this study suggest that both VMG and auditory feedback training are effective training strategies for reducing postural error state (Wilcoxon Signed-Rank, p < 0.05). However, results suggest that VMG does not provide a significant error state reduction compared to auditory feedback training (Mann-Whitney, p > 0.05).

INTRODUCTION

In recent years, the maritime industry has grown more aware of container lashers poor occupational health. One cause is technique, as a gap was disco- vered between the learned techniques and those applied on the work floor. A second cause is the workspace, as workers have had to adopt awkward body postures due to railings, and other obstructive elements. Other injuries such as impact, pinching, and falling (CommercialVesselsnz, 2021; BC Maritime Employer Association, 2018) are also oftentimes caused by the harsh work- space. However, redesigning this workspace and gear is hard to actualize due to highly standardized methods, which are not defined by the employer. A third cause are psychosocial factors, as container lashers often work alone on a two-person job, increasing the likelihood of injury (Zelck et al, 2020).

Internal documentation disclosed that repetitive strain injuries to the wri- sts, hands, back and shoulders are most common. These injury types are

© 2022. Published by AHFE Open Access. All rights reserved. 107

connected to the various lifting operations contained within the activity. The average load of one lifting operation is approximately 21.6 kg, while each lashing, including attachment and detachment, requires a minimum of twe- lve lift operations. Furthermore, one individual might lash 75 or un-lash 250 lashing bars during one shift (Zelck et al., 2020; Van Gastel, et al., 2021). This is indicative of the major roll that repetitiveness plays.

Due to these findings and observations that the activity of loosening and tensioning lashing bars was identified as a major cause of MSDs and RSIs to the wrists. Workers were specifically noticed using a harmful technique, repeatedly overextending and overflexing their wrists.

To solve the occurrence of long term injury as a result of MSD or RSI, new methods for biomechanical risk assessments supported by digital human models have been proposed (Zelck et al, 2020; Van Gastel, et al., 2021). However, risk assessment alone does not mitigate injury. According to the Occupational Safety and Health Administration (OSHA, 2016), managing occupational health is achieved through finding and fixing hazards before they cause injury or illness. One of the suggested solutions is education and training. In this paper vibrotactile feedback is used as a feedback method to augment worker’s awareness of wrist postures during training.

Vibrotactile feedback refers to stimulations given to us and received by our sense of touch through modalities such as vibration. Activities where vibrotactile feedback is supported by experimental observations are posture correction (Bark et al., 2011; Ying and Morrell, 2010), rehabilitation trai- ning of the upper limb (Kapur et al., 2009), spatial guidance (Meier et al., 2015), aid for vestibular balance disorders (Sienko et at., 2013), gait (Crea et al., 2016), human-robot collaboration (Casalino et al., 2018), VR (Louison et al., 2015), AR (Zhu, Cao and Cai, 2020), prosthetics feedback (Chen, Feng and Wang 2016), and sports training (Alahakone and Senanayake, 2009). Additionally, it provides a discreet and private feedback channel that avoids stigmatizing gear setups (Rantala et al., 2017).

Notably, Van Breda et al. (2017) assessed the notion that vibrotactile feed- back would improve motor learning and sports performance and found no evidence that suggests vibrotactile feedback is effective for acquiring or lear- ning new motor skills in sports training. However, another study observed a decrease in postural errors by 20% and an accelerated learning rate of 7% through vibrotactile feedback (Lieberman and Braezeal, 2007). Nevertheless, there is little evidence of this benefit materializing outside of research lab settings.

In this study, automated vibrotactile motion guidance’s (VMG) effective- ness in helping maritime workers achieve fewer postural error states during lashing and un-lashing is assessed through comparison with traditional audi- tory feedback. A device was developed that combines vibrotactile feedback with motion parameters gathered from a motion capture system Xsens (MVN Awinda, Netherlands). This system is developed with the aim to guide users towards more neutral wrist postures in the sagittal plane. In return, decreasing repetitive strain injuries (RSIs) and musculoskeletal disorders (MSDs).

Combining Motion Capture with Vibrotactile Feedback for Real-Time Posture Correction 109

Table 1. The four predefined reference conditions that the VMG system uses to determine a postural errors states, and the corresponding responses when one condition is met.

Table available in the original PDF paper.

Reference condition Response WFlexionLeft ≥40

  • Vibration to the palm side of the left hand. WExtensionLeft ≤−45
  • Vibration to the knuckle side of the right hand. WFlexionRight ≥40
  • Vibration to the palm side of the left hand. WExtensionRight ≤45
  • Vibration to the knuckle side of the right hand. 1) Positive values refers to flexion, 2) Negative values refer to extension.

METHOD

Ergonomic Technique A reference wrist posture defined by joint angles had to be established to determine the occurrence of postural error states. In general, the reference posture should be as neutral as possible, to reduce repetitive high load pres- sure on the carpal tunnel (Rempel, Keir and Bach, 2018). This is achieved by limiting the wrist’s freedom of movement in the sagittal and transverse plane towards zero while maintaining the ability to perform the action. The technique that was established during this study focusses on movements in the sagittal plane, namely flexion and extension angles of 40° and 45°, rela- tive to a straightened wrist. These joint angle parameters were based on a balance between performance and ergonomics. Notably, the parameters for error state feedback in extension are higher than those for flexion. The sligh- tly higher tolerance was established during a pilot test where the participant was unable to keep wrist extensions under 45° while screwing the turnbu- ckle. Thus, while decreasing the value closer to zero delivers a more desirable neutral posture, it also results in a high difficulty curve as the reference posture is harder to maintain. Through vibrational feedback received from a VMG system, users gain awareness when exceeding a reference parameter, augmenting their ability to adjust their posture accordingly.

Vibrotactile Motion Guidance Closed-loop feedback control of wrist postures requires real-time joint angle monitoring. In this study, measurements are received through a wearable motion capture system Xsens (MVN Awinda, Netherlands). The advan- tage of a closed-loop collaborative feedback control system is that when the system detects a non-desirable reference condition, in this case, high wrist flexion or extension - a feedback response can be delivered until the desired flexion or extension is achieved by the user.

The four non-desirable reference conditions, and resulting feedback responses are shown above (see Table 1). While one or more conditions are met, the corresponding haptic communication is given through vibrational motors placed on the knuckle and palm side of either hand. As the proposed device is based on error state feedback the system will only provide feed- back while one or more conditions are met. Lastly, the push principle was

used, meaning that the subject should move away from the direction of the stimulus.

Participants Participants were included after satisfying following criteria: 1) no familiarity with container lashing, 2) no prior or current RSIs, 3) no prior or current MSDs, 4) informed consent when the nature and any risk associated with this study were provided. In this study six participants were included: 6 males, age 24,16 ± 0,41 years, body mass 77,83 ± 11,32 kg; height 1,82 ± 0,07 m). Participants were divided into two groups of similar skill based on a pilot session: 1) experimental assisted motion guidance group, 2) auditory control group.

Experiment In this experiment the efficiency of VMS in reducing postural errors is compa- red to a typical auditory training method. Both groups’ participants are told to apply a strategy that focuses on neutral wrist postures. Container lashing was simulated by vertically securing a turnbuckle with a vice-bench.

These observations lasted five weeks and included the following sessions: 1) baseline test (BL), 2) training session one (T1), 3) training session two (T2), 4) training session three (T3), and 5) follow-up test (FU). Each session, par- ticipants fully turned the turnbuckle up and down three times, performing on average 720 turning actions in which postural error states could occur. The baseline test established the starting skill levels of both groups. Neither group wore the VMG equipment during the baselines test and simply recei- ved audio instructions. The same instructor gave standardized instructions to all participants. Three weekly training sessions followed, with the VMG and auditory groups training utilizing the VMG system and auditory training instructions, respectively.

To control the variable obtrusiveness, both groups wore the VMG and Xsens (MVN Awinda, Netherlands) modules. upper body modules during the three training sessions (T1, T2, and T3). The VMG system was turned off for the auditory group.

An algorithm, written in Processing (Processing, Processing, USA), logged occurrences where the actual wrist position did not reflect the conditions obtained from the reference position, defined as the variable: postural error state.

Statistical Analysis After three training sessions with a VMG system, the experimental group should be able to perform with much fewer errors than the control group. The average amount of postural error states for both groups is plotted for each weekly session.

Due to the small sample size, differences in postural error state reduction between baseline and follow-up sessions are examined using two non-parametric tests. The Wilcoxon signed-rank test is used to deter- mine a significant reduction in postural error states within groups, and

Combining Motion Capture with Vibrotactile Feedback for Real-Time Posture Correction 111

Figure 1: The number of errors as a function of the week during the experiments with VMG and auditory groups. Average (avg) values during the 5-week observation are shown.
Original PDF Figure 1:

a Mann-Whitney test to analyze the rate of reduction between groups (Peeters, 2021).

RESULTS

Error States Comparisons Following observations from a pilot session, groups were created on the assumption of equal skill. The baseline test, however, demonstrates that there was a considerable skill variance during the initial baseline test (see Figure 1). Nonetheless, at T1 both groups performed more or less equally, averaging ± 25 postural error states. Separation started to occur in T2, T3, and follow up (FU).

Both groups were able to reduce their average postural error states between baseline (BL) follow up (FU) sessions. The VMG group went from a baseline of ± 82.67 postural errors to ± 1 during the follow up (FU) sessions. The auditory control went from ± 138.33 postural error states to ± 10.33 during the follow up (FU) sessions. For both groups the statistical significance of this reduction was tested using a non-parametric Wilcoxon signed rank test. This test resulted in a p value of 0.028, p < 0.05. This statistical analysis suggests that the reduction in error rate for both groups is statically significant, and that the VMG system as well as auditory feedback are able to reduce postural errors between baseline (BL) and follow up (FU) observations.

On average the VMG group performed better in the final follow up (FU), averaging ± 1 postural error state, compared to ± 10.33 postural error states for the auditory control group. These figures suggest that the VMG system is better able to reduce postural error rates after three training ses- sions. However, the statistical significance of this difference is tested using the non-parametric Mann Whitney test and resulted in a p value of 0.275,

p > 0.05. Therefore, the main hypothesis is rejected. There is no evidence that suggests that there is a significant difference in postural error state reduction between both groups.

DISCUSSION

This study suggests that both vibrotactile and auditory feedback are effective training methods, as both methods were able to significantly reduce postural error states after three training sessions.

However, the main hypothesis is rejected. There is no evidence to sug- gest that vibrotactile motion-guidance gives a significant learning advantage compared to traditional auditory feedback after three training sessions.

While not supported by quantitative data, the experimental group’s par- ticipants were able to share more insights in how to perform the operation more safely. One possible explanation is that the experimental group’s par- ticipants were able to validate subtle changes in the provided technique, as they received feedback when the adaptation did not have the desired effect.

Future research should investigate the error state reduction ceiling for both VMG and traditional training by lowering the parameters for flexion and extension further. This is proposed as the VMG group reduced their error states to a point where almost no improvement could be made for all participants of that group (see Figure 1).

Lastly, future studies should evaluate the capacity of VMG to provide a training method without the need for a professional teacher’s presence. As a result, training sessions could last longer which could improve postural awa- reness as a result of increased training time. In this study, each participants took on average 11:04 ± 1.11 min to complete one session.

The limitation to this experiment is that it does not accurately simulate the container lashing environment and thus does not represent real-life con- ditions. Results should be taken with caution due to the limited sample size and the usage of only flexion and extension error states as outcome measu- res. Postural errors alone do not fully represent improvement towards a more ergonomic technique.

CONCLUSION

Results of this study suggest that both VMG and auditory training methods are effective in reducing wrist-related error states. However, the main hypo- thesis that VMG would offer a significant reduction in postural error states compared to traditional auditory training is rejected. There is currently no evidence to suggest that VMG significantly improves workers’ awareness of unergonomic wrist postures compared to traditional auditory feedback.

REFERENCES

Alahakone, A. and Senanayake, S., 2009. Vibrotactile feedback systems: Current trends in rehabilitation, sports and information display. 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

Combining Motion Capture with Vibrotactile Feedback for Real-Time Posture Correction 113

Bark, K., Khanna, P., Irwin, R., Kapur, P., Jax, S., Buxbaum, L. and Kuchenbecker, K., 2011. Lessons in using vibrotactile feedback to guide fast arm motions. 2011 IEEE World Haptics Conference. BC Maritime Employers Association, 2018. BCMEA Container Lashing Orientation

Video -Lashing 101. [video] Available at: https://www.youtube.com/watch?v=8C B6mKabgqQ&list=PLZr_HrwOYIpHeEh9t1ei-ytfJD29cEX1M&index=24&t= 1s [Accessed 11 December 2020]. Casalino, A., Messeri, C., Pozzi, M., Zanchettin, A., Rocco, P. and Prattichizzo, D., 2018. Operator Awareness in Human–Robot Collaboration Through Wearable Vibrotactile Feedback. IEEE Robotics and Automation Letters, 3(4), pp. 4289–4296. Chen, B., Feng, Y. and Wang, Q., 2016. Combining Vibrotactile Feedback with

Volitional Myoelectric Control for Robotic Transtibial Prostheses. Frontiers in Neurorobotics, 10. CommercialVesselsnz, 2012. Lashing Containers - Maritime New Zealand. [video] Available at: https://www.youtube.com/watch?v=pUle32fSfpw&list=PLZr_Hrw OYIpHeEh9t1ei-ytfJD29cEX1M&index=23&t=125s [Accessed 11 December 2020]. Crea, S., Edin, B., Knaepen, K., Meeusen, R. and Vitiello, N., 2016. Time-Discrete

Vibrotactile Feedback Contributes to Improved Gait Symmetry in Patients With Lower Limb Amputations: Case Series. Physical Therapy, 97(2), pp. 198–207. Kapur, P., Premakumar, S., Jax, S., Buxbaum, L., Dawson, A. and Kuchenbecker, K.,

2009. Vibrotactile feedback system for intuitive upper-limb rehabilitation. World Haptics 2009 - Third Joint EuroHaptics conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems,. Lieberman, J. and Breazeal, C., 2007. TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning. IEEE Transactions on Robotics, 23(5), pp. 919–926. Louison, C., Ferlay, F., Keller, D. and Mestre, D., 2015. Vibrotactile feedback for

collision awareness. Proceedings of the 2015 British HCI Conference,. Meier, A., Matthies, D., Urban, B. and Wettach, R., 2015. Exploring vibrotactile feedback on the body and foot for the purpose of pedestrian navigation. Proce- edings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction,. Occupational Safety and Health Administration, 2016. Recommended Practices for

Safety & Health Programs in Construction. OSHA, p. 24. Peeters, T., 2021. Design guidelines for vibrotactile motion steering. Ph.D. University of Antwerp. Rantala, J., Majaranta, P., Kangas, J., Isokoski, P., Akkil, D., Špakov, O. and Rai-

samo, R., 2017. Gaze Interaction With Vibrotactile Feedback: Review and Design Guidelines. Human–Computer Interaction, 35(1), pp. 1–39. Rempel, D., Keir, P. and Bach, J., 2008. Effect of wrist posture on carpal tunnel pressure while typing. Journal of Orthopaedic Research, 26(9), pp. 1269–1273. Sienko, K., Balkwill, M., Oddsson, L. and Wall, C., 2013. The effect of vibrotactile

feedback on postural sway during locomotor activities. Journal of NeuroEngine- ering and Rehabilitation, 10(1). Van Breda, E., Verwulgen, S., Saeys, W., Wuyts, K., Peeters, T. and Truijen, S., 2017.

Vibrotactile feedback as a tool to improve motor learning and sports performance: a systematic review. BMJ Open Sport & Exercise Medicine, 3(1), p. e000216.

Van der Linden, J., Johnson, R., Bird, J., Rogers, Y. and Schoonderwaldt, E., 2011.

Buzzing to play. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems,. Van Gastel, A., Scataglini, S., Zelck, S., Denteneer, L., Bossche, H. and Verwulgen, S.,

2021. Towards Wearable Technology Assisting in Training and Risk Assessment Against Musculoskeletal Disorders for Maritime Workers. Advances in Simulation and Digital Human Modeling, pp. 368–376. Ying, Z. and Morrell, J., 2010. A vibrotactile feedback approach to posture guidance.

2010 IEEE Haptics Symposium,. Zelck, S., Verwulgen, S., Denteneer, L. and Scataglini, S., 2020. Digital Human Modeling as a Risk Assessment Tool for Maritime Workers. Lieberman, J. and Breazeal, C., 2007. TIKL: Development of a Wearable Vibrotactile Feedback Suit for Improved Human Motor Learning. IEEE Transactions on Robotics, 23(5), pp. 919–926. Zhu, L., Cao, Q. and Cai, Y., 2020. Development of augmented reality serious games

with a vibrotactile feedback jacket. Virtual Reality & Intelligent Hardware, 2(5), pp. 454–470.

Cover of the proceedings volume Digital Human Modeling and Applied Optimization

Explore the complete digital volume

Digital Human Modeling and Applied Optimization

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-22-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →