Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Multimodal haptics perception of underwater flow for scuba diving safety training

Kogakuin University, 1-24-2 Nishishinjuku, Shinjuku-ku, Tokyo, Japan ↗
Publication year
Digital Object Identifier10.54941/ahfe1002639

Abstract

Water accidents require intuitive decisions and training for such decisions because of the short time between involvement and death. The goal is to use engineering techniques to train and prevent accidents that require split-second decisions or are caused by human actions or scenarios that are difficult to anticipate.In recent years, the number of fatalities in traffic accidents and disasters has decreased significantly due to the development of technology and new technologies. However, water-related accidents, especially those caused by currents, involve human factors, and the ratio of fatalities to the number of accidents is still high. It is difficult to cover and solve such human-caused accidents with engineering technology alone. Therefore, it is necessary to solve the fundamental problem. The most important solution is to train people. By training people, the fundamental awareness of human factors can be trained, which will reduce accidents. The most effective way to learn these factors is through hands-on education and training. Among these, VR can be performed regardless of the location. It is expected to be much more effective than those obtained from videos or books.Many studies on VR training and Evangelos Markopoulo et al. have conducted a study on maintenance and safety education of ship engine systems using VR. In another study, Victor Saint-Martin et al. studied fire VR training for hospital personnel who have difficulty participating in regular on-the-job training. While there are studies on VR training that focus mainly on visual training, few studies on VR training focus on water accidents and aim to reduce accidents by tactilely displaying the strength of underwater currents. In addition, many underwater VR studies have been conducted for entertainment purposes, and few studies evaluate the perception of danger. In this study, we propose a VR system and a multimodal interface to post the flow of water hazards caused by the flow. Here, we validated the experience of flow using multiple senses, including VR and tactile sensation, to confirm the validity of the risk assessment. We found that the degree of risk perceived by a person can be combined in an additive manner with the risk postings given to each sensory organ, resulting in a higher risk rating. We also found that force postings produced higher danger ratings than tactile postings. These results indicate that using multiple senses to assess the danger of water currents increases the danger rating, and combining it with force instead of tactile sensation further increases the danger rating. In the future, we plan to discuss the effectiveness of this training in developing intuitive judgment. Furthermore, we would like to utilize this training for water accidents and other intuitive accidents to develop the ability to grasp the situation and make a judgment instantly.

INTRODUCTION

In recent years, the number of accidents and disasters has trended dow- nward owing to the development and introduction of new technologies. In particular, the ratio of traffic accidents to fatalities is low, accounting for approximately 1% of all accidents (adapted from National Police Agency). However, the ratio of deaths from water-related accidents is approximately 50% of all accidents (adapted from Overview of Water-related Accidents in 2020). A major cause of water-related accidents is often related to water flow (K Sugawara). We separated accidents into four quadrants: predicta- bility, unpredictability, and time until the accident occurs, and focused on accidents that are unpredictable and have a short time until the accident occurs. Virtual reality (VR) training is used in many situations. We believe that education and training through physical sensation can be more effe- ctive than that obtained from images and books, learning from experience rather than from concepts. For example, Tsumiski Mfg. Co., Ltd. developed simulators for construction machinery and disaster training for Meidensha Corporation, providing training on “crashes,” “falls,” “burns,” etc. (adapted from Meidensha Corporation). Furthermore, case studies using multimodal interfaces include a tasting display that employs a sensory interaction cal- led “meta-cookie” (T Narumi) and surgical training using VR and haptics (E Collaço). Woods et al. used sand and AR technology to study water flow

© 2022. Published by AHFE Open Access. All rights reserved. 144

Multimodal Haptic Perception of Underwater Flow 145

Figure 1: Number of people involved in water-related accidents (adapted from National Police Agency).
Original PDF Figure 1:
Figure 2: Number of people involved in water-related accidents (adapted from Ove- rview of Water-related Accidents in 2020).
Original PDF Figure 2:

(TL Woods), and Jain et al. used VR and haptic technology to develop a system for experiencing scuba diving (D Jain).

Studies visually display the speed and direction of the water flow, but few studies visually or haptically post the force. In addition, in the haptic perce- ption of water flow, many studies have been conducted on active systems in which the participant moves to haptically post the force of the flow, and few studies have addressed passive systems. Moreover, underwater-related VR and AR are often researched for entertainment purposes, and few studies have focused on risk education and safety training for water-related accidents caused by water flow or training to cultivate intuitive judgment.

In this study, we developed a system that actively posts water flow using VR, and passively posts force and speed using motors. Furthermore, the degree of danger posed by the system to humans was verified, and a com- bination of multimodal interfaces that would increase the degree of danger was discussed.

146 Shirakawa and Misaki

Figure 3: Classification of accidents.
Original PDF Figure 3:

PROPOSAL VR TRAINING WITH A MULTIMODAL INTERFACE Water-related accidents caused by water flow are difficult to predict, and the time until death is short; therefore, they can easily lead to severe accidents and require split-second decisions. Therefore, we studied a system for trai- ning split-second decision-making and intuition. Accumulating experience and learning it from the body is necessary to train intuition. We propose a system for the visual and haptic perception of water flow information as a method for fulfilling these purposes. The visual posting system uses an HMD to actively obtain information. “Active” is defined as obtaining information by moving one’s head and gaze within a virtual space. For haptic perception, we propose a system in which people training for scuba diving can passively experience water flow. The reason for passive perception is that when trai- nees are “caught” in a water flow in a water-related accident, they do not experience the flow with their movement but by entering an environment where the flow is already present. We believe that experimenting with the degree of danger by combining these factors benefits training in developing intuition.

The influence: Visual information has a significant influence on decision- making. Studies have estimated that 75–80% of our perception, learning, and behavior are transmitted through vision (J Sagara). However, excessive dependence on vision in gauging water flow can risk accidents, such as when the actual speed of the flow is faster than expected, rather than what it is perceived. These errors in estimating the status of the water flow are rela- ted to human factors (human error). Daily and VR training are practical for accidents and disasters that are difficult to reproduce to prevent accidents caused by human error. We believe that training using multimodal interfa- ces and VR will be effective in learning to not overrely on visual perception and bridging the gap between information obtained from visual perception and actual information. We believe that recognizing essential information more accurately is possible by providing information to senses other than the sight.

Multimodal Haptic Perception of Underwater Flow 147

Figure 4: Proposed method of visual and haptic posting of water flow.
Original PDF Figure 4:
Figure 5: Proposed posting system.
Original PDF Figure 5:

EXPERIMENTS ON DANGER PERCEPTION Our experiment used a system that stimulates the senses of sight, hearing, and haptic perception to post dangers for the water flow and evaluated the degree of danger to the participants. We discuss the combination of multimodal interfaces that can increase the perception of danger in the posting of water flow hazards from the results. Participants were asked to wear an HMD, and the water flow was posted through VR. In addition, when haptic posting was used, the motor was synchronized with the VR image, and the sensation of being swept in the water flow was posted on their hand. In the first set of procedures, participants were asked to perform “vision,” “vision + hearing,” “vision + hearing + haptic (using a rubber),” and “sight + hearing + haptic (using a string)”in sequence, followed by a Likert scale of 1 to 7, with 1 being no sensation and 7 being an intense sensation. In the “visual” pattern, an image of “danger” was posted in front of them when they placed their hands in the flowing water environment, and the image of danger disappeared when they removed their hands from the water (a). Next, as a “visual + hearing” pattern, in addition to the visual danger posting system, participants experi- enced a sound warning when they placed their hands in the water, and the

148 Shirakawa and Misaki

Figure 6: Image that the subject sees in the HMD when putting hands in the water flow in the VR space.
Original PDF Figure 6:
Figure 7: (a) visual posting, (b) Haptic (rubber) posting, (c) Haptic (string) posting.
Original PDF Figure 7:

system stopped when they removed their hands (a). In the “visual + hea- ring + haptic (using a rubber)” pattern, a stocking was attached to the palm, and the stocking was connected to a servo motor controlled by an Arduino. When the hand enters the water in the VR, the servo motor moved to provide a sensation of water flowing by pulling on the rubber, and the pulling action is released when the hand is removed from the water. This was designated as a haptic (rubber) danger posting (b). Finally, in the “sight + hearing + haptic (using a string)” pattern, the same experiment was conducted using a string instead of a rubber (c).

In the second set, participants experienced the patterns of “only visual,” “only hearing,” and “only haptic (using a rubber)” in sequence and, as in the first set, were asked to complete a questionnaire about their perception of the degree of danger after the sessions. In the “only visual” pattern, an image with the word “danger” was displayed in front of the participants when they placed their hands in the water, and danger signs were not posted to the other senses. An audible warning was played in the “only hearing” mode when a hand was placed in the water, without posting about other sensory hazards. In the “only haptic (using a rubber)” mode, a servo motor pulled their hand when placed in the water, and haptic danger postings were not provided to the other senses.

EXPERIMENT ON DANGER PERCEPTION Our experiment used a system that stimulates the senses of sight, hearing, and haptic perception to post dangers for the water flow and evaluated the degree of danger to the participants. We discuss the combination of multi- modal interfaces that can increase the perception of danger in the posting of water flow hazards from the results. Participants were asked to wear an HMD, and the water flow was posted through VR. In addition, when haptic

Multimodal Haptic Perception of Underwater Flow 149

posting was used, the motor was synchronized with the VR image, and the sensation of being swept in the water flow was posted on their hand. In the first set of procedures, participants were asked to perform “vision,” “vision + hearing,” “vision + hearing + haptic (using a rubber),” and “sight + hearing + haptic (using a string)”in sequence, followed by a Likert scale of 1 to 7, with 1 being no sensation and 7 being an intense sensation. In the “visual” pattern, an image of “danger” was posted in front of them when they placed their hands in the flowing water environment, and the image of danger disappea- red when they removed their hands from the water (a). Next, as a “visual + hearing”pattern, in addition to the visual danger posting system, participants experienced a sound warning when they placed their hands in the water, and the system stopped when they removed their hands (a). In the “visual + hea- ring + haptic (using a rubber)” pattern, a stocking was attached to the palm, and the stocking was connected to a servo motor controlled by an Arduino. When the hand enters the water in the VR, the servo motor moved to provide a sensation of water flowing by pulling on the rubber, and the pulling action is released when the hand is removed from the water. This was designated as a haptic (rubber) danger posting (b). Finally, in the “sight + hearing + haptic (using a string)” pattern, the same experiment was conducted using a string instead of a rubber (c).

In the second set, participants experienced the patterns of “only visual,” “only hearing,” and “only haptic (using a rubber)” in sequence and, as in the first set, were asked to complete a questionnaire about their perception of the degree of danger after the sessions. In the “only visual” pattern, an image with the word “danger” was displayed in front of the participants when they placed their hands in the water, and danger signs were not posted to the other senses. An audible warning was played in the “only hearing” mode when a hand was placed in the water, without posting about other sensory hazards. In the “only haptic (using a rubber)” mode, a servo motor pulled their hand when placed in the water, and haptic danger postings were not provided to the other senses.

RESULTS

Figure 8 summarizes the results of the danger ratings for each combina- tion of senses. Figure 9 summarizes the results of the danger ratings for each sensory organ. The average risk ratings for the combination of sensory postings were approximately 1.8 for “visual”, 3.1 for “visual + hearing”, 4.1 for “visual + hearing + haptic (rubber)”, and 5.1 for “visual + hea- ring + haptic (string)”. The average risk rating for each sense was 2.8 for visual, about 2.3 for hearing, and 4 for haptic. The same conditions were used in Figures 8 and 9.

DISCUSSION

Figure 8 shows that, with a rating of 4 as the borderline, all respondents in the visual case were below 4, and over 80% of the respondents in the visual + hearing case were below 4. By contrast, for visual + hearing + haptic

150 Shirakawa and Misaki

Figure 8: Evaluation of a combination of posted senses.
Original PDF Figure 8:
Figure 9: Evaluation of each sensory organ.
Original PDF Figure 9:

(rubber), approximately 50% of respondents exceeded the rating of 4, and for visual + hearing + haptic (string), approximately 80% of respondents also exceeded 4. This suggests that the use of visual + hearing + haptic (both rubber and string) is more important than the use of visual, visual + hearing, and that posting to all senses plays an essential role in danger perception training.

Furthermore, a comparison of the median values shows a gradual increase from 2 for visual, 3 for visual + hearing, 5 for visual + hearing + haptic (rubber), and 6 for visual + hearing + haptic (string). This suggests that the influence of the degree of danger that a multimodal interface poses to a person increases with the combination of the senses. We suspect that posting danger to a combination of several sensory organs, rather than posting danger to independent sensory organs, increases the danger rating.

A comparison of the scatter of data between visual + hearing + haptic (rubber) and visual + hearing + haptic (string) shows that the scatter of data is higher for visual + hearing + haptic (rubber). We believe that this is because many respondents felt a weaker pulling sensation with rubber than with string because of the elasticity of the rubber. With string, the elastic force is small; the hand only responds to the force when pulled, and the hand is thought to feel an intense sensation of being carried by the flow.

Figure 9 shows that the median risk ratings for each sensory organ were less than 4, and that the median risk rating for visual and hearing was only 2.

Multimodal Haptic Perception of Underwater Flow 151

This suggests that the danger rating of a single sensory organ does not elicit a high danger rating from people. Therefore, we believe that posting hazards to several sensory organs effectively teaches hazard perception training.

The differences in the visual ratings in Figures 8 and 9 may be related to the experiment. The visual evaluation in Figure 8 was performed at the begin- ning of the experiment; therefore, the participants did not have a thorough understanding of the system. In contrast, the visual image in Figure 9 was captured during the experiment, when participants began to understand the system.

CONCLUSION

This study focused on VR training for water-related accidents that are diffi- cult to predict and require limited time before serious accidents, particularly those caused by water flow. We proposed a training method for experiencing danger using VR and a multimodal interface. This experiment evaluated the degree of water flow danger using a multimodal interface. We found that the degree of danger perceived by a person can be evaluated as high by combi- ning several danger posts to each sensory organ. In haptic danger posting, a difference in the evaluation of the degree of danger was observed between using a rubber or a string, and we plan to examine the factors that cause this difference in the future.

REFERENCES

Collaço, Elen, et al. “Immersion and haptic feedback impacts on dental anesthesia technical skills virtual reality training.” Journal of Dental Education 85.4 (2021): 589–598. Jain, Dhruv, et al. “Immersive terrestrial scuba diving using virtual reality.” Jiro Sagara, et al. “An Experimental Study of the Effects of Visual Deprivation on

Human Behavior.” arts and engineering (2010). Kazunari Sugawara, Hideo Yoshino, and Junji Miwa. “An Overview of Water Accidents in the Past Decade and Measures for the Future.” from Report of the River Research Institute. River Research Institute, River Foundation 20 (2016): 65-76 Date viewed: 4/25/2022 National Police Agency (Keishicho Web Saito) Date viewed: 4/25/2022 https://www.

npa.go.jp/english/Police_of_Japan/2020/poj2020_full.pdf Overview of Water-related Accidents in 2020, Life Safety Planning Division, Community Safety Bureau, Metropolitan Police Department Date viewed: 4/25/2022 http s://www.npa.go.jp/publications/statistics/safetylife/chiiki/R02suinan_gaikyou.pdf Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in

Computing Systems. 2016. Takushi Narumi, et al. “Meta-Cookie: Investigation of Taste Displays Using Sensory Interactions (< Special Issue> Fragrance, People, and Systems).” [Transactions of Virtual Reality Society of Japan] VR contents for safety education for Meidensha Corporation Date viewed:

4/25/2022 http://tsumikiseisaku.com/result/meiden.html Woods, Terri L., et al. “Pilot study using the augmented reality sandbox to teach topographic maps and surficial processes in introductory geology labs.” Journal of Geoscience Education 64.3 (2016): 199–214.

Cover of the proceedings volume Safety Management and Human Factors

Explore the complete digital volume

Safety Management and Human Factors

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-40-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →