Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Hazardous Training Scenarios in Virtual Reality - A Preliminary Study of Training Scenarios for Massive Disasters in Metaverse

1 Turku University of Applied Sciences, Finland ↗
2 JiiM Ltd, Finland ↗
3 University of Applied Sciences Karlsruhe, Germany ↗
4 University of Sfax, Tunis ↗
5 University of Turku, Finland ↗
Publication year
Digital Object Identifier10.54941/ahfe1002062

Abstract

Simulation training in aviation and maritime is widely used in competence training and assessment. These simulator centres have suffered a lot because of COVID-19 pandemic. Due to the rapid progress in technology development and the pandemic disruptive solutions are intensively searched in vocational and professional training. Flight and maritime simulators are examples of training environments where even hazardous scenarios can be trained in safe conditions. In the previous studies, we have shown that virtual reality offers for other fields tools to create training solutions which can be again hazardous such as our virtual fire safety application used in fire escape. In addition, virtual and augmented reality can be used to create digital learning environments in fire safety prevention training combining physical, psychological, social and pedagogic dimensions. In this paper, we will focus also on virtual fire safety training. Aircraft fires require special treatment in firefighting with regards to the burning materials. This is due to the fact that about half of the aircraft consists of fibre composites, which can release many fine particles that are harmful to the lungs during combustion. However, the training of aircraft firefighting is currently only possible with great effort on a few special training grounds. This training application with multiplayer functionalities was created with Unity game engine. In the design phase, emphasis in the creation of the game was in setting up environment where teamwork and leadership is needed to accomplish the scenario. This approach is quite close to the metaverse concept where social communication is combined with hands on training activities among a large group of participants in an immersive digital training environment. The task of the participants is to first assess the situation, extinguish the fire and prevent the fibres from spreading to the surrounding area. This is done by collecting individual smaller pieces of composite debris or covering larger ones with foam so that they can no longer be carried away with the smoke and wind. Aside from distinguishing the fire at the crash site, the firefighters are also trained to collect debris from the crash site and discard it into the bin. Both tasks are equally as important and require a Standard Operational Procedure guideline in order to realistically implement them in the application. Research was needed to find appropriate solutions for multiplayer functionalities, for fire and smoke behavior, and for extinguishing the fire. Photon Unity Networking framework was used to enable multiplayer functionalities. Fire Propagation plugin in turn enabled to make the fire spread, to configure the appearance of the fire including the size, and the location of flames, the amount and the shape of smoke and sparks based on given requirements. Extinguishing the fire required the use of the water particle system with suitable collision detection. We found multiplayer functionalities to be an important element in virtual training. Scenario was designed so that participants had to communicate well with each other to ensure a fast firefighting. Our application is still in a prototype phase and more efforts will be needed to make the training more realistic. We will in the next phase present the story of the scenario more in details, and increase the stress level of participants by adding more tasks. In addition, our aim is to improve the assessment system analyzing user data, including difficulty levels, high score list, and feedback system. This solution can also be seen as a preliminary study for a massive catastrophe training experiment where tens or hundreds of professionals will be trained in the metaverse environment utilizing in-house metaverse technology.

INTRODUCTION

Fire scenarios are both expensive and dangerous to practice in real life envi- ronments. Virtual reality applications provide a meaningful platform for training. Compared to standard training activities, such as lectures or videos, VR games enable the learner to participate and, the experience is both immer- sive and engaging (Oliva et al., 2019). Simulation training especially in aviation and maritime is widely used in competence training and assessment. Flight simulator centers have been one of the most vulnerable verticals during the COVID-19 pandemic (AviationPros, 2020). Due to the rapid progress in technology development and the pandemic disruptive solutions are intensi- vely searched in vocational and professional training. Flight and maritime simulators are examples of training environments where even hazardous sce- narios can be trained in safe conditions. In the previous studies, we have

© 2022. Published by AHFE Open Access. All rights reserved. 64

Hazardous Training Scenarios in Virtual Reality 65

shown that virtual reality offers for other fields tools to create training solu- tions which can be again hazardous such as our virtual fire safety application used in fire escape (Somerkoski et al., 2020).

When learning with digital simulations, we have to think about digital learning and especially learning environments. We consider the concept of learning environment in a broad sense, widely used in educational sciences. The concept of learning environment refers to the diverse physical locations, contexts, and cultures in which students learn. Also activities, places, spaces, communities, or modes of action that support and promote learning can be considered learning environments. A prerequisite for a learning environment is that it is pedagogically planned (Piispanen, 2008).

The dimension of physical learning environment refers to materials, techni- cal solutions or tools; the dimension of social learning environment is focused on interaction between individuals and on the group dynamics, the dimen- sion of psychological learning environment refers to cognitive processes such as remembering, decision making and understanding, and the dimension of pedagogic learning environment refers to pedagogic solutions and learner- based working methods. Based on the theories of learning, it has to be noted that the learner is not just the person who absorbs information. Instead, the learner is an active body who constructs his or her learning (Dewey, 2021). Moreover, during the learning, there is a constant interplay between the lear- ner and the information (Gulikers et al., 2005; Ding et al., 2015). As a rough grouping, physical learning environment refers to concrete spaces, materials, technical solutions or tools; the social learning environment is focused on interaction between individuals and on the group dynamics, and the peda- gogic learning environment gets its shape form the pedagogical phenomena, such as the learner-based approach or the problem-based learning. Some rese- archers state that there are to opposing views considering the digital learning; the dichotomy refers firstly to the environments for the transfer of knowledge from the teacher to the learner and, the other as cooperative learning environ- ment where teachers guides learners through a series of steps by cooperating with them. This supportive design allows the learner to acquire some com- petencies and especially skills. The first mentioned approach is based on the web while the second approach requires specific learning environments and team work. Collecting data from the monitoring and collecting the feedback from learners can provide instructors with possibilities for developing better learning materials, enhancing the learning processes and identifying the main areas of learning difficulties (Balla, 2009).

Whatever learning environment we are talking about, the teacher, with multiple roles in the multiple learning environments, is a key factor (Lan- Ying and Xue-Mei, 2010; Jones, 2010). In addition, in the research literature, also concepts, such as digital learning environment, virtual (versus authentic) learning environment or e-learning environment are used (Manninen et al., 2007). The use of these concepts is not exactly determined, but the basic pri- nciple is that an on-line technical device, often PC, mobile phone or tablet, supports this kind of learning (Ding et al., 2015; Gulikers et al., 2005). Alre- ady early, 1970, Abt stated, that serious games are synonymous with games for learning that are deliberately developed for more than pure entertainment

Figure 1: Smoke flow (left) and two of the tasks to be completed (right) in Virpa game.
Original PDF Figure 1:

(Abt, 1970). Whatever the dimension we are focusing at, the teacher who plans the pedagogy around the environment (Somerkoski et al., 2020) is the key factor. The overall goal for learning is achieving competence which requi- res integrating and implementing knowledge, skills, and attitudes (Baartman and De Bruijn, 2011).

In addition, virtual and augmented reality can be used to create digital learning environments in fire safety prevention training combining physical, psychological, social and pedagogic dimensions. The latest development of multi-player opportunities in VR provides new possibilities for social learning environment, such as decision making and peer or colleague negotiation pos- sibilities in case of emergency. Concerning safety and security, learning skills is in the focus. Virtual simulations provide an excellent tool for that. If the digital learning environment is well designed we might have a possibility to provide knowledge that is built in the game. However, usually attitudes are developed during a longer time and it might be difficult to have an impact on those with a simulation.

PREVIOUS STUDIES In our previous studies related to the fire safety, we have focused mainly on single player VR and AR applications. We have seen that VR is useful in pre- and post-training while AR is to be used as an on-site training tool (Somerkoski et al., 2020). In FiAAR project (Bellalouna et al., 2020), we have utilized AR as a potential technology to provide assembly and configuration assistance for firefighteers working in harsh conditions. We have shown that VR can also be an exceptional evaluation tool (Figure 1) to track and trace human behavior under stressful and hazardous conditions such as fire escape. As a matter of fact, in our Virpa VR game children did not make so much eye contact with exit signs and floor plans (Oliva et al., 2019).

During the COVID-19 vocational schools and industrial competence cen- ters have had challenges to assess competences. Kiwa Inspecta and Ade for example have launched in November 2020 European Hot Work Certification including five different virtual exercises (hazard situations and protection, action in the event of a fire, and the use of a foam extinguisher, a fire hydrant cabinet, and a fire blanket). The exercises use virtual equipment controllers as well as a foam fire extinguisher controller designed for training (Ade Ltd., 2022). These training scenarios such as the use of extinguisher (Figure 2) can be customized based on needs in industry such as electric cabin tested in Oman (Al-Adawi and Luimula, 2019).

Hazardous Training Scenarios in Virtual Reality 67

Figure 2: The use of extinguisher in ship (left) and industrial (right) environments.
Original PDF Figure 2:
Figure 3: Multiplayer functionalities used in teacher student communication.
Original PDF Figure 3:

Multiplayer functionalities are still typically lacking in virtual fire safety training solutions. Now when term metaverse has become more frequently used it is obvious that multiplayer functionalities will be widely utilized in virtual training in the near future. Game industry has utilized multiplayer functionalities in games for years. The use of multiplayer functionalities in virtual reality applications is more complex due to the needs for motion dete- ction of users (from head movements to upper limbs and to finally full body tracking). Depending on the number of joints to be tracked more and more resources will be needed. We have started applying first Photon Bolt and later Mirror networking tools to develop own metaverse technology. As the first step in this development the use of fire extinguisher (Figure 3) was tested with Photon Bolt enabling teacher student interaction and various other useful features to be used in metaverse (Österman, 2021; Luimula et al., 2021).

PROTOTYPE DESCRIPTION In this paper, we will focus on describing our preliminary study related to the virtual fire safety training in disaster management and leadership training. Aircraft fires require special treatment in firefighting with regards to the bur- ning materials. This is due to the fact that about half of the aircraft consists of fiber composites, which can release many fine particles that are harmful to the lungs during combustion. However, the training of aircraft firefighting is currently only possible with great effort on a few special training grounds.

In Emergency Services Academy Finland training description, following procedures and methods are recognized as essential means in fighting a modern passenger plane fire. The personnel must be protected according to the situation and the task at hand. The protection gear must provide

Figure 4: Crash site in the training environment.
Original PDF Figure 4:

cover for skin, eyesight and breathing. When using pressurized extinguishers, avoidance of too high pressure is essential to minimize spreading composite particles.

To minimize the exposure to fibers, foam can be used to prevent the fibers from spreading to the environment. Other means are to restrict the amount of personnel operating in the scene, isolate the area and if necessary, evacuate people nearby. Weather and especially wind must be considered carefully. In the post extinguishing phase careful and systematic actions are needed to collect all fiber pieces from the area. Use of plastic bags or covering bigger parts with plastic film or similar means is advisable. A thorough cleaning of protective gear and equipment used in the scene, using a protective gear is essential to avoid any exposure.

This training application with multiplayer functionalities was created with Unity game engine. Some of the objects which should be found in the trai- ning scene were not already accessible in the Unity Asset Store. Concretely this includes the little composite debris which has to be picked, the bag to put them into, a lightning and a fan as additional equipment for the fire- fighters. The Blender software was used for modelling and preparing these additional objects to the training environment (Figure 1). Research was nee- ded to find appropriate solutions for multiplayer functionalities, for fire and smoke behavior, and for extinguishing the fire. Photon Unity Networking framework was used to enable multiplayer functionalities. Fire Propagation plugin in turn enabled to make the fire spread, to configure the appearance of the fire including the size, and the location of flames, the amount and the shape of smoke and sparks based on given requirements. Extinguishing the fire required the use of the water particle system with suitable collision detection.

In the design phase, emphasis in the creation of the game was in setting up environment where teamwork and leadership is needed to accomplish the scenario. This approach is quite close to the metaverse concept where social

Hazardous Training Scenarios in Virtual Reality 69

Figure 5: Multiplayer functionalities used to train collaboration skills in the hazardous training scenario.
Original PDF Figure 5:

communication is combined with hands on training activities among a large group of participants in an immersive digital training environment.

In this scenario a total of ten person can take part in the training. There are two firefighting vehicles in use. The scenario starts with the engines in fire but the fire spreads rather aggressively. The task of the participants is to first assess the situation, extinguish the fire and prevent the fibers from spreading to the surrounding area. Participants can operate the firefighting vehicles by placing them in predefined places as preferred and use the water cannon and aim the water where needed. As the scenario itself is very simple, and straight forward, the decision how to operate and communicate stays with the users (not yet including voice communication).

We found multiplayer functionalities to be an important element in virtual training (Figure 2). Scenario was designed so that participants had to com- municate well with each other to ensure a fast firefighting. Our application is still in a prototype phase and more efforts will be needed to make the trai- ning more realistic. We will in the next phase present the story of the scenario more in details, and increase the stress level of participants by adding more tasks. In addition, our aim is to improve the assessment system analyzing user data, including difficulty levels, high score list (team work and the ove- rall result is that counts), and feedback system. This solution can also be seen as a preliminary study for a massive disaster training experiment where tens or hundreds of professionals will be trained in the metaverse environment utilizing in-house metaverse technology.

CONCLUSION

As a preliminary study, we have not yet been able to conduct any pilot tests with professionals. In these upcoming tests, we will focus on usability and user experience studies. These studies are needed especially because virtual training episodes including multiplayer functionalities have wide range of new aspects to be covered compared to single player episodes. We have new

variables such as communication and interaction methods between players which will make test procedures more complex to be executed.

In the future, we plan to focus on massive disaster episodes where emerge- ncy management and leadership will be in the center. This kind of episodes will setup special requirements for the technology to be used. Using metaverse technology is challenging firstly, as the need is to provide learning environ- ment where tens of people cooperate, excercise and train together. Secondly, the emergency leadership in major scenarios and hazards is very demanding. Yet the setting allows the learners to face real-life problems and moreover, the session can be easily stopped and discussed and started over again. The massive number of users would be needed for example in the training scena- rios where professionals are helping passengers. This action is importans as it is actually the first priority issue Training immersively is challenging using existing technology. The use of metaverse would enable us to for example ask 2/3 of trainees to act passengers and the rest to act professionals. Or it would be also possible to train AI with the behavioral data collected from participants acting as human passengers. And in the digital training environ- ment it will be easy to switch the roles of trainees between training sessions. One of the future directions can also be to focus on different roles together with more sophisticated simulation of equipment used in the scenario.

We have also plans in the future to compare the trainer vs AI based feed- back systems. For the trainers we have plans to develop a dashboard with a strategic overview enabling a real-time evaluation of the trainees. AI based feecback system in turn could utilize neural network to provide additional real time metrics for the human trainer or directly for trainees as a tool for evaluation. Neural network operated evaluation could also include various stress indicators which are difficult to notice by trainer. From pedagogical per- spective a strategic overview is needed to create a picture of all participants working simultaneously. Also, a recording is needed for post analysis.

As a summary, with these resources we were able to just focus on a couple of core functionalities. More graphic assets will be needed and this simplified training scenario just proofed for us that this concept seems to work based on our expectations including multiplayer functionalities, vehicles, basic training tasks etc. In the next phase, we have to increase the realism. For example, to extinguish fire requires in the real life conditions much more time and resou- rces. This can be achieved by focusing on physics (e.g. high temperatures, water consumption, smoke behavior) and more complex environments (e.g. trees and other obstacles). Compared to the single-user mode where physics calculations can occur individually, in metaverse as a multi-user environment the physics calculation must also be synchronized between participants (e.g. everyone sees the same smoke behavior).

We are aiming at a digital learning environment - a massive emergency sce- nario where the leader of the exercise learns emergency management skills and the other players are able to learn cooperative actions in case of emerge- ncy. The scenarios of this kind are difficult or impossible to manage as there are very few suitable physical learning environments available in reality. In the future, we plan to produce hands-on-training tasks which can improve immersions. However, there are still several bridges to gap. The use of virtual

Hazardous Training Scenarios in Virtual Reality 71

training instead of a traditional one, contains a risk of misunderstandings of the time consumption during the management and leadership. If we would rather focus only on work processes there is a risk to jump quickly forward in the timeline. So the use of hands-on-training activities forces trainees to focus on handling multiple tasks in the same time and generating delays and extra stress which will be the case in the real life conditions.

Compared to the single player mode where a trainee is teleporting him/h- erself between places, multiplayer functionalities enable rich communication between trainees and immediate feedback from the trainer or AI. We mana- ged to create a multiplayer environment as a framework which can be used in many ways to train management and leadership topics in fire safety. So our developed prototype can be seen as a platform for next generation lear- ning both in management and leadership even that our focus was here more on leadership and further studies will be needed to understand better which kind of new ways we can offer for training center to create training scenarios orchestrating professionals.

AKNOWLEDGEMENTS The discussed working prototype with outcomes were achieved during the project IC xR-P that is part of the programme Baden-Württemberg- STIPENDIUM for University and Students (BWS plus) and funded by the Baden-Württemberg Foundation. The authors would like to thank everyone who have participated in the development of the prototype study.

REFERENCES

Abt, C.C., (1970) Serious Games. Viking Press, New York. Ade Ltd. (2022) Kiwa European Hot Work Certification. (February 10, 2022). DOI = https://virtualtrainings.ade.fi/en/certified/european-hot-work-certificate-2/ Al-Adawi, M., and Luimula, M. (2019) Demo Paper: Virtual Reality in Fire Safety

– Electric Cabin Fire Simulation In: Proceedings of the 9th IEEE Conference on CogInfoCom 2019, Naples, Italy, pp. 547–548. AviationPros (2020) Flight Simulator Sales Ramp Up With AR and VR Integration (May 21, 2020). DOI: https://www.aviationpros.com/education-training/simula tor-training/press-release/21139115/future-market-insights-flight-simulator-sale s-ramp-up-with-ar-and-vr-integration-global-airline-closure-due-to-covid19-pa ndemic-temporarily-arrests-demand-growth Baartman, L. K. and De Bruijn, E. Integrating knowledge, skills and attitudes:

Conceptualising learning processes towards vocational competence. Educational Research Review, 6, 2 (2011), 125–134. Balla, A. (2009) “Designing Pedagogical Learning Environment”, International Journal of Advanced Science and Technology, 6, 1–14. Bellalouna, F., Luimula, M., Markopoulos, P., Markopoulos, E., and Zipperling, F.,

FiAAR: An Augmented Reality Firetruck Equipment Assembly and Configuration Assistant Technology, In: Proceedings of the 11th IEEE International Conference on Cognitive Infocommunications CogInfoCom 2020, Sep 23-25, online, 2020, pp. 237–244. Dewey, J. (2021) 4. The Challenge of Democracy to Education. Columbia University Press, City.

Ding, J., Xiong, C. & Liu, H. (2015) “Construction of a digital learning environment based on cloud computing”, British Journal of Educational Technology, (46:6), 1367–1377. Ding, J., Xiong, C. and Liu, H. (2015) Construction of a digital learning environ-

ment based on cloud computing. British Journal of Educational Technology, 46, 6, 1367–1377. Gulikers, J. T., Bastiaens, T. J. and Martens, R. L. (2005) The surplus value of an authentic learning environment. Computers in Human Behavior, 21, 3, 509–521. Gulikers, J.T., Bastiaens, T.J. & Martens, R.L. (2005) “The surplus value of

an authentic learning environment”, Computers in Human Behavior, (21:3), 509–521. Jones, C. (2010) “Entrepreneurship education: Revisiting our role and its purpose.

Journal of Small Business and Enterprise Development”, (17:4), 500–513. Lan-Ying, H., and Xue-Mei, D. (2012) “An Empirical Study of Learner-Based Teaching in EFL”, Sino-US English Teaching, (9:4), 1061–1064. Luimula, M., Markopoulos, E., Österman, M., Markopoulos P., Aho J., Ravyse,

W., Saarinen, J., and Reunanen, T. (2021, Sep 23–25). Avatar Based Multiplayer Functionalities in Next Generation Communication and Learning in Virtual Rea- lity Social Platforms – Case MarISOT Room, In: Proceedings of the 12th IEEE International Conference on CogInfoCom, online, pp. 447–452. Manninen, J., Burman, A., Koivunen, A., Kuittinen, E., Luukannel, S., Passi, S., & Särkkä, H. (2007) Environments that support learning. Introduction to learning environments approach. Helsinki, Finnish National Board of Education, 2007. Oliva, D., Somerkoski, B., Tarkkanen, K., Lehto A., and Luimula, M. (2019)

Virtual reality as a communication tool for fire safety – Experiences from the VirPa project. In: Proceeding of the 3rd GamiFIN conference, Levi, Finland, pp. 241–252. Österman, M. (2021) Development of a Virtual Reality Conference Application. BSc Thesis, Turku University of Applied Sciences. Piispanen, M. (2008) Hyvä oppimisympäristö. Oppilaiden, vanhempien ja opettajien

hyvyyskäsitysten kohtaaminen peruskoulussa. University of Jyväskylä. Somerkoski, B., Oliva, D., Tarkkanen, K., and Luimula, M. (2020). Digital Learning Environments - Constructing Augmented and Virtual Reality in Fire Safety, In: Proceedings of the 11th International Conference on E-Education, E-Business, E- Management, and E-Learning (IC4E 2020), Jan 10–12, Osaka, Japan, 6p.

Cover of the proceedings volume Human Factors in Virtual Environments and Game Design

Explore the complete digital volume

Human Factors in Virtual Environments and Game Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-26-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →