Rida Kamal, A K M Ishtyak Hossain, Janne Heilala, Jussi Kantola (2025). Assessing Virtual Reality Fire Extinguisher Training Effectiveness: A Quantitative Empirical Study. In: Human Factors, Business Management and Society. AHFE Open Access, vol. 176, pp. 186–196. https://doi.org/10.54941/ahfe1006316
This study conducts a quantitative assessment based on empirical evidence to measure the effectiveness of virtual reality-based fire extinguisher training. The data were collected from 71 participants who first underwent virtual reality (VR) fire extinguisher training and then practical fire extinguisher training with a gap of one week between these sessions. After finishing the practical training, participants filled out a structured quantitative questionnaire evaluating five key aspects: Knowledge Acquisition, Knowledge Retention, Realism, Usability, and Engagement. To assess the effectiveness and identify the stronger predictor of the effectiveness, a quantitative analysis was conducted encompassing descriptive statistics, t-tests, and regression analysis. The findings from this research reveal that Engagement and Usability were ranked highest in terms of the effectiveness of VR training. Participants felt that for VR training Engagement and Usability enhanced their overall experience. Furthermore, Engagement and Usability had the strongest impact on the effectiveness of the overall training. Respondents also reflected on the insufficiency of the VR environment in relation to realism, emphasizing the need for advancements in this area. These results imply that VR is an effective, engaging, and usable tool for safety training, but that refinements in interactive fidelity are needed in order to boost learning outcomes. Future research may investigate realism enhancements and knowledge retention strategies to further optimize VR-based safety training programs.
In today’s fast paced society with rapidly evolving technology, one of the biggest assets to any organization is the human ability to learn and adapt (Mystakidis et al., 2022). The industrial requirement for competence is evolving to keep pace with rapidly changing technologies and new regulations (Hager et al., 2022). Competence refers to the blend of knowledge, experience, abilities, and skills required to perform a particular task successfully (Hager et al., 2022). Global industries need to train and certify their employees to fulfill job demands with regard to work safety. This market demand is higher among vocational training organizations
Assessing Virtual Reality Fire Extinguisher Training Effectiveness 187
(Drake et al., 2022), however, individual businesses require customized solutions in order to retain commercial confidentiality. Immersive VR is a revolutionary technology within the educational and vocational training industry by which humans can be taught and equipped with the desired skill sets (Mystakidis et al., 2022). The key strengths of VR technology are replicability of real-life scenarios and repeatability of training sessions (Drake et al., 2021). Immersive games are able to produce more engagement and knowledge retention than traditional safety cards (Chittaro and Buutussi, 2015). VR is an effective platform for procedural skills acquisition (Morélot et al., 2021) that can provide simulated dangerous scenarios that are too risky or ethically inappropriate to be replicated in the real-world environment (Bailenson, 2018).
Virtual Training Certification (VTC) - an industrial project that involves other stakeholders funded by Business Finland - aims at solving this challenge of training both groups and individuals while ensuring usability, validity, and safety of the training. The VTC project intends to build a standardized certification system for remote validation of competencies using VR technology. This paper presents empirical evidence in support of the VTC project’s goals by assessing the effectiveness of VR-based fire extinguisher training.
BACKGROUND
Fires are a common and significant hazard in building environments that can cause mass destruction (Levreglio et al., 2020). It is crucial to have the right fire extinguishing equipment as well as the knowledge and skills to use this equipment properly in order to stop fires. In recent decades, VR technology has created a breakthrough in the fire safety sector by providing a safer and more reliable training approach than conventional methods (Levreglio et al., 2020). Studies comparing VR-based training methods with traditional learning methods, such as text-based materials (Rahouti et al., 2021), PC games (Ristor et al., 2021), recorded videos, and safety cards, consistently show that VR-based training outperforms other conventional methods in terms of usability, knowledge acquisition, and engagement (Levreglio et al., 2020). By utilizing serious games in an immersive 3D gas-powered plant engine simulation, Ebo and Kantola (2021) provided factual data and proved that 3D simulation-based technology has significant potential in amplifying safety training.
Since VR is emerging as a spotlight technology for offering an engaging training experience, it is important to assess its effectiveness and applicability in safety training (Scorgie et al., 2023). VR environments, despite being a suitable medium for safety training, require assessment, and companies must evaluate VR effectiveness before investing in such expensive technologies (Morélot et al., 2021).
There has been extensive research on the effect of VR fire safety training in terms of knowledge acquisition and self-efficacy as key comparative metrics (Sun et al., 2024). Saghafian et al. (2020) assessed the effectiveness of VR fire extinguisher training compared to conventional training methods focusing on
aspects such as emotional experience, knowledge acquisition, self-efficacy, and knowledge retention. Morélot et al. (2021) focused on how immersion and a sense of presence influence learning in VR fire safety training.
However, there is a lack of research measuring training effectiveness in a real fire situation (Scorgie et al., 2024; Sun et al., 2024). Therefore, research specifically focusing on psychometric-based quantitative evidence assessment of VR fire extinguisher training effectiveness remains limited and warrants further investigation.
OBJECTIVE In recent decades, VR technology has created a breakthrough in the fire safety sector by providing a safer and more reliable training approach than conventional methods (Levreglio et al., 2020). VR has the edge over other methods for its ability to simulate dangerous scenarios, therefore it is very crucial to validate and evaluate its effectiveness before its implementation (Stefan et al., 2023).
This research aims at evaluating the effectiveness of VR-based fire extinguisher training to elevate participants’ knowledge acquisition and retention by integrating qualitative and quantitative findings. In this paper, we will specifically discuss quantitative findings. Furthermore, to validate the impact of VR training, we analyze the participants’ overall experience by investigating key factors such as usability, engagement level, and their perception of realism.
In this paper, the effectiveness of VR fire extinguisher training is measured through psychometric-based quantitative assessment. Seventy- one participants were involved in the training process. The process required participants to conduct VR fire extinguisher training followed by practical fire extinguisher training, concluding by responding to an evaluation questionnaire. The evaluation form is a structured Likert-scale- based questionnaire used to measure the effectiveness of VR fire extinguisher training.
RESEARCH QUESTIONS (RQS) RQ1. How effective is VR-based fire extinguisher training in elevating participants’ knowledge acquisition?
RQ2. How effective is VR-based fire extinguisher training in enhancing participants’ knowledge retention?
RQ3. How do participants rate the realism of fire scenarios and the VR environment?
RQ4. How do participants perceive and rate their engagement and overall experience during VR fire extinguisher training?
RQ5. How do participants evaluate the usability, such as clarity of information and ease of navigation, in VR fire extinguisher training?
RESEARCH DESIGN PROCESS This study encompassed a quantitative research design to evaluate the effectiveness of using VR technology for fire extinguisher training. This study
Assessing Virtual Reality Fire Extinguisher Training Effectiveness 189
involved 71 participants who underwent fire safety training in Finland. At the start of the training, participants underwent VR fire safety training and later they were exposed to a real fire situation and used fire extinguisher equipment.
The practical training was held on four different occasions and the same procedure was followed on all four days. The fire extinguisher equipment (fire extinguisher bottles, fire control unit, fire pool, gas cylinders, safety jacket, and safety helmet) were brought outside into a yard area to carry out the training tasks. All participants were provided with safety gear and an informed consent form to fill out before commencing their training. The training required participants to perform the task individually, in the presence of instructors, researchers and facilitators.
After finishing the practical training, all participants were asked to fill out evaluation forms. The researchers designed three sets of evaluation questionnaires with one being a Likert-scale-based quantitative questionnaire form (discussed in this paper). The questionnaire consists of 16 Likert-scale- based (0-5) questions with a total of 71 participants. The questionnaire was created to measure five key aspects of VR training such as Knowledge Acquisition, Knowledge Retention, Realism, Usability, and Engagement of the training as shown in Table 1. Table 1 provides a structured view of the distribution of the questions over the themes.
Table 1: Five key factors (themes) and their corresponding variables.
Table available in the original PDF paper.
Distribution of Questions Over Themes Theme Knowledge Acquisition Q1 This training has offered you with adequate fire extinguisher knowledge Q4 The training has enabled you to know what to do in case of an accident Q7 The training will affect your capability to quench an emergency fire
positively Q16 The training provided you with adequate learning experience Theme Knowledge Retention Q6 You will remember what you’ve learned about fire extinguishers a year from now Q14 The training will most likely help you avoid accidents at the workplace Theme Realism Q2 The dangers were demonstrated realistically Q5 The demonstrations represented real factory situations Q8 The level of the theoretical fire extinguisher representation complied
with practice Q9 The training illustrates realistic situations in the field Theme Usability Q3 The obtained information was adequately understandable to you Theme Engagement Q10 You concentrated during the fire extinguisher training Q11 The training provided you with a pleasant learning experience Q12 You will most likely recommend this kind of training to others Q13 You would most likely participate in this kind of training in the future Q15 The time of the training was a worthwhile investment
RESEARCH METHODOLOGY This study encompassed descriptive analysis incorporating Mean (M), Standard Deviation (SD), and Frequency Distribution (FD). Further, Regression Analysis (RA) is conducted by creating a composite score for the dependent variable to represent the overall perceived effectiveness of the VR training. This composite score is a combination of all variables and serves as a measure of VR training effectiveness. For independent variables, five suggested themes - Knowledge Acquisition, Knowledge Retention, Realism, Usability, and Engagement - are considered.
For the data analysis we used the IBM SPSS software. The data entered on SPSS has 71 rows as responses and 16 columns as variables. To ensure the accuracy of quantitative research, there are four key criteria of assessment: Reliability - assess consistency in responses, Internal Validity - assess accuracy of casual relationships between factors, External Validity - assess generalizability of research findings, and Objectivity - assess research bias (Barroga et al., 2023). To ensure the rigor of this research, these four well-established criteria were used by dividing data analysis into three key stages:
1. Descriptive Statistics – Conducted to ensure reliability and objectivity by summarizing response patterns for each variable. Mean, Standard Deviation, and Frequency Distributions are calculated. 2. Effectiveness Measurement – Used to verify external validity by grouping responses into themes and calculating their mean score. 3. Regression Analysis – Used to confirm internal validity by identifying which factors contribute most to the overall effectiveness of VR training.
RESULTS AND DISCUSSION
The descriptive statistics analysis reveals that Usability and Engagement were the most highly rated themes by the respondents. Knowledge Acquisition and Knowledge Retention were moderately ranked, while Realism had the lowest rating. Regarding Realism in particular, the respondents showed a negative response. This signifies that that the VR training lacked sufficient realism and may require the development of a more realistic virtual environment. The results obtained from the one-sample t-test verified the findings from the descriptive statistical analysis showing that Usability, Engagement, and Knowledge Actuation had a notably higher mean score than the benchmark of 3.5. Although Knowledge Retention was slightly higher than the benchmark of 3.5 but closer, depicting more room for improvement, while the Realism mean score was lower than 3.5 and had a negative t-value. Below are the details for each analytical aspect.
Descriptive Statistical Analysis In the given descriptive statistics, the minimum value depicts the lowest rated value, and the maximum represents the highest rated value from the respondents for a given theme. The M value aggregates the average rating points, while the SD represents how spread out the datasets are. For M, the
Assessing Virtual Reality Fire Extinguisher Training Effectiveness 191
scale from 1–2.5 depicts low efficacy, 2.6–3.5 is neutral, and 3.6–5 indicates higher efficacy of VR-related themes. For SD, values less than 1 show consistency in the responses, while values above 1 indicate inconsistency of data. Table 2 shows the results produced by SPSS. It is clearly shown from Table 2 that the highest rated factors according to descriptive analysis were Usability with M = 4.07, SD = 0.76; and Engagement, M = 3.98, SD = 0.72. This indicates that VR training was considered easily comprehensible and engaging for trainees. Further, Knowledge Retention (M = 3.61, SD = 0.87) was lower than Knowledge Acquisition (M = 3.82, SD = 0.72). Though Knowledge Retention was M = 3.6 which falls under the positive side of scale, but closer to neutral. This implies that participants were able to acquire the knowledge well. They could retain the information but not confidently. In addition, SD is slightly higher which means, while some participants were able to retain information, others found information retention more difficult. However, the M was the lowest on Realism (M = 3.37, SD = 0.71), demonstrating that this aspect has room for improvement in any future iterations of VR to enhance the immersion experience.
Table 2: Descriptive statistics results from SPSS.
Figure 1 provides a visual illustration of the Frequency Distribution (FD) of each theme. Ratings ranging from 1.0 and 2.999 are considered negative for the corresponding theme, 3 to 3.99 are considered neutral, and 4 and above are categorized as positive. Figure 1 depicts that Usability has the highest positive rating with positive feedback from 57 participants, a neutral response from 12, and a negative rating from two participants only. Next to Usability is Engagement with 42 positive, 22 neutral, and seven negative responses from the participants. Similarly, Knowledge Acquisition and Knowledge Retention received 42 and 37 positive responses, 20 and 22 neutral, and nine and six negative responses respectively. This means that participants agree that VR training is usable, engaging and has a substantial knowledge acquisition and retention rate. Realism mostly received a neutral response (34) along with 21 positive and 16 negative responses. As shown in Figure 1, participants’ general perception regarding the overall training effectiveness is neutral with 30 positive, 34 neutral, and nine negative responses. This indicates that the participants perceived VR training as effective, engaging, and usable. The results also show that the VR training method has high knowledge acquisition and knowledge retention rates, however, their moderate response to realism demands more realistic VR scenarios.
Figure 1: Frequency distribution of five themes over responses–graphic representation.
Effectiveness Measurement To compare and analyze the impact of each theme, a one-sample t-test was conducted by setting the benchmark for effectiveness threshold at 3.5. This effectiveness threshold is set to 3.5 as the Likert scale ranges from 1–5. The mean value for a particular theme greater than 3.5 is considered effective, 3.5 is considered neutral, and those less than 3.5 may need improvement. Table 3 demonstrates the effectiveness of VR training using a one-sample statistic against a threshold of 3.5. The M values for Usability, Engagement, Knowledge Acquisition, and Knowledge Retention are 4.07, 3.98, 3.82, and 3.61 respectively. These four aspects of training were ranked above the benchmark of 3.5, confirming their effectiveness. Furthermore, Table 3 also illustrates that the one-sample t-test result value for Realism (p = −1.1585) is noticeably lower than p < 0.05 and signifies the need for potential improvements in the virtual realm in terms of realism. This t-test result also validates the research findings revealed in the descriptive statistic and frequency distribution.
Regression Analysis In this study, regression analysis was conducted to verify internal validity by measuring the impact of five key factors (Knowledge Acquisition, Knowledge Retention, Perceived Realism, Usability, and Engagement) on VR training effectiveness. Table 4 shows the results derived from IBM SPSS. Table 4 shows that the R (correlation coefficient) is 0.998 which depicts a strong relation between the dependent variable (overall Effectiveness) and the five independent variables (Knowledge Acquisition, Knowledge Retention,
Assessing Virtual Reality Fire Extinguisher Training Effectiveness 193
Perceived Realism, Usability, and Engagement). An R value closer to 1 shows a strong correlation between dependent and independent variables. Table 4 also indicates that the value of R square (R2) is 0.997. This describes that independent variables are explaining 99.7% of the variation in Effectiveness. Moreover, the values of Adjusted R2 and Standard Error of the Estimate are 0.996 and 0.03659 confirm high reliability and high predictive accuracy of the predictors (themes), respectively.
Table 4: Model summary of regression analysis.
Table available in the original PDF paper.
Model R R Square Adjusted R Square Std. Error of the Estimate 1 .998a .997 .996 .03659
To determine the impact of predictors (themes) on the effectiveness of VR training, a multiple linear regression analysis was conducted and the results are shown in Table 5. The results show that all five independent variables are statistically significant (p < 0.001). In the given Table 5, the value of Beta depicts the relative strength and effectiveness of each independent variable on the dependent variable. Engagement appeared as the strongest predictor of effectiveness with Beta = 0.392, followed by Realism (Beta = 0.334), Knowledge Acquisition (Beta = 0.317), and Knowledge Retention (Beta = 0.108). However, Usability with the lowest Beta = 0.060 reveals Usability as the weakest predictor of effectiveness. These results as shown in Table 5 statistically demonstrate that the most significant predictors of effectiveness are Engagement, Realism, and Knowledge Retention. Engagement is the most important factor in making VR training effective. This concludes that VR training effectiveness can be enhanced by improving Engagement, Realism, and Knowledge Acquisition. Usability and Knowledge Retention are important, but a more immersive and engaging experience has a greater impact on effectiveness.
This paper presents a quantitative analysis of VR training effectiveness based on empirical evidence. The effectiveness was measured against five key factors (Knowledge Acquisition, Knowledge Retention, Realism, Usability, and Engagement). Data collected from 71 participants offer valuable insights
into the perceived effectiveness of VR-based training. Table 6 gives an overview of all the results combined from the descriptive analysis, regression analysis, and one-sample t-test below.
The descriptive statistical analysis reveals that Usability was the most effective theme of the VR fire extinguisher training with M = 4.07. Next to Usability was Engagement with M = 3.98. These two were the most highly rated themes by the respondents. Knowledge Acquisition (M = 3.82) and Knowledge Retention (M = 3.61) were moderately rated, while Realism (M = 3.37) was the least favorable theme. This shows that the participants found that VR training lacked sufficient realism. This may suggest that VR training may require a more realistic environment.
The one-sample t-test conducted to analyze the impact of each theme also verifies the findings from the descriptive statistical analysis showing that Usability, Engagement, and Knowledge Acquisition had significantly higher M values than the benchmark of 3.5. Knowledge Retention, however, was higher but closer to the benchmark. Realism had the lowest M value, with a lower score than the threshold of 3.5, and a negative t-value. This indicates the need for improvements in VR training in the context of realism.
As shown in Table 6, the regression analysis disclosed that the selected five predictors were very effective at predicting VR training effectiveness and explain 99.7% of the variation in effectiveness. Engagement was the strongest predictor followed by Realism and Knowledge Acquisition. This highlights the significance of an engaging environment in order to keep participants involved and interested in the process.
By analyzing the results in Table 6, it can be concluded that immersive content and interactive elements enhance user involvement and consequently improve learning outcomes. Gamification, interactivity, and immersive features should be the top priorities. To provide a realistic environment in virtual realms, several measures can be taken such as enhanced visual fidelity, real-world simulation, improvised sound, realistic temperature, a timer for each action to create a sense of emergency, and so on. The safety training must also provide well-structured and comprehensible learning content for improved knowledge acquisition using scenario-based learning and interactive sessions. Knowledge retention is a significant predictor of effectiveness but not the primary driver. This can be increased using scenario- based learning and repetition of tasks. Usability is important but as long as the VR environment is easy to navigate, usability alone does not have a drastic impact on effectiveness. However, VR training must be easy to comprehend and be more engaging and realistic.
In conclusion, VR training is effective particularly in the context of Usability and Engagement. However, there is a need for improvement in Realism to enhance overall effectiveness. This research offers insightful guidance for researchers, educators, and developers in optimizing VR environments to deliver better services and to meet users’ expectations.
Assessing Virtual Reality Fire Extinguisher Training Effectiveness 195
Table 6: Overview of the combined analytical results - table summary.
Table available in the original PDF paper.
RQs Theme Mean (Average Rating Given By Participants)
We are immensely thankful to Business Finland for making this research possible by funding this project. Special tribute goes to the late Ebo Kwegyir- Afful, whose invaluable contributions and enthusiasm were instrumental in shaping this work. We highly appreciate the participants for their time and effort, as their involvement was essential to the success of this study.
REFERENCES
Bailenson, J. (2018). Experience on demand: What virtual reality is, how it works, and what it can do. New York: W. W. Norton & Company. Barroga, E., Matanguihan, G. J., Furuta, A., Arima, M., Tsuchiya, S., Kawahara, C.,
Takamiya, Y. & Izumi, M. (2023). Conducting and Writing Quantitative and Qualitative Research. Journal of Korean Medical Science, 38(37), e291. https://doi.org/10.3346/jkms.2023.38.e291 Drake, M. & Kauppinen, R. (2021). New Technology - A New Approach to Further Vocational Training? ICERI2021 Proceedings, 8005–8011. Hager, P. & Gongzi, A. (1996). What is Competence? Medical Teacher, 18(1), 15–18. Kwegyir-Afful, E. & Kantola, J. (2021). Simulation-Based Safety Training for Plant
Maintenance in Virtual Reality. In: Cassenti, D., Scataglini, S., Rajulu, S., & Wright, J. (Eds.), Advances in Simulation and Digital Human Modeling. AHFE 2020. Advances in Intelligent Systems and Computing, vol. 1206. Cham: Springer. Lovreglio, R., Duan, X., Rahouti, A., Phipps, R. & Nilsson, D. (2020). Comparing the effectiveness of fire extinguisher virtual reality and video training. Virtual Reality, 25(1) , 133–145. https://doi.org/10.1007/s10055-020-00447-5 Morélot, S., Garrigou, A., Dedieu, J. & N’Kaoua, B. (2021). Virtual reality for fire
safety training: Influence of immersion and sense of presence on conceptual and procedural acquisition. Computers & Education, 166, 104145. Mystakidis, S., Besharat, J., Papantzikos, G., Christopoulos, A., Stylios, C., Agorgianitis, S. & Tselentis, D. (2022). Design, Development, and Evaluation of a Virtual Reality Serious Game for School Fire Preparedness Training. Educ. Sci, 12, 281. https://doi.org/10.3390/educsci12040281 Rahouti, A., Lovreglio, R., Datoussaïd, S. & Descamps, T. (2021). Prototyping and
validating a non-immersive virtual reality serious game for healthcare fire safety training. Fire Technology, 57(6) , 3041–3078. https://doi.org/10.1007/s10694- 021-01098-x
Ristor, R., Morélot, S., Garrigou, A. & Kaoua, B. N. (2023). Virtual reality for fire safety training: Study of factors involved in immersive learning. Virtual Reality, 27(3) , 2237–2254. https://doi.org/10.1007/s10055-022-00743-2 Saghafian, M., Laumann, K., Akhtar, R. S. & Skogstad, M. R. (2020). The Evaluation
of Virtual Reality Fire Extinguisher Training. Frontiers in Psychology, 11, 593466. https://doi.org/10.3389/fpsyg.2020.593466 Scorgie, D., Feng, Z., Paes, D., Parisi, F., Yiu, T. W. & Lovreglio, R. (2024). Virtual reality for safety training: A systematic literature review and meta-analysis. Safety Science, 171, 106372. https://doi.org/10.1016/j.ssci.2023.106372 Stefan, H., Mortimer, M. & Horan, B. (2023). Evaluating the effectiveness of virtual
reality for safety-relevant training: A systematic review. Virtual Reality, 27, 1–22. https://doi.org/10.1007/s10055-023-00843-7 Sun, L., Lee, B. G. & Chung, W. Y. (2024). Enhancing Fire Safety Education Through Immersive Virtual Reality Training with Serious Gaming and Haptic Feedback. International Journal of Human–Computer Interaction, 1–16. https://doi.org/ 10.1080/10447318.2024.2364979
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors, Business Management and Society
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-52-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional