Md Abdullah, Vibhav Nirmal, Mahmudur Rahman (2024). Comparing the Efficacy of Virtual Reality Training, Augmented Reality Instruction and Traditional Paper-Based Instruction Methods for Assembly Tasks. In: Human Factors in Design, Engineering, and Computing. AHFE Open Access, vol. 159, pp. 1038–1047. https://doi.org/10.54941/ahfe1005670
The landscape of instructional methods is continually evolving, driven by advancements in technology. Virtual Reality (VR) and Augmented reality (AR) are emerging technologies offering innovative approaches to training and providing real-time assistance. In order to compare the efficacy of these methods in assembly tasks, in this study, 24 participants were randomly assigned to three groups: one group used paper-based instructions, while another used instruction displayed through the HoloLens 2.0 (AR), and the remaining group was trained in a fully immersive VR environment and were asked to perform the same assembly task afterward. The participants were tasked with assembling a monster truck Lego set, and their performance was measured using objective metrics such as completion time and the number of errors made. Subjective measures were obtained through the NASA Task Load Index (TLX) questionnaire, which assessed the perceived workload of each instructional method. Participants using paper-based instructions completed the task in an average of 5.92 minutes, which was significantly faster than those using AR (average completion time of 8.21 minutes), and those using VR training (average completion time of 7.23 minutes). The number of errors was highest with the VR training, averaging 2 errors per participant, compared to the paper-based instructions (0.625 errors) and AR (1.25 errors). Subjectively, participants rated the AR experience slightly higher, with an average NASA TLX score of 23.26, compared to 26.25 for VR training. Paper-based instructions had the lowest workload value, with a mean NASA TLX score of 17.60. The findings suggest that while VR and AR offer advanced learning experiences, they may not always outperform traditional paper-based instructions in terms of task completion time and error rates. These results emphasize the need to consider task complexity and user experience when evaluating instructional methods. Further research is needed to explore the benefits of VR and AR in different contexts.
An assembly task is a type of task that involves putting together various components or parts to create a finished product. Manual assembly work is a critical task in various industries, from manufacturing to construction, where the efficiency and accuracy of assembling products can significantly
Comparing the Efficacy of Virtual Reality Training, Augmented Reality Instruction 1039
impact overall productivity and quality. Optimizing the assembly process by reducing cycle time and minimizing errors is essential for enhancing operational efficiency and maintaining competitive advantage. In order to enhance productivity and smooth assembly operation, proper instructional and effective training methods are vital. As technology advances, new instructional methods such as VR and AR are emerging, offering innovative approaches to training and real-time assistance in manual assembly work. Technological progress has led to a constantly changing landscape of learning, training, and instructional methods. Traditional paper-based instructions have long been the standard for guiding people through various tasks (Daling and Schlittmeier, 2024). With the advent of VR and AR, new training and instructional methods are emerging that have been proven to be more effective in literature.
AR superimposes digital information over the real world, enhancing the user’s perception and interaction with their environment (Carmigniani and Furht, 2011). AR devices such as HoloLens 2.0 project instructions and guidance directly into the user’s field of view. This real-time assistance helps users complete tasks more efficiently by providing contextual information and reducing the need to look away from the workspace. Literature has reported that VR is highly effective in training and skill retention. In assembly work, skill plays a vital role (Abdullah and Suer, 2019). VR provides a fully immersive digital environment that simulates real-life scenarios and provides users with immersive and interactive learning without using actual physical resources (Pirker and Dengel, 2021). This technology has the potential to improve skills by allowing people to train and improve their learning in a controlled and fully immersive environment (Checa and Bustillo, 2020). VR is particularly useful for complex tasks that require spatial awareness and hands-on interaction because it can mimic the physical environment and related objects. Until recently, traditional paper instructions have been widely used due to their simplicity, ease of use, and lack of technical infrastructure (Daling and Schlittmeier, 2024).
Different studies reported different outcomes of VR and AR instruction. Carlson et al. (2015) conducted a study to assess knowledge retention from virtual reality training for assembly operations. They reported that participants who received physical training completed the assembly task faster (15.09 seconds) compared to those trained virtually (55.41 seconds). Similarly, Hamblin (2005) found real-world training to be more effective than virtual training for an assembly task within a virtual environment. In another study, Adams et al. (2001) investigated learning transfer by having participants construct a Lego biplane model in a virtual environment with haptic force feedback, concluding that haptic feedback was essential for effective learning transfer. Drouot et al. (2022) compared computer-based and AR-based instruction for simple and complex workstation assembly tasks, finding that participants completed the task faster in the computer condition (540.50 seconds) than in the AR condition (851.58 seconds). Yang et al. (2019) reported that AR assistance reduced the overall time required for assembly tasks (131.43 seconds) compared to screen-based documentation (159.64 seconds), while also minimizing errors and reducing
cognitive load during the commissioning subtask, though it increased cognitive load during the joining task. Hou and Wang (2013) compared 3D manual and AR training for assembly tasks, focusing on gender effects, and found that AR-based assembly significantly reduced the mean completion time (7.37 minutes) compared to 3D manual assembly (11.91 minutes).
From previous literature, it is evident that the impact of technology on improving assembly tasks in terms of completion time and error rate is mixed, and comparisons between augmented reality and virtual reality are limited. The purpose of this study is to compare the effectiveness of VR training, AR instruction, and traditional paper instructions in the context of assembly tasks. By examining how each method affects performance and user experience, we aim to identify the strengths and limitations of VR and AR technologies compared to traditional approaches. In this study, participants assembled a monster truck Lego set, a moderately difficult task requiring attention to detail and accuracy. Performance was measured using objective metrics such as completion time and error rate, and subjective measures were obtained using the NASA Task Load Index (TLX) questionnaire to assess the workload associated with each learning method. Understanding the effectiveness of these teaching methods is crucial for optimizing education in various fields. Insights from this study will help develop more effective training programs that leverage the strengths of VR and AR technologies while considering the practicality and accessibility of traditional methods.
METHODOLOGY
Participants A total of 24 participants (20 males and 4 females) were recruited. Participants were aged between 18 and 45. All the participants were the University of Texas at Arlington students. Five students reported that have used AR or VR at least once. All participants were informed about the study and were given a consent form.
Experimental Task In this study, a Lego Monster Truck was used for assembly tasks. The Lego originally had 148 pieces. 120 pieces were pre-assembled, and a main body was formed to reduce the number of steps to 10. The task was to assemble the last 28 pieces in the main body in 10 unique steps. Figure 1a) shows the monster truck and Figure 1b) shows the main body and the remaining 28 parts.
Procedure Three different methods of instruction were used to guide participants for assembly tasks: (i) Paper-based instruction (ii) Augmented reality-based instruction and (iii) Virtual reality (VR) training. AR instruction was shown in Hololens 2.0 and VR training was implemented in MetaQuest 3.0. Figure 2a) shows instruction in Hololens 2.0, and Figure 2b) shows the VR field of view. The instruction in Hololens 2.0 and MetaQuest 3.0 was developed in Unity 3D.
Comparing the Efficacy of Virtual Reality Training, Augmented Reality Instruction 1041
A between-subject design was implemented. A total of 24 participants were recruited and they were divided into three groups. The first group followed the step-by-step instructions from printed paper to complete the assembly task. The second group followed the instructions in AR (Hololens 2.0). In AR, the main body and relevant parts were shown in the field of view (FOV) as shown in Figure 2a). There were four buttons in every step such as ‘Next Animation’, ‘Previous Animation’, ‘Go to Next Step’, and ‘Go to Previous Step’. Participants needed to press the buttons using hand gestures.
Figure 1: a) Monster truck (final product) b) Lego main body and parts.Figure 2: a) Field of view in Hololens 2.0 b) Field of view in MetaQuest3.0.
Pressing the ‘Next Animation’ button will show them the animation of parts moving and getting attached to their designated place. If the participant wants to see the animation again and make sure where to attach the part he/she can press the ‘Previous Animation’ button. Also, they had the option to walk near the body and look from a different angle to confirm the location where to attach. Once they are done with one step, they need to press the ‘Go to Next Step’ button to watch the animation of the next assembly task. For the third group at first, they were asked to watch the whole assembly process several time step-by-step using the same process as described for Hololens 2.0. There was no time restriction during the training period. Once they completed the training, they need to take off the headset and start the assembly task. They were allowed to go back to the VR headset if they needed to recheck the instructions.
Two objective measures were recorded:
1. Completion time: Time to complete the assembly tasks. 2. Number of errors: Error was defined as i. if a part is grabbed and placed in the wrong place ii. If the sequence was violated (for VR this was not the case).
Subjective measure for workload was recorded using NASA TLX. Additionally, a usability survey was conducted to assess the usefulness of all three methods and to evaluate users’ comfort and willingness to use AR or VR for extended periods.
RESULTS AND DISCUSSIONS Completion Time
A one-way ANOVA was conducted to compare means for completion time. The p-value for the mode of instruction (MOI) is 0.271, which is greater than 0.05. This indicates that the effect of MOI on completion time (CT) is not statistically significant at the 5% significance level. Therefore, there is insufficient evidence to reject the null hypothesis, suggesting that the mode of instruction does not have a significant effect on completion time. The homogeneity of variance and the normality assumptions were tested using Bartlett’s test and Shapiro-Wilk test. The results of these tests indicated that the assumptions are met.
The mean completion times for three different modes of instruction (Table 1) Paper, AR, and VR reveal distinct differences in efficiency. Participants using the paper-based method had the shortest average completion time of approximately 5.92 minutes (SD 2.89 minutes). In contrast, participants using AR took the longest, with an average completion time of about 8.21 minutes (SD 3.86 minutes). The VR method falls in between, with an average completion time of 7.24 minutes (SD 1.76 minutes).
These results suggest that the complexity of interacting with digital interfaces like AR and VR may contribute to longer completion times compared to the straightforward nature of paper-based tasks. The longer times for AR and VR could also be attributed to a steeper learning curve
Comparing the Efficacy of Virtual Reality Training, Augmented Reality Instruction 1043
and less familiarity with the technology. While advanced technologies offer innovative interaction methods for complex tasks, the data indicates that traditional paper-based methods currently enable quicker task completion since the tasks were relatively simple consisting of only 10 steps without much requirement of maintaining sequence. To make AR and VR more competitive in terms of efficiency, additional user training and interface optimization may be necessary. The boxplot indicates that participants using paper instructions generally completed the assembly task faster and more consistently than those using AR or VR. AR had the longest and most variable completion times, while VR had intermediate completion times with moderate variability.
Figure 3: Boxplot of completion time.
These results suggest that the complexity of interacting with digital interfaces like Hololens 2.0 and MetaQuest 3.0 may contribute to longer completion times compared to the straightforward nature of paper-based tasks. The longer times for Hololens 2.0 and VR could also be attributed to a steeper learning curve and less familiarity with the technology. While advanced technologies offer innovative interaction methods, the data indicates that traditional paper-based methods currently enable quicker task completion.
Table 1. Mean completion time, mean error, and mean NASA TLX in different modes of instruction.
Error The analysis of error rates across the different modes of instruction (Paper, Hololens, and VR) revealed some interesting findings. The ANOVA results indicated a borderline non-significant difference in error rates across the groups, with a p-value of 0.06. This suggests that, while there is some variation in error rates between the modes of instruction, it is not statistically significant at the 0.05 level. The mean error rates per participant (Table 1) show that paper-based instructions had the lowest error rate (0.63), followed by AR (1.25), and VR had the highest error rate (2.0). The mean and standard deviation values of Error are listed in Table 1. Examining the descriptive statistics and boxplots, we observe that participants using VR tended to make more errors compared to those using Paper and AR. The assumptions of homogeneity of variance and normality were assessed using Bartlett’s and Shapiro-Wilk tests, respectively. The test outcomes confirmed that these assumptions were satisfied.
Figure 4: Boxplot of error.
NASA TLX The NASA TLX scores which measure the perceived workload associated with each mode of instruction indicate that participants experienced the lowest workload with paper-based instructions (17.60), followed by AR (23.65), and the highest workload with VR (26.25). The mean and standard deviation value of NASA TLX are listed in Table 1. This suggests that traditional paper-based methods are perceived as less mentally and physically demanding compared to AR and VR methods. The analysis of NASA
Comparing the Efficacy of Virtual Reality Training, Augmented Reality Instruction 1045
Task Load Index (NASA TLX) scores indicated no statistically significant difference across the groups. The ANOVA results showed a p-value of 0.458, suggesting that the perceived workload did not vary significantly between participants using Paper, AR, and VR. This insight, although not statistically significant, provides a valuable understanding of the user experience across different instructional methods.
The Bartlett test for homogeneity of variances indicated no significant difference in variances of NASA TLX scores across the groups (p- value = 0.105), suggesting consistent variability in perceived workload among the different instructional methods.
Figure 5: Boxplot of NASA TLX.
The normality assumptions were tested using Shapiro-Wilk test. The results of these tests indicated that the normality assumption is met.
From the usability survey, participants rated both Hololens and VR highly. The average usefulness rating for Hololens was 6.37, (SD = 0.74) out of 7.0, with an average comfort rating of 6.12 (SD = 0.35) out of 7.0. For VR, the average usefulness rating was 6.0 (SD = 1.06), and the average comfort rating was 6.12 (SD = 0.83). However, none of the participants were willing to wear the devices (Hololens 2.0 or MetaQuest 3.0) for more than three hours at a stretch.
CONCLUSIONS
The results of this study provide valuable insights into the effectiveness of different instructional methods for assembly tasks. Although the mean completion time was shortest for the paper-based instruction method, the differences in completion times across the three modes of instruction (Paper,
AR, and VR) were not statistically significant. This suggests that while advanced technologies like VR and AR offer innovative ways to interact with the instructional material, they may not necessarily improve efficiency in terms of task completion time compared to traditional paper-based methods for simple tasks. The error rates did show a trend towards higher errors with VR, followed by AR, and the least with paper instructions. This indicates that the complexity and potential unfamiliarity with digital interfaces may lead to more mistakes, highlighting the need for additional user training and interface optimization to reduce errors and improve performance. Additionally, participants reported the lowest perceived workload with paper-based instructions, as reflected in the NASA TLX scores. These findings suggest that while VR and AR have potential benefits, especially for complex tasks requiring spatial awareness, their current implementations may require further refinement to match the efficiency and ease of use of traditional methods. Future research should continue to explore the contexts in which VR and AR can be most beneficial and investigate ways to optimize these technologies to enhance learning and performance in assembly tasks.
The study has several limitations. First, the animations and 3D model designs could benefit from improvements to enhance their realism and seamlessness. Additionally, incorporating an on-screen list of instructions might increase efficiency and reduce the time spent flipping through pages, as opposed to relying solely on model animations for each step. Although the paper-based instructions were clear and easy to follow, there may be instances where visualizing and understanding the context or model is essential. Finally, the results could differ for participants with extensive experience using HoloLens or VR (more than 3–4 hours), potentially affecting the study’s outcomes.
REFERENCES
Abdullah, M. and Süer, G. A., 2019. Consideration of skills in assembly lines and seru production systems. Asian Journal of Management Science and Applications, 4(2), pp. 99–123. Adams, R. J., Klowden, D. and Hannaford, B., 2001. Virtual training for a manual
assembly task. Carlson, P., Peters, A., Gilbert, S. B., Vance, J. M. and Luse, A., 2015. Virtual training: Learning transfer of assembly tasks. IEEE transactions on visualization and computer graphics, 21(6), pp. 770–782. Checa, D. and Bustillo, A., 2020. A review of immersive virtual reality serious games
to enhance learning and training. Multimedia Tools and Applications, 79(9), pp. 5501–5527. Carmigniani, J. and Furht, B., 2011. Augmented reality: an overview. Handbook of augmented reality, pp. 3–46. Daling, L. M. and Schlittmeier, S. J., 2024. Effects of augmented reality-, virtual
reality-, and mixed reality–based training on objective performance measures and subjective evaluations in manual assembly tasks: a scoping review. Human factors, 66(2), pp. 589–626. Drouot, M., Le Bigot, N., Bricard, E., De Bougrenet, J. L. and Nourrit, V., 2022.
Augmented reality on industrial assembly line: Impact on effectiveness and mental workload. Applied Ergonomics, 103, p. 103793.
Comparing the Efficacy of Virtual Reality Training, Augmented Reality Instruction 1047
Hamblin, C. J., 2005.Transfer of* training from virtual reality environments. Wichita State University. Hou, L. and Wang, X., 2013. A study on the benefits of augmented reality in retaining
working memory in assembly tasks: A focus on differences in gender. Automation in Construction, 32, pp. 38–45. Hou, L., Wang, X., Bernold, L. and Love, P. E., 2013. Using animated augmented reality to cognitively guide assembly. Journal of Computing in Civil Engineering, 27(5), pp. 439–451. Pirker, J. and Dengel, A., 2021. The potential of 360 virtual reality videos and real VR for education—a literature review. IEEE computer graphics and applications, 41(4), pp. 76–89. Koumaditis, K., Chinello, F., Mitkidis, P. and Karg, S., 2020. Effectiveness of virtual
versus physical training: The case of assembly tasks, trainer’s verbal assistance, and task complexity. IEEE Computer Graphics and Applications, 40(5), pp. 41–56. Yang, Z., Shi, J., Jiang, W., Sui, Y., Wu, Y., Ma, S., Kang, C. and Li, H., 2019.
Influences of augmented reality assistance on performance and cognitive loads in different stages of assembly task. Frontiers in psychology, 10, p. 458057.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Design, Engineering, and Computing
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-35-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional