Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluating an Immersive Learning Environment for Robotics Training

Florida International University, Architecture Department, 11200 SW 8 th Street, Miami, FL 33199, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1002400

Abstract

In Spring 2021, an interdisciplinary team of researchers at Florida International University (FIU) designed a virtual reality (VR) training prototype for novices to learn how to work with industrial robots. Developed with the support of a grant from the National Science Foundation (NSF) by a team of architecture and computer science faculty, the Robotics Academy immersive learning environment prototype leverages advanced technologies to teach robotics in a fully immersive VR environment. This paper will describe the learning environment, the introductory lesson prototype, the learning evaluation tools, and the comparative outcomes of testing this learning prototype with a test group and a control group.As robotic automation continues to transform manufacturing, construction, and other industries, VR may offer a solution for training the labor force for more technically demanding jobs. VR provides computer-generated simulations of the real or an imagined environment that can serve as a rich and engaging space for learning (Mantovani et al., 2003). Recent research demonstrates that immersive environments can facilitate learning (and the assessment of learning) by providing a safe and low-cost setting for practice and rehearsal (Beck, 2019). Training workers to operate robots in a traditional classroom setting often relies on low teacher to student ratios as a means for accommodating individualized or small group coaching using a dedicated training robot. This pedagogical method can be both costly and time consuming. Meanwhile, on-the-job training can both slow down production and expose inexperienced trainers and trainees to potentially hazardous conditions. Immersive virtual learning environments offer a potential solution to reduce the cost of traditional training and mitigate exposure to hazardous conditions while learning how to operate industrial robots.The design team for the Robotics Academy created an immersive learning environment with simulated robots and input devices while the curriculum team developed both a script introducing the fundamentals of industrial robotic safety and a series of self-directed activities for learning how to operate an industrial robot. To measure the effectiveness of our VR learning tool the evaluation team offered 45 minutes of self-directed learning using a VR headset to a test group of twenty-one second year architecture students with no prior knowledge or experience working with industrial robots. A control group of twenty-one second year architecture students with similar background received training using the same script paired with an image-based slide lecture in a traditional classroom setting, but they were not provided access to the VR training tool or practice time to work with a robot. Both groups were tested with a short quiz to assess their retention of key concepts from the script and a practicum test using a teach pendant input device for controlling an industrial robot. Finally, students were asked to rate their own level of confidence, self-reliance, and readiness to proceed to the next level of training. On the written test students showed similar rates of retention of key concepts from the training script with a modestly higher average score for in-person training over the VR training tool. However, in a series of timed exercises, students who used the VR training tool demonstrated higher levels of task accomplishment with fewer errors and faster completion times for practicum testing. Finally, those who used the VR training tool reported higher levels of self-confidence. While more learning outcome testing is necessary, these initial results indicate that immersive learning environments like our VR tool may be an effective method for educating the labor force for jobs that involve automation with technology such as industrial robots.

© 2022. Published by AHFE Open Access. All rights reserved. 188

Evaluating an Immersive Learning Environment for Robotics Training 189

INTRODUCTION

The Robotics Academy is an immersive learning environment developed by an interdisciplinary team of researchers from FIU with support from the NSF. The Robotics Academy aims to address anticipated changes in labor needs in the Architecture, Engineering, and Construction (AEC) industry, particu- larly in automated construction, by offering robotics training in a low-cost and hazard-free learning environment. Our team designed an introductory robotics curriculum in virtual reality (VR) to determine whether immersive learning environments offer a viable solution for upskilling and training for high-tech automated construction jobs. The curriculum covered basic topics including robotic safety, robot anatomy, and moving (or jogging) the robot using an input device. We developed a series of elementary lessons and evalu- ation tools to 1) evaluate the effectiveness of an immersive environment for learning fundamental concepts about robotics; 2) test the transferability of practical skills from a virtual environment to a real-world scenario; and 3) assess the relative confidence level of novice robot operators who had practi- ced robotic operations in a VR environment versus those who had not been offered an opportunity to practice.

The qualities of immersion and interactivity that VR technologies offer can promote active experiential learning, a key foundation of constructivist theory of learning. Constructivist theory proposes that learning is an active process in which individuals create meaning from their own experiences by differentiating between the attributes of objects and their abstractions (Pia- get, 1977). Immersive learning environments have been shown to promote embodied cognition for conceptual development and to facilitate knowledge production as a result of experience. At the same time, interactivity can pro- vide targeted feedback to facilitate experimentation, and exploration (Jarmon et al., 2009). Our hypothesis was that learning would be enhanced due to the immersive nature of the VR environment which promotes a sense of situa- ted embodiment, a quality that has been shown to have positive effects on learning outcomes (Lindgren & Johnson-Glenberg, 2013). Furthermore, we anticipated that the affordance for self-directed interactivity within the lear- ning environment would result in better learning outcomes for those who used the VR learning tool versus those who did not (Beck, 2019). Finally, we anticipated that skill transferability between the virtual learning environment and the real world would lead to higher levels of self-confidence for those who rehearsed robotic operations in a virtual environment (Dede, 2009).

THE ROBOTICS ACADEMY The Robotics Academy immersive learning environment was developed in the Unity Game Engine for the Vive Pro Eye headset as the Robotics Academy Virtual Training Facility.

It was modeled using Autodesk Maya as an idealized robotics training facility that includes a series of standard work cells with a variety of Kuka industrial robots. The work cells demonstrate simulations of industrial robots performing typical tasks such as pick and place, drilling, and fastening

Figure 1: Robotics Academy immersive learning prototype features voice-over instru- ctions to guide students through course navigation and virtual work cells with simulated robots.
Original PDF Figure 1:

operations. The VR space allows the user to move from one work cell to another in order to view educational content delivered in voice-over text.

The Unity Game Engine allowed the integration of real-time user inputs with immediately responsive robotic movement simulations and voice-over text. Robotic movements were generated using the real-time animation functionality of the robotic movement simulation software RoboDK. Voice- over text was processed using Microsoft Azure text-to-voice service to pro- vide an expressive and life-like audio track to accompany real-time visual narratives and to provide explanatory directions for task-oriented activities.

The initial training session features a preliminary tutorial describing how to interact in the virtual space by turning and moving the body or by using the Vive Pro Eye hand-held controllers for orienting or teleporting toward a desired location. User input is simulated using a virtual input device that resembles the layout and functionality of a Kuka SmartPAD Teach Pendant. Students are oriented to the virtual input device using voice-over instructi- ons following a standard training protocol that replicates instructions that would be provided in a classroom environment. Wherever possible, buttons on the virtual input device have been designed in analogous locations to the Kuka SmartPAD Teach Pendant including the left-hand Safety Interlock swi- tch, actuated by the trigger on the left-hand Magic Leap controller, and the Emergency Stop button located above and outside the main body of the input panel.

The Introductory Lesson Prototype The introductory lesson on the Fundamentals of Industrial Robotic Arms displays several robotic use cases with either embedded video in a floating window or an animated simulation in a work cell demonstrating the sequ- ential steps of each example with voice-over narration providing additional informative context with a particular emphasis on safety and best practices. The lesson focuses on essential safety issues for working with robotic arms by including the concepts of work envelope, work cell, industrial robot versus coworking robot, operation modes and their features, and the use of safety devices such as the safety interlock switch and the emergency stop button.

After introducing the functionality and use of the virtual input device, a series of animations in a simulated work cell demonstrate the movement and

Evaluating an Immersive Learning Environment for Robotics Training 191

Figure 2: The virtual input panel closely mirrors the design and layout of the Kuka SmartPAD.
Original PDF Figure 2:
Figure 3: Students are challenged to match the position and orientation of a “ghosted” robotic arm using both Axis-specific and World-specific motion systems.
Original PDF Figure 3:

payload capabilities of a Kuka KR10-R1100 industrial robot highlighting its specific features and identifying each axis and its range of movement. The lesson introduces two coordinate systems for moving or jogging the robot: Axis-specific motion describes the motion of an industrial robotic arm using polar coordinates assigned to each of the 6 rotational axes (A-1 through A-6); World-specific motion uses cartesian coordinates with orientation positions pitch, roll, and yaw (X, Y, Z, A, B, C).

The lesson includes a fifteen-minute period in which to explore the Robo- tics Academy virtual learning environment and test out various ways of jogging the robot using both Axis-specific and World-specific motion types. We developed a position matching protocol whereby a second robot arm

Figure 4: Students in the control group received a standard slide lecture presenta- tion by an instructor using an identical script to the voice-over text in the immersive learning environment.
Original PDF Figure 4:

is displayed in a “ghosted” view-mode that illuminates when the simu- lated robotic arm controlled by the student aligns with the target posi- tion. The ghosting protocol serves as a reinforcement tool for learning to memorize each axis and its movement parameters through self-directed rehearsal.

Control Group While the experimental group of twenty-one students completed their indi- vidual, self-directed learning in virtual reality, a control group of twenty-one students with similar background experience were offered a thirty minute in-person slide lecture covering the same content provided in the immersive learning environment. The text for the lecture was drawn from the same source that was used to create the voice-over text in the immersive lear- ning environment but was edited to remove any reference to navigation or description of the virtual reality setting.

The presentation was delivered verbally in our robotics laboratory using a series of visual aids that complemented the text and were analogous to images, simulations, or animations presented in the immersive learning envi- ronment. These included an image of a Kuka KR10-R1100 robotic arm, an illustration of its work envelope, a live demonstration of axis identification (rotating each axis in sequence to identify its numeric designation and range of motion) and movement parameters, and an image of the Kuka SmartPAD Teach Pendant with various buttons or regions of the display highlighted to show their particular characteristics, functionality, and practical use. In this way, participants in the control group were presented with the same infor- mation and similar visual aids provided to the experimental group. While the classroom lecture included real-time demonstrations of robotic movement and visual access to the Kuka SmartPAD it did not allow direct interaction by students with the robot or its input device.

Evaluating an Immersive Learning Environment for Robotics Training 193

Testing Protocol Approximately forty-eight hours after their classroom or VR learning expe- rience both the experimental group and the control group were given an identical ten-question quiz. They were then invited to the lab individually to perform a brief practicum test using the Kuka SmartPAD to control a KR10- R1100 training robot. The practicum test included instructions to position the robot in specific poses and to take corrective actions in hypothetical sce- narios such as a person unexpectedly entering the work cell while the robot is in motion. Testing proctors timed these exercises and reaction times counting the number of button misidentifications (errors in both axis identification and rotational direction). Finally, study participants were asked to assess their level of self-confidence, self-reliance, and readiness to proceed to the next level of training.

Written Quiz: Ten questions covering topics directly referenced in verbal text, images, animations, or live demonstrations of both the lecture and the immersive learning environment administered two days after learning content delivery.

Practicum Test: Five scenarios covering robotic movement, motion types, and safety protocols administered two days after learning content delivery and on the same day as the written quiz.

Self-Assessment: Self-reported levels of self-confidence, self-reliance, and readiness to progress to next level of training using a ten-point scale admi- nistered two days after learning content delivery, after the written quiz, and following practicum testing.

Unconscious Bias Mitigation: Testing and VR training proctors were provided with scripts for greeting study participants and delivering verbal instructions during written and practicum testing, as well as a rubric for recording performance results.

Analysis: Individual written test, practicum test, and post-testing self- assessment results were compiled and analyzed using a standardized percentile-based scoring system. The sum of scores in each testing protocol was divided by the number of participants in each group to establish ave- rage comparable scores for the experimental group and the control group. Finally, the granular data for each testing protocol was analyzed for common error cluster patterns to identify potential correlations between the training modality and testing outcomes.

Comparative Outcomes of Testing Comparative analysis of testing results reveals meaningful findings:

Written quiz scores were 2.38% lower for the experimental group indi- cating that while the immersive learning environment provided comparable learning outcomes to an in-person class, some aspect of the VR experience led to lower-than-expected levels of concept formation and learner recall of key information appeared to be less robust than expected. The signifi- cantly longer average completion time for the written quiz (43% longer) further indicates that the experimental group struggled to recall information

Table 1. Testing scores for experimental group and control group.

Table available in the original PDF paper.

delivered in the immersive learning environment when compared to peers who receive the same information in a classroom setting.

Practicum testing scores were 1.90% higher for the experimental group indicating that the opportunity for interactive and self-directed practice pro- vided a modest advantage for those afforded the use of the immersive learning environment. While axis rotation direction errors were comparable between both groups (a common error for novice industrial robot operators), the expe- rimental group exhibited 53.34% fewer axis identification errors indicating greater familiarity with robot anatomy. However, in hypothetical scenarios such as a person unexpectedly entering the work cell while the robot is in motion, the experimental group demonstrated significantly lower ability to respond quickly and correctly to such real-world safety concerns.

Self-assessment scores were 3.02% higher for the experimental group indi- cating that use of the immersive learning tool translated to higher levels of self-confidence and a greater sense of self-reliance when operating the trai- ning robot. Moreover, the experimental group reported 4.3% higher levels of readiness to move on to the next training lesson, indicating that practice in the virtual training environment may have served as a kind of proxy for real-world experience.

CONCLUSION

Initial testing of the Robotics Academy immersive learning environment suggests several potential strengths and shortcomings to the design of our robotics curriculum and our immersive learning environment.

Evaluating an Immersive Learning Environment for Robotics Training 195

Prototype Strengths Preliminary analysis suggests that, at the very least, our immersive learning tool can produce comparable, quantifiable learning outcomes to a classroom setting. Given that our results fall within statistical margins of error, we can infer that the learning in the immersive learning environment is at least as effe- ctive as in-person training for learning fundamental concepts about robotics. This indicates that our VR learning tool can serve as an effective replace- ment for certain components of robotics training. However, our sample size is small, and our VR curriculum is limited to fundamental lessons. Additional development and testing will be required to verify our findings.

Further analysis of practicum testing shows that learners in the experi- mental group who were given the opportunity to practice jogging a robot in a virtual environment using a simulated input device were able to tran- sfer skills acquired in VR to the real world despite modest differences to the input device. This reinforces our assumption that practice operating simula- ted robotic tools in an immersive environment can support skill development for novice robotics operators.

Finally, results from the self-assessment testing bolsters our position that skill transfer between the virtual and real environments leads to greater levels of self-confidence among learners who are afforded the opportunity to practice skill development using VR. This suggests that practice in an immersive environment can be a significant factor in novice robotics ope- rators’ sense of their own readiness to proceed to more advanced levels of training.

Prototype Shortcomings It was our assumption that learning outcomes would be higher for those using the immersive learning environment, but we recorded mixed results. This indicates that some aspect of the learning environment may have inhibited effective learning. The affordance for students to identify with a teacher as a corporal being in a physical space may have more powerful impacts than we anticipated. Vygotsky (1978) argues that social interaction within the lear- ning process is a critical factor for supporting effective concept formation. It is possible that deficiencies in our own conception of embodiment and its expression in the immersive learning environment may have negatively impacted anticipated learning outcomes. Instruction within the VR lessons is provided using a procedurally generated voice from the Microsoft Azure text to speech service that, while somewhat lifelike, is identifiably produced by a computer. Meanwhile, the period of self-directed exploration and rehearsal at the end of the lesson features no voice-over interaction, furthermore, it situates individual learners in front of a robotic work cell without proximity to another person or a simulated being. These two qualities may hinder the social aspect of learning enough to inhibit effective and durable concept for- mation. The embodied personage of a human lecturer may promote a level of engagement or social connection that results in sufficiently scaffolded lear- ning to help improve recall of information. This suggests that a more lifelike voice-over narrative and an avatar or figure to deliver that information in a

simulated personage is an important consideration for the next iteration of the Robotics Academy Virtual Training Facility.

While practice in the VR environment appears to have improved overall scores in practicum testing, students who received their training in VR were slower to respond to hypothetical emergency scenarios. This is a concerning result that indicates that experiences in a simulated environment may depress the sense of urgency to respond in the real world. While there is some research that supports this observation (Smith & Burd, 2019), additional testing will be required to verify the results and form more robust conclusions about the impact of VR exposure on reaction times.

Finally, we anticipated that the ability to practice operating simulated robots in a virtual environment would lead to high levels of skill transfe- rability from the virtual to the real world, and it was our assumption that this would result in high levels of self-confidence, self-reliance, and sense of readiness to proceed to more advanced lessons. While testing bears out this assumption, we were surprised that the ability to develop transferrable skills did not appear to have a greater impact on self-confidence in real-world sce- narios. Further study will be required to substantiate our initial findings and we look forward to conducting additional research to discover what under- lies the relationship between practice in immersive learning environments and perceptions of self-confidence in real-world scenarios.

Results of the Study The Robotics Academy immersive learning environment prototype demon- strates the effectiveness of VR as a tool for novices to learn about and develop new skills to work with industrial robots. Preliminary testing indicates that there is significant skill transferability between the VR setting we designed and the real world. Furthermore, our testing indicates that the experience of practicing or rehearsing in a virtual environment is correlated with higher levels of self-confidence compared to a control group who were not affor- ded opportunities to practice. While additional testing is required, our initial results indicate that VR can be an effective method for educating the future AEC labor force for jobs that involve automation with industrial robots.

ACKNOWLEDGMENT

This material is based upon work supported by the National Science Founda- tion under Grant No. OIA-1937019. Any opinions, findings, and conclusions or recommendations are those of the author(s) and do not reflect the views of NSF.

REFERENCES

Beck, D., 2019. Augmented and virtual reality in education: Immersive learning research. Journal of Educational Computing Research, 57(7), pp. 1619–1625. Dede, C., 2009. Immersive interfaces for engagement and learning. science, 323

(5910), pp. 66–69.

Evaluating an Immersive Learning Environment for Robotics Training 197

Jarmon, L., Traphagan, T., Mayrath, M. and Trivedi, A., 2009. Virtual world teaching, experiential learning, and assessment: An interdisciplinary communication course in Second Life. Computers & Education, 53(1), pp. 169–182. Lindgren, R. and Johnson-Glenberg, M., 2013. Emboldened by embodiment:

Six precepts for research on embodied learning and mixed reality. Educational researcher, 42(8), pp. 445–452. Mantovani, F., Castelnuovo, G., Gaggioli, A. and Riva, G., 2003. Virtual reality training for health-care professionals. CyberPsychology & Behavior, 6(4), pp. 389–395. Piaget, J., 1977. The role of action in the development of thinking. In Knowledge

and development (pp. 17–42). Springer, Boston, MA. Smith, S.P. and Burd, E.L., 2019. Response activation and inhibition after exposure to virtual reality. Array, 3, p.100010. Vygotsky, L.S. and Cole, M., 1978. Mind in society: Development of higher

psychological processes. Harvard University Press.

Cover of the proceedings volume Training, Education, and Learning Sciences

Explore the complete digital volume

Training, Education, and Learning Sciences

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-35-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →