Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Adaptive Immersive Learning Environments for Teaching Industrial Robotics

1 Florida International University, FL 33174, USA ↗
2 University of California, Irvine, CA 92697, USA ↗
3 University of Hawaii, HI 96844, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1004411

Abstract

AI, robotics, and automation are reshaping many industries, including the Architecture, Engineering, and Construction (AEC) industries. For students aiming to enter these evolving fields, comprehensive and accessible training in high-tech roles is becoming increasingly important. Traditional robotics education, while often effective, usually necessitates small class sizes and specialized equipment. On-the-job training introduces safety risks, particularly for inexperienced individuals. The integration of advanced technologies for training presents an alternative that reduces the need for extensive physical resources and minimizes safety concerns. This paper introduces the Intelligent Learning Platform for Robotics Operations (IL-PRO), an innovative project that integrates the use of Artificial Intelligence (AI), Virtual Reality (VR), and game-assisted learning for teaching robotic arms operations. The goal of this project is to address the limitations of traditional training through the implementation of personalized learning strategies supported by Adaptive Learning Systems (ALS). These systems hold the potential to transform education by customizing content to cater to various levels of understanding, preferred learning styles, past experiences, and diverse linguistic and socio-cultural backgrounds.Central to IL-PRO is the development of its ALS, which uses student progress variables and multimodal machine learning to infer students’ level of understanding and automate task and feedback delivery. The curriculum is organized into modules, starting with fundamental robotic concepts, and advancing to complex motion planning and programming. The curriculum is guided by a learner model that is continuously refined through data collection. Furthermore, the project incorporates gaming elements into its VR learning approach to create an engaging educational environment. Thus, the learning content is designed to engage students with simulated robots and input devices to solve sequences of game-based challenges. The challenge sequences are designed similarly to levels in a game, each with increasing complexity, in order to systematically incrementally build students' knowledge, skills, and confidence in robotic operations. The project is conducted by a team of interdisciplinary faculty from Florida International University (FIU), the University of California Irvine (UCI), the University of Hawaii (UH) and the University of Kansas-Missouri (UKM). The collaboration between these institutions enables the sharing of resources and expertise that are essential for the development of this comprehensive learning platform.

© 2023. Published by AHFE Open Access. All rights reserved. 116

Adaptive Immersive Learning Environments for Teaching Industrial Robotics 117

INTRODUCTION

The global economy is being rapidly reshaped by sophisticated robots that enhance human dexterity, visual perception, speed, and strength. This intense focus on creating and implementing new automation technologies is bring- ing disruptive change to job markets and the requisite training and skill sets needed for employment (Bogosian et al., 2020, Lost, 2017). In the current context it has now become essential to equip the workforce with the knowl- edge, skills and abilities required for an economy that is increasingly shaped by these technologies. This imperative extends to addressing the educational needs of Architecture, Engineering, and Construction (AEC) students, aspir- ing professionals, and industry workers, as their readiness for changes in the job market significantly influences the competitiveness of a substantial segment of the US labor force.

The project Intelligent Learning Platform for Robotics Operations (IL- PRO) integrates Artificial Intelligence (AI), including Machine Learning (ML) and Natural Language Processing (NLP), with immersive technologies including Virtual Reality (VR) to develop an automated, responsive, game- based learning platform. The project involves an interdisciplinary team of faculty from Florida International University (FIU), the University of Cal- ifornia Irvine (UCI), the University of Hawaii (UH), and the University of Kansas-Missouri (UKM).

Traditional classroom setups for teaching robotics, although effective, often demand a low student-to-teacher ratio and access to specialized equip- ment (Peterson et al., 2021). On-the-job training, while pragmatic, can potentially expose newcomers to safety hazards and risks. This AI-powered, VR-driven approach aims to minimize the dependence on extensive physical resources and mitigate safety concerns for learners.

The project builds on principles of personalized learning, enabling edu- cational content to be designed to accommodate the distinct proficiencies, past experiences, and learning preferences of individual students. The project harnesses recent strides in learning technologies to develop adaptive learning pathways that are responsive to each student’s needs and learning progress. This paper discusses the development of IL-PRO as a dynamic learning platform for industrial robotics operations. It also examines the current methodologies used in robotics teaching, reviews the state of adaptive learn- ing systems, and references established learning theories. The final sections of the paper outline the project’s milestones achieved so far.

Currently, learning industrial robotics operations and programming largely follows patterns associated with traditional classrooms or workshop settings. Before attending specialized training sessions, students or trainees often begin with textbooks or proprietary manuals. These training sessions typically encompass lectures, live demonstrations, and supervised hands-on practice with a specific training robot. Training content is carefully crafted to support students in learning, reproducing, and ultimately mastering the cor- rect concepts and techniques, avoiding spontaneous decision-making by the learner in order to minimize misconceptions, mistakes, and the development of poor habits. The response to tasks with this approach is expected to be

previously learned or known by the learner and programmed by the instruc- tor as there is just a single correct response. At the extreme, any deviation is considered to be an error that should be corrected through feedback and repetition of the correct response.

Further, to allow for personalized coaching, these classes often maintain smaller sizes, with each student working closely with a designated robot. As various robot manufacturers use different hardware and software con- figurations, there is a general lack of standardization in control interfaces, nomenclature, and programming procedures. Therefore, training and profi- ciency working with one brand of robotic arm does not easily translate to competency with another robotic arm produced by another manufacturer.

Over the past 25 years, developments in the learning sciences have fun- damentally altered our understanding of how people learn, and the ways learning content and training should be designed. At the same time, advances in computer modeling, simulation, and computer animation have also led to changes in the delivery models associated with teaching and learning (Wang et al., 2018). As institutions worldwide adapt to these changes in how learn- ers are conceived and learning content is delivered, a dynamic education landscape has resulted (Palvia et al., 2018).

Despite these changes, the state of robotics training often remains grounded in a cartesian view of the learner engaged in a traditional face-to- face training model. While there is evidence of modest integration of online channels for delivering robotics training, alternative approaches using more advanced visualization technology including VR remain underdeveloped and underutilized. In addition, recognition of variability in learners’ pathways and the use of Adaptive Learning Systems appears to be largely absent from robotics training courses.

ADAPTIVE LEARNING SYSTEMS The increasing focus on customizing learning content to fit individual needs is challenging traditional educational delivery models and introducing new perspectives on the learner. Tailored learning or responsive instruction has roots in the work of researchers such as Snow and Farr (2021) who high- lighted the importance of learning theories that consider both cognitive and emotional aspects of learners (Snow, 1989 & Snow & Jones, 2001). Work of other has continued to conceive of the learner, continuing to depart from tra- ditional, Cartesian views that emphasize cognition only. Varela, Thompson, and Rosch (1991) for instance, have focused on the body’s role in cognition, while Thelen and Smith (2007) have re-casted learners as dynamic systems that adapt to changes in their task and environment.

Building on these foundational works, subsequent educational schol- ars have incorporated alternative views of the learner into their teaching and learning experiences, and have continued to develop adaptive learn- ing models, with many focusing on real-time tracking of learner behaviors, repertoires, and emotional states for adjusting instruction (Mödritscher, Garcia-Barrios, & Gütl, 2004). In addition, the integration of data analyt- ics and Artificial Intelligence has further enhanced the development of ALS,

Adaptive Immersive Learning Environments for Teaching Industrial Robotics 119

enabling inferences about learners and automating adjustments to learning material (Educause Review, 2016).

At their core, Intelligent Adaptive Learning Systems can include a “Learner Model,” which captures the unique knowledge base and performance level of each student. There’s also an “Expert Model” containing domain-specific expertise; a “Pedagogical Model” which outlines specific teaching methods and strategies; and a “Data Analytics Engine” that analyzes student interac- tions to guide and enhance their learning trajectory. In our work on IL-PRO, we’ve prioritized the development of detailed Learner and Domain Models, which we will discuss further in the paper.

GUIDING THEORY OF LEARNING: DYNAMIC SYSTEMS THEORY A view of learners as self-organizing complex dynamic systems (Kelso, 1999; 2000) informs the IL-PRO learner model and, consequently, the design of the IL-PRO learning experiences. Learners are viewed as self-organizing in the sense that when faced with changes in tasks or their environment, they are able to explore potentially large solution spaces to discover, test, and iterate new responses. Through exploration of one or more solution spaces, learners adapt and as a result, learn new patterns of movement and ways of thinking. The point of training is to present well-considered tasks and environmental constraints that lead the learner to discover and reproduce desired solutions.

The dynamic systems approach is well-suited for game-based tasks. These game-based tasks introduce challenges that prompt learners to explore inno- vative approaches to thinking about and operating robotic systems. Oper- ationalizing this perspective involves employing design-based research to explore and understand the tasks and environments that facilitate learners’ discovery of robust solutions that can be identified with mastery of robotics. In the context of IL-PRO, successful learning activities crafted under this approach focus on the utilization of game-based tasks that foster awareness and mastery of key concepts in robotics and the facility to direct the move- ments and behaviors of a robotic arm. In many cases, the game-based tasks are designed to lead to discovery through failure in the form of error mes- sages that occur because of excessive torque, self-collision, and movements that bring the arm beyond the defined safety cage. In others, students are motivated to iterate toward alternative or increasingly efficient solutions to tasks presented in game-like formats such as navigating balls through one or more mazes.

GAME-BASED LEARNING ENVIRONMENTS The student experience and persistence are improved when learning tasks are designed to incorporate one or more aspects of what may be referred to as serious play. There is growing evidence that well-designed games motivate learners to persist in challenging tasks (Hidi and Renninger, 2006; Gee, 2007; Rotgans and Schmidt, 2011); engender high levels of cognitive, affective, sociocultural, and behavioral engagement (Plass, Homer and Kinzer, 2016); and destigmatize failure (Juul, 2013). More specifically, games in immersive

VR settings can also provide context and motivation for situated practice (Dawley and Dede, 2014) of patterns of movement (Rutkowski et al., 2021) through careful use of game mechanics and rigorous level-design (Dormans, 2010; Hullett, 2010). Design strategies for game-based learning experiences include several attributes that support personalized learning. Immersive learning environ- ments situate the learner in a sensory-rich environment, provide embodied experience, and foster a sense of presence that can contextualize the learn- ing experience in various realistic settings to support situated cognition (De Gloria et al., 2014). Games also facilitate simulated physical interaction as well as more complex and implicit cognitive engagement (Sims, 2000).

Aiming to motivate learners to persist, the IL-PRO leverages the affor- dances of games and immersive VR to develop experiences that engage learners in new ways of moving and thinking that are helpful for under- standing and coding appropriate movement patterns for the project’s virtual robotic arm.

IL-PRO COMPONENTS The main objective of the project lies in integrating immersive gaming strate- gies in VR to develop an effective learning experience while considering the unique growth of each learner. Central to addressing this challenge has been the integration of progress variables which are ultimately used as an impor- tant part of an automated assessment system inferring students’ growing levels of knowledge, skill, and ability as they complete the IL-PRO learning activities (Kennedy et al., 2005). As stated by the National Research Council (NRC, 2001), progress variables act as waymarkers that chart the learning journey of students as they progress to increasingly sophisticated ways of thinking and acting.

Most educational paradigms treat the domain model and characteriza- tions of the student’s level of knowledge, skill, and ability as two separate abstractions. However, progress variables in IL-PRO bridge this distinction by integrating the domain model with learners’ growth and development (refer to Figure 1). This integration of the domain and learner models is criti- cal for the success of ALS. In conjunction with one or more statistical models, the use of progress variables in this way permits the ALS to link observations of student performance with qualitative descriptions of what students know and can do, and in turn, informs its decisions regarding task and feedback selection.

Domain Model The IL-PRO’s domain model encompasses a range of content and strate- gies essential for successfully operating robotic arms. The formulation of the curriculum is based on the incorporation of insights from leading robotics training initiatives, KUKA e-learning, KUKA College, Universal Robots Academy, and ABB University. Further enriching this foundation has been the feedback and invaluable insights from an array of people involved in the Robotics Academy (Vassigh et al., 2021), which spanned robotic experts,

Adaptive Immersive Learning Environments for Teaching Industrial Robotics 121

pioneering industry leaders, dedicated educators, and students. The culmina- tion of this research and collaboration has resulted in a curriculum that is organized into six modules that emphasize game-based tasks. These modules advance from foundational aspects of robotic anatomy to more intricate ele- ments of motion planning, culminating in programming (refer to Figure 1). Each module’s content and associated activities and tasks are designed to sys- tematically advance students’ conceptual understanding and skill level along trajectories defined in the project’s progress variables. These progress vari- ables describe qualitative changes students undergo as they become more knowledgeable and able.

Figure 1: Diagram showing various components of the adaptive learning system.
Original PDF Figure 1:

Learner Model Student progress variables inform the project’s learner models, offering qual- itative insights into the learner’s evolving comprehension of robotic arms and their operations. Further, the IL-PRO progress variables not only inform the curriculum content and organization but also inform the design of individ- ual student tasks and feedback messages, facilitating real-time adjustments to feedback and task selection and delivery based on a student’s performance and inferred level of understanding. This approach empowers the educational team to offer tailored and automated instruction, enhancing the learning experience.

To develop the learner model, we have been collecting two types of data sets from students: First Learner Profile Data, which includes demographic data and academic background. This information forms the foundational understanding of each learner’s individual history with the targeted robotics content. Second is Performance Data which is constructed from three cate- gories: i) screening tests, and diagnostic tests before and after each learning activity resulting in scores, ii) questions during a learning activity to gauge engagement level and conceptual understanding, and iii) telemetry reflecting student actions and other process data students generate while conducting tasks including time to make a decision, time to complete a task, number of attempts to complete a task or lesson, error rate, and error type (refer to Figure 1). In addition, we plan to collect Biometric Data from VR Head Mounted Devices (HMD). This will include eye gaze data and foot trackers.

Data Collection and Analysis To develop the ALS, the project team collects and correlates various types of data from Control Groups and Experimental Groups of students. The “Control Groups” of students engage in a conventional educational model, learning industrial robotics through instructor-led teaching, without expo- sure to Virtual Reality (VR) instruction Despite not experiencing the VR element, these students are introduced to the complete six-module curricu- lum, each module being structured as a game-based learning experience (refer to Figure 2). The “Experimental Group” of students is exposed to the same curriculum but it is delivered through immersive VR environments.

Testing the project with the first Control Group of students, the team has developed a learner profile from surveys that include demographic data and information indicating each student’s personal and academic background in relation to robotics, laying the groundwork for gauging prior subject engagement. We have also collected multimodal performance data, captur- ing real-time student engagement during their interactions with the robotic arm, through audio-visual recordings and direct telemetry from the robotic arm. Complementing this is the input from instructor observation notes, which shed light on student dynamics both with the robot and the instruc- tor. Quizzes have further refined our understanding of a student’s grasp and troubleshooting abilities in robotics. An in-depth layer of insight is also added through data from the robot’s interface, painting a clear picture of student performance metrics.

Figure 2: Image showing student interactions with the robotic arm in the control group.
Original PDF Figure 2:

Apart from the ALS development, we will utilize the collected data from this Control Group for another essential purpose. This data will serve to crit- ically evaluate and iterate upon the curriculum, enhancing the effectiveness and engagement levels of the game-designed modules in conveying complex concepts. Second, this group will serve as a comparative benchmark to assess the impact of our intervention in the “Experimental Groups” of students, who are exposed to the same curriculum but will receive content through

Adaptive Immersive Learning Environments for Teaching Industrial Robotics 123

the immersive VR environment. By juxtaposing the data from these two dis- tinct groups, we aim to get valuable insights into student responses to the VR learning environment and assess whether this innovative approach fosters a measurable enhancement in student performance.

Closing Remarks The project described here has undertaken an approach to combine innova- tive pedagogical strategies with advancements in emerging technologies and conceptions of the learner. By merging game-based learning experiences with VR immersion and aligning them with the student’s developmental journey, the project promises a dynamic and interactive learning experience. Central to the project’s effectiveness is the application of progress variables, ensur- ing that the curriculum remains adaptive to changes in individual students’ knowledge and abilities. This is made possible through the data collection methods implemented since the project’s inception, encompassing both qual- itative and quantitative data. This foundation ensures that the IL-PRO system not only trains students to use robotic arms but also adapts, fostering an enriched learning environment tailored for every individual student’s growth.

ACKNOWLEDGMENT

This material is based upon work supported by the National Science Founda- tion (NSF) Award No. 2315647: Improving Undergraduate STEM Education program, and NSF Award No. 2202610: Research on Emerging Technologies for Teaching and Learning Program. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

REFERENCES

Bogosian, B., Bobadilla, L., Alonso, M., Elias, A., Perez, G., Alhaffar, H., & Vassigh, S. (2020, March). Work in progress: towards an immersive robotics training for the future of architecture, engineering, and construction work- force. In 2020 IEEE World Conference on Engineering Education (EDUNINE) (pp. 1–4). IEEE. Dawley, L., & Dede, C. (2014). Situated learning in virtual worlds and immersive

simulations. In Handbook of Research on educational communications and Technology (pp. 723–734). Springer, New York, NY. De Gloria, A., Bellotti, F., & Berta, R. (2014). Serious Games for education and training. International Journal of Serious Games, 1(1), p. 1. Dormans, J. (2010, June). Adventures in level design: generating missions and spaces

for action-adventure games. In Proceedings of the 2010 workshop on procedural content generation in games (pp. 1–8). EDUCAUSE Review. (n.d.). Adaptive Learning Systems: Surviving the storm.

Retrieved from https://er.educause.edu/articles/2016/10/adaptive-learning- systems-surviving-the-storm Gee, J. P. (2007). Good video games+ good learning: Collected essays on video games, learning, and literacy. Peter Lang. Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development.

Educational psychologist, 41(2), 111–127.

Juul, J. (2013). The art of failure: An essay on the pain of playing video games. MIT press. Kelso, J. A. S. (1999). Dynamic Patterns. USA: Massachusetts Institute of Technology. Kelso, J. A. S. (2000). Principles of dynamic pattern formation and change for a

science of human behavior. Kennedy, C. A., Brown, N. J., Draney, K., & Wilson, M. (2005). Using progress variables and embedded assessment to improve teaching and learning. American Education Research Association, San Francisco, California. Lost, J. (2017). Jobs Gained: Workforce transitions in a time of automation, p. 11. Mödritscher, F., Garcia-Barrios, V. M., & Gütl, C. (2004). The Past, the Present and

the Future of Adaptive E-Learning. Proceedings of ICL 2004. National Research Council. (2001). Classroom assessment and the national science education standards. National Academies Press. National Research Council. (2001). Knowing what students know: The science and

design of educational assessment. National Academies Press. Palvia, S., Aeron, P., Gupta, P., Mahapatra, D., Parida, R., Rosner, R., & Sindhi, S.

(2018). Online education: Worldwide status, challenges, trends, and implications. Journal of Global Information Technology Management, 21(4), 233–241. Peterson, E., Bogosian, B., Tubella, J., & Vassigh, S. (2021, June). Teaching Robotics with Virtual Reality: Developing Curriculum for the 21st Century Workforce. In International Conference on Applied Human Factors and Ergonomics (pp. 12–18). Cham: Springer International Publishing. Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015). Foundations of game-based

learning. Educational Psychologist, 50(4), 258–283. Rotgans, J. I., & Schmidt, H. G. (2011). Situational interest and academic achievement in the active-learning classroom. Learning and Instruction, 21(1), 58–67. Rutkowski, S., Adamczyk, M., Pastuła, A., Gos, E., Luque-Moreno, C., &

Rutkowska, A. (2021). Training Using a Commercial Immersive Virtual Reality System on Hand–Eye Coordination and Reaction Time in Young Musicians: A Pilot Study. International Journal of Environmental Research and Public Health, 18(3), 1297. Snow, R. E. (1989). Toward assessment of cognitive and conative structures in learning. Educational Researcher, 18, 8–14. Snow, C. E,. & Jones, J. (2001, April 25). Making a silk purse. Education Week

Commentary. Snow, R. E., & Farr, M. J. (2021). Cognitive-Conative-Affective Processes in Aptitude, Learning, and Instruction An Introduction. In Aptitude, learning, and instruction (pp. 1–10). Routledge. Sims, R. (2000). An interactive conundrum: Constructs of interactivity and learning

theory. Australasian Journal of Educational Technology, 16(1). Thelen, E., & Smith, L. B. (2006). Theories of dynamical systems. In S. Vosniadou (Ed.), Handbook of child psychology (6th ed., Vol. 1, pp. 258–312). New York: John Wiley & Sons. Varela, T., & Thompson, E. (1991). Rosch. The Embodied Mind, 22. Vassigh, S., Peterson, E., Bogosian, B., & Tubella, J. (2021). The Robotics Academy:

An Immersive Learning Game for Training Industrial Roboticists. Building Technology Educator’s Society, 2021(1), 4. Wang, P., Wu, P., Wang, J., Chi, H. L., & Wang, X. (2018). A critical review of the use of virtual reality in construction engineering education and training. International journal of environmental research and public health, 15(6), 1204.

Cover of the proceedings volume Emerging Technologies and Future of Work

Explore the complete digital volume

Emerging Technologies and Future of Work

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-93-3

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →