Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Embodied Cognition in Virtual Lunar Exploration: A Multi-modal Interactive Installation for Space Science Education

1 Department of Visual and Digital Media Design, Zhejiang University of Science and Technology, Hangzhou, China ↗
2 Computing in the Creative Industries (Modular), University of the Arts London, London, UK ↗
Publication year
Digital Object Identifier10.54941/ahfe1006023

Abstract

This paper presents "Space Exploration: Moon Gazing," an interactive installation that applies embodied cognition theory to space science education. Traditional space science exhibits typically rely on visual and textual information, creating a passive learning experience that fails to internalize abstract astronomical concepts. Our installation addresses this limitation through a "body-environment-tool" framework that transforms space knowledge into direct bodily experience. The system integrates a six-screen collaborative interface with gesture-based interaction, VR immersion, and multi-sensory feedback mechanisms. Users experience simulated lunar gravity constraints, manipulate virtual objects through natural gestures, and engage with content through a three-layer narrative structure that connects traditional Chinese lunar imagery with modern space technology. This research contributes to human-centered design in digital environments by demonstrating how embodied interaction can bridge the gap between abstract scientific concepts and intuitive understanding, providing a new paradigm for immersive educational experiences in public science venues.

INTRODUCTION

In recent years, Virtual Reality (VR) technology, with its unique immersiveness and interactivity, has demonstrated significant application potential across various fields including education, entertainment, and healthcare (Lei, Su, & Cheng, 2023). Particularly in science education, VR can make abstract scientific concepts and remote exploratory scenes accessible, thereby significantly enhancing learners’ engagement and learning outcomes (Wang et al., 2016). Concurrently, the rise of Embodied Cognition theory offers a new perspective for understanding human cognitive processes. This theory posits that cognition is not merely an abstract computation of the brain but is deeply rooted in the dynamic interaction between the body and the environment (Kirsh, 2013). Applying embodied cognition theory to VR

© 2025. Published by AHFE Open Access. All rights reserved. 158

Embodied Cognition in Virtual Lunar Exploration: A Multi-Modal Interactive Installation 159

experience design holds the promise of creating more natural, intuitive, and effective learning tools.

Space exploration, as a significant theme for expanding human cognitive boundaries and stimulating scientific interest, has always been a focal point and a challenge in science popularization. Traditional methods of science communication (e.g., books, videos, model exhibitions) have limitations in conveying the breadth and depth of space knowledge and fail to meet the public’s growing demand for experiential learning. This research aims to explore how to integrate embodied cognition theory with VR technology to design a multi-modal interactive installation themed on virtual lunar exploration—“Wangyu Zhuyue.” This installation seeks to provide a novel solution for space science education by simulating a realistic lunar environment, integrating multi-sensory feedback, and designing meaningful interactive tasks, guiding users to explore the mysteries of the moon through their physical perception and actions, thereby deepening their understanding and interest in space science.

This paper first reviews the relevant research on embodied cognition, VR experience design, and the application of space-exploration themes. Secondly, it details the design philosophy, system architecture, and key technological implementations of the “Wangyu Zhuyue” interactive installation. Figure 4 below illustrates the overall research logic and flow of this paper. Finally, it discusses the potential of this installation in space science education and looks forward to future research directions.

LITERATURE REVIEW

Embodied Cognition and Design Embodied cognition theory, which asserts that cognitive processes are deeply intertwined with the body’s physical structure, sensory experiences, and environmental interactions (Varela, Thompson, & Rosch, 1991; Shapiro, 2011), offers a transformative lens for design. In design disciplines, this perspective encourages a shift from purely functional or aesthetic considerations to a more holistic understanding of user experience, where the body’s engagement is paramount (Norman, 2013). The development of Natural User Interfaces (NUI), emphasizing interactions via gestures, voice, and other natural human actions, is a direct application of these principles (Wigdor & Wixon, 2011). Research consistently demonstrates that embodied interactions within technological environments can significantly enhance users’ sense of immersion, improve task performance, and bolster learning effectiveness (Johnson-Glenberg, Birchfield, Tolentino, & Koziupa, 2014). Dourish (2001) in his seminal work on “embodied interaction,” highlighted the active role of the physical body in shaping how individuals interact with and comprehend computational systems, moving beyond the traditional view of the user as a disembodied mind. Recent meta-analyses on technology- based embodied learning (TBEL) confirm a statistically significant positive effect on learning outcomes, although the degree of impact can be moderated by factors such as educational level and the specific type of embodiment implemented (Zhang, Meng, & Bond, 2023). Furthermore, the Cognitive

160 Cai and Jiang

Affective Model of Immersive Learning (CAMIL) suggests that higher levels of immersion in environments like VR can lead to more profound embodied learning experiences, which in turn are predictive of better procedural learning outcomes and knowledge transfer (Makransky & Petersen, 2021). However, the relationship between the level of bodily engagement and learning outcomes can be complex, with some studies indicating that highly complex bodily movements might not always yield superior results if not carefully integrated with cognitive tasks (Georgiou & Ioannou, 2019).

Virtual Reality Immersive Experience Design The efficacy of VR technology is often attributed to its “3I” characteristics: Immersion, Interaction, and Imagination (Burdea & Coiffet, 2003). Immersion, the sensation of being enveloped within a virtual environment, is fundamental to the VR experience. It is shaped by a confluence of factors, including the fidelity of visual displays, the authenticity of auditory feedback, the intuitiveness of interaction mechanisms, and the user’s subjective sense of “presence” or “being there” (Slater, 2009; Cummings & Bailenson, 2016). Designing effective VR experiences necessitates a comprehensive approach that considers technological capabilities, narrative coherence, and user psychology (Sanchez-Vives & Slater, 2005). Recent advancements have focused on integrating multi-sensory feedback, including haptic and even olfactory cues, to further amplify the realism and immersiveness of VR environments (Shin, 2018; Hoffman, Sharar, & Coda, 2013). Such multi-sensory approaches are believed to create stronger and more complex neural connections, leading to better retention and recall, and catering to diverse learning styles (Radianti et al., 2020). Immersive VR offers distinct affordances for science learning, such as the ability to manipulate virtual objects, experience changes in scale (e.g., exploring atomic structures or vast cosmic distances), sequence events, build models, and view phenomena from multiple perspectives (Alhalabi, 2016).

EVR Applications in Space Exploration The inherent mystery, grandeur, and challenge of space exploration make it an exceptionally compelling theme for VR applications. Leading space agencies, including NASA, have been at the forefront of utilizing VR for sophisticated astronaut training programs, complex mission simulations, and intuitive scientific data visualization (Noor, 2016; Kalawsky, 1999). For instance, applications like “Mission: ISS” provide users with a visceral experience of the zero-gravity conditions aboard the International Space Station and the intricacies of performing extravehicular activities (spacewalks). In the domain of public science education and popularization, VR-based space exploration experiences offer unprecedented opportunities for individuals to engage directly with otherworldly environments, such as traversing the Martian landscape in the “Access Mars”project or navigating through distant star systems (Kenche et al., 2019; Buck, 2017). These applications serve not only to ignite public curiosity about the cosmos but also to effectively communicate complex concepts in astronomy, astrophysics, and spaceflight

Embodied Cognition in Virtual Lunar Exploration: A Multi-Modal Interactive Installation 161

engineering (Yildirim, 2020). Multi-sensory VR experiences, such as those incorporating motion floors and environmental effects (e.g., wind, heat) for simulated lunar missions, further enhance engagement and the sense of presence (Paladini et al., 2021). While many current applications excel at delivering stunning visual spectacles, there remains a significant opportunity to more systematically integrate principles of embodied cognition. The goal is to move beyond passive observation towards designs that leverage multi-modal interaction and physical engagement to foster deeper cognitive understanding and more meaningful learning outcomes in space science education.

“WANGYU ZHUYUE” MULTI-MODAL INTERACTIVE INSTALLATION DESIGN The “Wangyu Zhuyue” interactive installation designed in this study aims to provide users with a deeply immersive, scientifically accurate, and educationally meaningful virtual lunar exploration experience, guided by embodied cognition theory through multi-modal interaction.

Design Philosophy and Objectives The core philosophy of “Wangyu Zhuyue” is to use the “bright moon” as a central image, connecting traditional Chinese cultural reflections on the moon with the technological vision of future space exploration, thereby constructing an embodied cognitive experience through the dynamic coupling of “body-environment-tool.” The design aims to achieve several key objectives. A primary goal is to create an Immersive Experience by rendering a highly realistic lunar environment, making users feel genuinely present. Multi-sensory Interaction is another crucial objective, achieved by integrating visual, auditory, and conceptual tactile channels to build a comprehensive exploratory experience. The installation also serves an important function in Science Education, transforming abstract space science knowledge— such as lunar topography, the low-gravity environment, and Chinese space achievements—into tangible, experiential content. Furthermore, it seeks to foster Emotional Resonance, stimulating users’ curiosity and desire for space exploration and thereby cultivating scientific interest. Lastly, Cultural Inheritance is embedded in the design, incorporating elements like the Chinese “Wangyu” lunar suit to showcase cultural confidence and technological innovation.

System Architecture The system architecture of the “Wangyu Zhuyue” interactive installation is composed of distinct hardware and software systems. The Hardware System is built around a high-performance Main Control Computer that runs the UE5 virtual environment, manages TouchDesigner interaction logic, and drives the multi-screen display. For VR immersion, a Pico 4 VR headset and dual hand controllers are utilized, offering first-person immersive visuals and 6DoF tracking. The Multi-screen Display System includes Large Screen 1 (Main Immersion Screen) which mirrors the VR user’s first-person

162 Cai and Jiang

perspective in real-time; Large Screen 2 (Interactive Map Screen) which displays a global lunar map and user location, also supporting bystander interaction via Leap Motion or a touch panel; and four Small Screens (Auxiliary Information Screens) that present supplementary content such as lunar structure, solar system dynamics, space science facts, and astronomical observation data, all synchronized by TouchDesigner. Interactive input is facilitated by a Leap Motion Controller for gesture-based interaction by non-VR users with Large Screen 2, and a conceptual Microphone Array (AtomS3R + Atomic Echo Base) for voice control. A conceptual Wearable Lunar Suit Device is also envisioned to enhance the embodied experience by simulating the physical constraints and sensory feedback of a spacesuit through modular components.

Software System and Technical Implementation The Virtual Environment Construction is a critical component, realized through the combined capabilities of UE5 and Blender. High-precision 3D models, including the lunar surface (derived from actual terrain data), Earth, other celestial bodies, space station modules, and the “Wangyu” lunar suit virtual avatar, are meticulously created using Blender. These assets are then imported into UE5 for comprehensive scene assembly, material application, and advanced lighting rendering. UE5’s Lumen technology is employed for dynamic global illumination, and Nanite is used to handle complex geometric detail, ensuring an accurate and visually rich simulation of lunar surface light and shadow interplay, as well as its characteristic low-gravity physics.

The Interaction Logic and Multi-screen Control are orchestrated by TouchDesigner, which functions as the central nervous system for interaction. It is tasked with receiving and processing a variety of input signals originating from the VR controllers, the Leap Motion sensor, and the voice control modules. Based on a predefined logical framework, TouchDesigner dynamically updates and synchronizes the content displayed across the multiple screens (Large Screens 1 & 2, and Small Screens 1–4). It also governs the playback of associated multimedia elements, such as ambient lighting changes and sound effects, to create a cohesive experience.

Virtual Avatar and Embodiment Mechanisms are central to the user’s immersion and sense of agency. Within the VR environment, users are represented as an astronaut clad in the “Wangyu” lunar suit. The movements of this virtual avatar are synchronized in real-time with the user’s actual head and hand movements, captured by the VR system. To simulate the lunar environment authentically, physical parameters within the virtual world are adjusted; for example, gravitational acceleration is set to approximately 1/6th of Earth’s. This allows users to directly experience the effects of low gravity through their physical actions, such as modified walking gaits and exaggerated jump heights.

A comprehensive Multi-sensory Feedback System is designed to deepen immersion by engaging multiple senses. Visually, users are presented with high-resolution VR displays showcasing a realistic lunar landscape, the vastness of deep space, and breathtaking views of Earth as seen from the

Embodied Cognition in Virtual Lunar Exploration: A Multi-Modal Interactive Installation 163

moon. Auditory feedback is delivered through a sophisticated spatial audio design, which includes the astronaut’s breathing sounds within the helmet, radio communications, the operational sounds of equipment, and the subtle, simulated bone-conducted sounds of footsteps on the lunar surface in a vacuum. While full tactile immersion is complex, conceptual tactile feedback is envisioned through vibrations from the VR controllers, which can simulate the sensation of operating tools, and potential pressure and temperature feedback from the conceptual wearable lunar suit.

The Spatial Narrative and Experience Path is carefully structured into distinct but interconnected layers to guide the user’s journey. The Outer Layer Experience prioritizes Environmental Realism and Immersion, achieved through meticulous modeling, high-fidelity rendering, accurate lighting, and appropriate color palettes to recreate the stark beauty of the lunar environment. The Middle Layer Experience introduces Cultural Context and Popular Science Elements, opportunistically presenting information about Chinese lunar exploration achievements (like the “Chang’e”program landing sites), fundamental lunar science knowledge (such as the formation of maria and craters), and aspects of ancient Chinese astronomical culture as the user interacts with the environment. Finally, the Inner Layer Experience facilitates Dynamic Embodied Interaction of Body-Environment-Tool, where users, embodied in their virtual avatars, actively engage with the lunar environment using virtual tools (e.g., geological hammers, sample collectors) to complete exploration tasks. These interactive behaviors are designed to directly influence and shape the user’s cognitive construction of the experience and the knowledge gained.

Interaction Design Details The interaction design details are meticulously crafted to ensure a natural, intuitive, and engaging user experience. For VR First-Person Exploration, users equipped with the Pico 4 headset and controllers navigate the virtual lunar surface from a first-person perspective. They can freely walk, execute low-gravity jumps, conceptually operate a lunar rover, and interact with various environmental objects and mission-related items. Gesture Interaction, primarily facilitated by the Leap Motion controller, allows non-VR users or observers to interact with content displayed on Large Screen 2. For instance, they can use hand gestures to rotate and scale a 3D model of the moon, or to select and view detailed information about specific lunar regions or features. Furthermore, Voice Interaction capabilities enable users to issue voice commands for specific actions, such as inquiring about particular space- related knowledge (e.g., “Tell me about lunar maria”) or controlling certain scene transitions or information displays within the installation. To provide structure and purpose to the exploration, Task Guidance is implemented. This involves a series of predefined exploration tasks, such as locating specific geological formations, collecting virtual samples, or deploying simulated scientific instruments. Completing these tasks is designed to encourage active exploration and provides users with a sense of accomplishment, along with contextual knowledge feedback related to their actions and discoveries.

164 Cai and Jiang

Figure 1: Conceptual design of the multi-modal interactive lunar exploration system (source: author’s design).
Original PDF Figure 1:

POTENTIAL OF EMBODIED COGNITION IN SPACE SCIENCE EDUCATION The design and implementation of the “Wangyu Zhuyue” interactive installation preliminarily demonstrate the significant potential of applying embodied cognition theory within space science education. This approach promises to Enhance Learning Interest and Engagement by transforming traditionally abstract or passive scientific information into an engaging, hands-on exploration. Through immersive multi-sensory experiences and active physical participation, learners are shifted from being mere recipients of information to active constructors of meaning. It also serves to Promote Conceptual Understanding and Knowledge Internalization. Abstract scientific concepts, such as the principles of low gravity, the vacuum of space, or the vastness of celestial scales, become more concrete and comprehensible when users can directly perceive and interact with their simulated effects. For example, by controlling an avatar to walk and jump on the virtual lunar surface, users gain a more profound, kinesthetic understanding of how low gravity affects movement, thereby facilitating deeper comprehension and more durable memory retention of these concepts.

Moreover, the installation is designed to aid in the Cultivation of Scientific Thinking and Exploratory Spirit. The interactive tasks encourage users to adopt a scientific mindset, prompting them to observe, analyze, form hypotheses, and verify their understanding through action within the virtual environment. This process of engaging in virtual exploration tasks helps to exercise and develop their scientific inquiry skills and problem-solving abilities. The Reinforcement of Contextualized Learning Effects is another key benefit, as scientific knowledge points are not presented in isolation but are integrated into the specific context of space exploration missions and environments. This “learning by doing” approach aligns more closely with natural human cognitive processes and can significantly improve the

Embodied Cognition in Virtual Lunar Exploration: A Multi-Modal Interactive Installation 165

transfer and application of learned knowledge to new situations. Finally, “Wangyu Zhuyue” contributes to the Expansion of New Paradigms for Science Popularization. It explores an innovative model that synergistically integrates technology, art, culture, and education. By disseminating not only scientific facts but also the spirit of scientific endeavor and a sense of humanistic connection to the cosmos, it provides fresh and engaging exhibition ideas for science museums, planetariums, and other institutions dedicated to public science education.

Table 1: Configuration of the “Wangyu Zhuyue” interactive installation.

Table available in the original PDF paper.

Component Content Type Interaction Method Primary Technical Solution VRHeadset (Pico 4) First-person Head & hand motion UE5, Pico SDK

tracking astronaut lunar surface exploration

Large Screen 1

Synchronized VR

No direct interaction

TouchDesigner, Leap

(Main)

first-person view (viewing) TouchDesigner

TouchDesigner

Motion SDK Small Screen 1 Lunar internal

structure cross-section

synchronized display TouchDesigner TouchDesigner

Small Screen 2 Solar system planetary motion simulation

synchronized display

Small Screen 3 Chinese space

TouchDesigner

TouchDesigner synchronize

program history/science Small Screen 4 Real-time TouchDesigner

TouchDesigner

astronomical data (simulated)

synchronized display Natural Language Voice Module Voice commands/- Open-source LLM Processing

knowledge Q&A Body movement Arduino/Sensors

API, M5Stack AtomS3R Wearable Device

Simulates lunar suit (Concept)

(conceptual)

constraints & sensory feedback

DISCUSSION

The “Wangyu Zhuyue” interactive installation applies embodied cognition theory to space science education, moving beyond visual-dominant VR to create holistic cognition through multi-sensory design, physical interaction, and cultural narrative elements. Limitations include technical constraints preventing full implementation of certain sensory feedback elements and the ongoing challenge of balancing scientific accuracy with user experience. Future work should focus on optimizing the installation through user studies, exploring advanced sensing technologies, and expanding applications to other scientific fields.

CONCLUSION

This paper presents “Wangyu Zhuyue,” a multi-modal interactive lunar exploration installation based on embodied cognition theory. By integrating

166 Cai and Jiang

VR, multi-sensory feedback, and interactive tasks, it transforms abstract space knowledge into tangible experiences. The project demonstrates embodied cognition’s value for VR design in science education, showing how immersive, interactive environments engage multiple senses to enhance learning outcomes. As technologies advance, such installations will likely play increasing roles in science education and cultural heritage preservation.

ACKNOWLEDGMENT

The authors would like to express their sincere gratitude to all individuals who provided support throughout this research journey. We are deeply grateful to the experts in embodied cognition and virtual reality who generously shared their insights and expertise. Special thanks to our research assistants for their dedicated help with technical implementation and testing of the “Wangyu Zhuyue” installation. We also acknowledge the inspiration drawn from China’s lunar exploration program and traditional Chinese astronomical culture.

REFERENCES

Alhalabi, W. (2016). Virtual reality systems enhance students’ achievements in engineering education. Behaviour & Information Technology, 35(11), 919–925. Buck, L. (2017). Virtual Reality: An Emerging Tool for Informal Science Education.

White Paper, NASA Jet Propulsion Laboratory. Burdea, G. C. and Coiffet, P. (2003). Virtual Reality Technology. 2nd ed. Hoboken, NJ: Wiley-IEEE Press. Cummings, J. J. and Bailenson, J. N. (2016). How Immersive Is Enough? A

Meta-Analysis of the Effect of Immersive Technology on User Presence. Media Psychology, 19(2), 272–309. Dourish, P. (2001). Where the Action is: The Foundations of Embodied Interaction.

Cambridge, MA: MIT Press. Georgiou, Y. and Ioannou, A. (2019). Embodied learning in a digital world: A systematic review of empirical research in K-12 education. In P. Díaz, A. Ioannou, K. K. Bhagat, & J. M. Spector (Eds.), Learning in a Digital World: Perspective on Interactive Technologies for Formal and Informal Education (pp. 155–177). Singapore: Springer. Hoffman, H. G., Sharar, S. R. and Coda, B. (2013). Virtual reality for acute pain

management. In F. Steinicke, H. Visell, J. Campos, & A. Lécuyer (Eds.), Human Walking in Virtual Environments (pp. 537–552). New York: Springer. Johnson-Glenberg, M. C., Birchfield, D. A., Tolentino, L. and Koziupa, T.

(2014). Collaborative Embodied Learning in Mixed Reality Motion-Capture Environments: Two Science Studies. Journal of Educational Psychology, 106(1), 86–104. Kalawsky, R. S. (1999). VRNET: a virtual reality network for co-operative working, training and experience. BT Technology Journal, 17(4), 107–118. Kenche, N., Moghe, A., Auti, T., Mohanraj, I. and Pandhare, S. (2019).

Space Exploration and Education using Virtual Reality. International Journal of Scientific Research in Computer Science Engineering and Information Technology, 5(2), 121–126. Kirsh, D. (2013). Embodied Cognition and the Magical Future of Interaction Design.

ACM Transactions on Computer-Human Interaction, 20(1), 1–30.

Embodied Cognition in Virtual Lunar Exploration: A Multi-Modal Interactive Installation 167

Lei, Y., Su, Z. and Cheng, C. (2023). Virtual reality in human-robot interaction: Challenges and benefits. Electronic Research Archive, 31(5), 35. Makransky, G. and Petersen, G. B. (2021). The Cognitive Affective Model of

Immersive Learning (CAMIL): A Theoretical Research-Based Model of Learning in Immersive Virtual Reality. Educational Psychology Review, 33, 937–958. Noor, A. K. (2016). The Hololens Revolution. Mechanical Engineering, 138(10), 30–35. Norman, D. A. (2013). The Design of Everyday Things: Revised and Expanded

Edition. New York: Basic Books. Paladini, R. E., Diana, B., Nyffeler, T., Mosimann, U. P., Nef, T., Urwyler, P.

and Müri, R. M. (2021). The angle of perspective in virtual reality exploration modulates space perception and cognitive performance. Computers in Human Behavior, 118, 106678. Radianti, J., Majchrzak, T. A., Fromm, J. and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 147, 103778. Sanchez-Vives, M. V. and Slater, M. (2005). From presence to consciousness through

virtual reality. Nature Reviews Neuroscience, 6(4), 332–339. Shapiro, L. (2011). Embodied Cognition. New York: Routledge. Shin, D. (2018). Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience? Computers in Human Behavior, 78, 64–73. Slater, M. (2009). Place illusion and plausibility can lead to realistic behaviour in

immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3549–3557. Varela, F. J., Thompson, E. and Rosch, E. (1991). The Embodied Mind: Cognitive Science and Human Experience. Cambridge, MA: MIT Press. Wang, C., Tang, Y., Kassem, M. A., Mu, T., Meng, X., Lu, Q. and Lu, W.

(2016). Application of VR technology in civil engineering education. Computer Applications in Engineering Education, 24(5), 726–736. Wigdor, D. and Wixon, D. (2011). Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Burlington, MA: Morgan Kaufmann. Yildirim, B. (2020). A Review of Augmented and Virtual Reality Applications in

STEM Education. Journal of Turkish Science Education, 17(3), 493–514. Zhang, M., Meng, F. and Bond, M. (2023). Technology-based embodied learning on students’ learning outcomes: A meta-analysis. British Journal of Educational Technology, 54(1), 217–242.

Cover of the proceedings volume Accessibility, Assistive Technology and Digital Environments

Explore the complete digital volume

Accessibility, Assistive Technology and Digital Environments

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-38-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →