Yaoying Peng (2026). Multimedia Design from the Perspective of Embodied Interactive Narratives. In: Human Factors in Communication of Design. AHFE Open Access, vol. 216, pp. 195–203. https://doi.org/10.54941/ahfe1007637
The integration of narrative design and multimedia design has become a major trajectory in recent developments within the field of digital art and design, marking a shift from technology-driven production toward experience-oriented creation, and from visual centrality toward contextual and situational participation. This integration not only extends the typological evolution of exhibition-based works, but also serves as a critical path through which contemporary creators articulate concepts, generate affective resonance, and construct mechanisms for audience engagement. Especially under a cultural condition that increasingly values interactivity and immersion, the dynamic coupling between narrative and multimedia design has emerged as a core issue across disciplines such as digital art, interactive design, and exhibition curating.Since the 2020s, the rise of AI-generated content (AIGC) has exerted a profound influence on content production processes, modes of expression, and even the authorship paradigm. On one hand, AIGC has significantly enhanced the efficiency of content generation, leading to an unprecedented explosion in the quantity and diversity of virtual media modalities. On the other, audiences now expect more sophisticated narrative engagement within immersive cultural environments. Faced with the dual pressures of content oversaturation and experiential fatigue, multimedia design practices reliant on a single-mode virtual narrative have begun to reveal their limitations in both effectiveness and participatory appeal, falling short of audience expectations for embodied, site-specific, and resonant forms of expression.Against this backdrop, Chapter 1 of this study begins with the concept of embodied interaction, revisiting two decades of theoretical development around key topics such as interaction design, human–technology relations, and material mediation. This forms the basis for an archaeology of knowledge focused on interactive narrative mechanisms. Through a review of landmark literature and representative works, the study defines embodied interactive narrative as a design strategy in which narrative meaning emerges through the affordances of embodied interaction, or through real-world objects assuming partial narrative functions. In such works, the narrative no longer resides solely in text or image, but in the systemic coupling of body–space–object relations, generating what we term a participatory narrative grammar of situation.Chapter 2 presents a multi-layered analysis of three recent, representative embodied interactive works from China, selected for their narrative construction and interactional mechanics. These works not only integrate site-specific spatial contexts and sociocultural settings, but also draw upon traditional Chinese temporal frameworks, including seasonal rhythms and the solar terms (节气). This enables the construction of hybrid narrative configurations across spatial, temporal, and behavioral layers. In these scenarios, audiences are no longer passive receivers of meaning; instead, they become active participants and co-constructors of narrative through embodied movement, tactile feedback, and real-time decision-making. This shift from author-centered to audience-co-constructed narration enhances both the clarity of conceptual expression and the depth of emotional transmission.Building upon these insights, Chapter 3 proposes a triadic narrative framework—Space–Time–Action—as a core theoretical model for embodied interactive design. Within this model, the temporal dimension is articulated as non-linear and collage-based, allowing audiences to reconstruct narrative perception through disrupted sequence and fragmented plots. The spatial dimension takes the form of juxtaposed and experiential layouts, enabling multi-directional narrative unfolding within physical environments. The actional dimension maps audience behaviors onto narrative progression: through interactive gestures and responses, latent narrative nodes are activated, and behavior itself becomes a structural component of storytelling—disrupting the traditional unidirectional logic of “work–spectator” relations.In conclusion, this study aims to offer a referential theoretical and methodological framework for contemporary multimedia creators exploring embodied interactive narratives. It responds to the structural transformations of content creation under the influence of AIGC, while also opening up new narrative pathways grounded in locality, materiality, and interactivity. Such approaches not only deepen audience engagement and on-site immersion, but also signal a potential future direction for the development of narrative-centered multimedia works.
Over the past decade, digital art has undergone a significant creative transformation—from image- and interface-centered “visual design” toward an experience-centered paradigm grounded in context, participation, and interaction. Within this shift, the convergence of narrative design and multimedia design has become increasingly prominent, signaling a deeper transition: from technology-driven production to experience-oriented creation, from static presentation to dynamic audience engagement. This change is not only reflected in the evolution of artistic forms such as interactive installations, immersive exhibitions, and digital performances, but also in the transformation of the designer’s conceptual logic—from the transmission of content to the construction of participatory environments. Particularly within exhibitions, urban interfaces, and online platforms, narrative mechanisms have emerged as key structuring devices for activating audience emotions, behaviors, and cognitive involvement.
At the same time, new content generation technologies—exemplified by AIGC (Artificial Intelligence Generated Content)—are restructuring both creative workflows and audience experience frameworks. On one hand, such technologies have vastly increased the speed and scale of content production, enabling the explosive proliferation of visual, sonic, and textual media within virtual environments. On the other hand, audiences now expect greater levels of immersion, interactivity, and embodied presence in the narratives they Received February 6, 2026; Revised April 2, 2026; Accepted April 21, 2026; Available online July 20, 2026
196 Peng engage with. In this context of content saturation and experiential fatigue, multimedia works that rely on linear narrative paths increasingly reveal their limitations: they struggle to deliver meaning effectively and fail to sustain audience participation. As a result, contemporary design practice must explore alternative narrative pathways—ones that emphasize embodiment, materiality, and co-construction.
It is against this multifaceted backdrop that this paper introduces the concept of tangible narrative, a framework intended to reveal how contemporary digital art embeds storytelling functions within the relations among body, space, and object. By going beyond the traditional visual– textual paradigm, tangible narrative enables new forms of interactive design in which meaning is constructed through situated action and spatial engagement. Drawing upon both theoretical excavation and case-based analysis, this study proposes a triadic narrative model—comprising time, space, and behavior—as a theoretical tool for understanding and designing future multimedia works. In doing so, it seeks to address structural transformations in digital creation under the influence of algorithmic technologies and provide a concrete methodological framework for future design practices.
THE EMERGENCE OF TANGIBLE NARRATIVE Over the past decade, digital art has undergone a profound creative shift, moving from image- and interface-centered “visual design” toward an experience-centered paradigm grounded in context, participation, and interaction. Within this transformation, the convergence of narrative design and multimedia design has become increasingly visible, signaling a deeper turn: from technology-driven production to experience-oriented creation, and from static presentation to dynamic audience engagement. This shift is evident not only in the evolution of artistic forms such as interactive installations, immersive exhibitions, and digital performances, but also in the changing conceptual logic of design itself—from the delivery of content to the construction of participatory environments. In exhibitions, urban interfaces, and online platforms in particular, narrative mechanisms have become crucial structuring devices for activating audience emotion, behavior, and cognitive involvement.
At the same time, emerging content-generation technologies, especially AIGC (Artificial Intelligence Generated Content), are reshaping both creative workflows and the frameworks through which audiences experience media. On the one hand, these technologies have dramatically accelerated and expanded content production, enabling the rapid proliferation of visual, sonic, and textual media across virtual environments. On the other hand, audiences increasingly expect narratives to offer deeper immersion, stronger interactivity, and a greater sense of embodied presence. Under conditions of content saturation and experiential fatigue, however, multimedia works that rely on linear narrative structures are showing clear limitations: they often struggle to communicate meaning effectively and fail to sustain
Multimedia Design Research From the Perspective 197 audience participation over time. Contemporary design practice, therefore, must explore alternative narrative strategies that foreground embodiment, materiality, and co-creation.
Against this backdrop, this paper introduces the concept of tangible narrative as a framework for understanding how contemporary digital art embeds storytelling functions within the relationships among body, space, and object. By moving beyond the traditional visual-textual paradigm, tangible narrative opens up new possibilities for interactive design, in which meaning emerges through situated action and spatial engagement. Through a combination of theoretical inquiry and case analysis, this study proposes a triadic narrative model consisting of time, space, and behavior as a conceptual tool for interpreting and designing future multimedia works. In doing so, it seeks both to respond to the structural transformation of digital creation under algorithmic conditions and to offer a concrete methodological framework for future design practice.
THREE MODES OF TANGIBLE NARRATIVE IN PRACTICE Within the evolving cultural landscape in which digital art increasingly intersects with multimedia installation design, tangible interaction has moved beyond its original role as a technical triggering mechanism and developed into a composite system that intertwines audience behavior, spatial context, and narrative structure. It mediates not only users’ perception of physical environments but also the production of information, the transmission of meaning, and the activation of emotional resonance. The three case studies examined in this paper—City Memory: Beijing, A Curious World, and Luxor in Reflection—differ in production context, technological framework, and modes of interaction, yet all demonstrate the mediating role of tangible interaction in digital storytelling. More importantly, each case points to a distinct narrative typology: spatial memory reconstruction, behavior-driven generation, and historical-cultural dialogue.
The first case, City Memory: Beijing, exemplifies a narrative approach centered on the reconstruction of spatial memory. Presented in the central atrium of the Zhaolong Hotel, the work superimposes three historical moments—“Old Beijing Memory,” “Era of Transition,” and “New Community”—within a single physical setting through the use of 3D scanning and digital human modeling. Rather than encountering the story through a linear script, visitors engage with it by entering, moving through, and lingering within overlapping spatial zones, thereby reconstructing memory through embodied experience. By embedding these narrative layers into the existing architectural structure, the work transforms historical space into a semantic carrier: physical architecture becomes an active narrative agent, while the audience assumes the role of co-creator of meaning through spatial engagement. This narrative logic foregrounds temporal layering and spatial juxtaposition, sustained by the emotional resonance between space and personal memory.
Figure 1: Visual presentation of the work city memory: Beijing.
In contrast, A Curious World adopts a behavior-driven narrative model. Developed around the social building game Rui·Xun, this work offers an open-ended storytelling framework where participants generate their own future cities through the act of “selecting–combining–placing” modular components. These actions are not merely procedural but serve as the primary narrative units, establishing a strong correlation between behavior and storytelling. Here, the artist is no longer a singular author but a facilitator of creative agency, transforming users into both creators and experiencers. Through embedded social features such as “alliances” and “collaborations,” the narrative expands from individual creation to networked collective authorship, forming a multi-vocal storytelling ecology. By disrupting the fixed triangle of “author–work–audience,” and embracing nonlinear browsing, shuffled sequencing, and user-driven decision-making, the project constructs a polysemantic narrative universe.
Figure 2: Visual presentation of the work a curious world.Figure 3: Visual presentation of the social building game Rui·Xun.
Multimedia Design Research From the Perspective 199 The third case, Luxor in Reflection, presents a historical-cultural dialogic form of tangible narrative. Using the 3,000-year-old Luxor Temple in Egypt as a projection surface, the piece overlays dynamic visual elements—such as galloping horses and rotating pillars—onto the temple’s stone architecture via 3D mapping. Unlike the previous two cases that emphasize constructive manipulation, this work focuses on activation. The light-based interaction does not physically alter the site but instead animates its symbolic and historical presence, creating a multi-sensory dialogue between ancient space and contemporary imagery. The resulting narrative structure embodies a temporal and cultural overlay, reframing the sacredness of the site while integrating new symbolic orders through contemporary media. It achieves a non-invasive spatial translation, combining ritual continuity with technological reinterpretation.
Figure 4: Visual presentation of the work luxor in reflection.Figure 5: Visual presentation of the work luxor in reflection.
Taken together, the three case studies demonstrate three key functions of tangible interaction within narrative multimedia design:
Spatial Capacity – in City Memory: Beijing, where historical space becomes a platform for temporal collage;
Behavioral Generation – in A Curious World, where “to build is to narrate”;
Cultural Dialogicity – in Luxor in Reflection, where asymmetric interplay between past and present becomes a narrative engine.
In conclusion, these three narrative paths illustrate the restructuring of relationships among site, technology, and audience in contemporary digital art. Though divergent in method, structure, and engagement, they share an emphasis on embodied participation, contextualized spatiality, and
200 Peng technologically mediated storytelling. Tangible narrative thus emerges not only as a design technique but as a cultural methodology for reorganizing expression, temporality, and interaction across physical and digital domains. In the era of AIGC, it offers a promising strategy to resist content homogenization, decentralize authorship, and reactivate meaning production through immersive, co-created narrative systems.
CONSTRUCTING THE TRIADIC NARRATIVE MODEL: TIME – SPACE – BEHAVIOR Drawing on the analysis of the three tangible narrative works—City Memory: Beijing, A Curious World, and Luxor in Reflection—this chapter develops a triadic narrative model organized around three core dimensions: time, space, and behavior. The model not only clarifies the underlying mechanisms through which tangible installations generate information and activate perception, but also marks a critical shift in digital storytelling—from the logic of passive viewing to that of active co-construction.
In the temporal dimension, all three works employ non-linear narrative structures. City Memory: Beijing unfolds through the sequence of “Memory – Transition – Community,” allowing viewers to move across three temporal layers within a single space and thereby experience urban transformation as a historical collage. A Curious World disrupts temporal continuity even more radically, enabling users to move across different user-generated spaces, choose their own narrative routes, and construct individualized temporal experiences. Luxor in Reflection, by contrast, overlays ancient architecture with contemporary imagery to produce a hybrid timespace in which ancestral totems and modern experience coexist. Rather than following conventional plot progression, these non-linear temporal frameworks present narrative as rhythm—something to be perceived, reconfigured, and navigated.
On the spatial level, all three projects use juxtaposition and immersion to organize relationships between physical and virtual layers. City Memory: Beijing embeds multiple urban temporalities within the same architectural setting, transforming physical space into a vessel of overlapping cultural memory. A Curious World creates a symbiotic relation among virtual, networked, and real spaces, allowing creators to build in digital environments, audiences to engage through physical installations, and participants to connect across platforms. Luxor in Reflection, through 3D mapping technology, animates the surfaces of Luxor Temple with projected imagery, turning columns and walls into narrative interfaces. In each case, storytelling moves beyond any single medium or visual surface and becomes embedded in space itself, thereby intensifying the audience’s embodied sense of presence.
In the behavioral dimension, audience engagement shifts from passive reception to the generation and activation of narrative. In City Memory: Beijing, memory is reconstructed through wandering, sensing, and spatial immersion. In A Curious World, the acts of building, exploring, and socializing do not merely accompany the narrative but constitute it. Although Luxor in Reflection does not involve direct user interaction in a strict sense, the audience’s sensory encounter with the temple environment, together with
Multimedia Design Research From the Perspective 201 their cultural resonance with the projected imagery, functions as a form of embodied response. In this sense, behavior is no longer a secondary operation; it becomes a trigger for meaning and emotion, forming a dynamic loop between action and narrative semantics.
Taken together, the Time–Space–Behavior triadic model reveals the operational logic of tangible narrative systems: temporal collage, immersive spatial juxtaposition, and behavioral-semantic mapping. These three dimensions collectively form an embodied narrative system that moves beyond image-centered and linear modes of storytelling. In an era increasingly shaped by AIGC technologies and immersive cultural environments, this model offers not only a theoretical foundation for multimedia design but also a practical framework for future experiments in collaborative narrative, cultural expression, and spatial organization.
Figure 6: Schematic diagram of the embodied narrative model.
CONCLUSION
This study has focused on the concept of tangible narrative in order to examine how digital art and multimedia design might respond to the challenges of content oversaturation and experiential fatigue in the context of emerging AIGC (Artificial Intelligence Generated Content) technologies and the rise of immersive culture. Beginning with a review of the historical development of human-computer interaction (HCI) and interface technologies, the paper traces the evolution of tangible interaction and ultimately` proposes the Time– Space–Behavior triadic model as a conceptual framework for understanding contemporary narrative design.
Through close analysis of three representative case studies—City Memory: Beijing, A Curious World, and Luxor in Reflection—the study identifies three distinct narrative modalities: spatial memory reconstruction, behavior-driven generation, and historical-cultural dialogue. Taken together, these cases demonstrate how embodied behavior, physical environments, and layers of
202 Peng virtual information converge to form complex narrative systems. The findings suggest that tangible narrative not only departs from the linear structures and medium-specific constraints of traditional storytelling, but also redefines the role of the audience, shifting it from passive viewer to active co-creator.
At the disciplinary level, this research expands the dialogue between interaction design and narrative theory by foregrounding embodiment, spatial situatedness, and participatory mechanisms as central dimensions of narrative construction in digital media art. The proposed model thus offers a practical analytical framework for artists, designers, and researchers seeking to create more situated, responsive, and interactive storytelling experiences.
In a broader cultural sense, tangible narrative points toward a mode of expression grounded in empathy, connection, and collaboration. It therefore holds the potential to push future narrative forms toward greater democratization, plurality, and experiential richness. More than a technical method, it can be understood as an emergent cultural practice that weaves together technology, embodiment, and social relations, reshaping how stories are produced, experienced, and shared in the age of intelligent media.
ACKNOWLEDGMENT
The successful completion of this research has greatly benefited from the profound artistic explorations and outstanding scholarly contributions of Mr. Wang Zhigang and Mr. Fei Jun. Through an in-depth analysis of their works, I have drawn valuable creative insights and theoretical inspiration. Their artistic practices not only provided the central case studies for this paper but also laid an important conceptual foundation for its analytical framework. I would therefore like to express my sincere gratitude to Mr. Wang Zhigang and Mr. Fei Jun.
REFERENCES
Dourish, P. (1999). Embodied interaction: Exploring the foundations of a new approach to HCI. Work, 1(1), 1–16. Dourish, P. (2001). Where the action is: The foundations of embodied interaction.
MIT Press. Guo, Z.-C. (2022). Embodied interactive narrative: An experiential interpretation of narrative forms in the intelligent era. Journalism & Communication Review, 75(2), 23–34. Liu, M.-Y., Liu, Y.-F., & Lu, X.-T. (2023). Interactive narrative research empowered
by AIGC. Journal of Beijing Film Academy, (11), 92–99. McErlean, K. (2021). Jiaohu xushi yu kuameiti xushi: Xin meiti pingtai shang de chenjinshi gushi chuangzuo [Interactive narrative and transmedia storytelling: Immersive story creation on new media platforms] (B. Sun, R. Li, & Y. Ding, Trans.). Communication University of China Press. Streeck, J., Goodwin, C., & LeBaron, C. (2011). Embodied interaction in the
material world: An introduction. Embodied Interaction, Language and Body in the Material World, 1–26. Wang, Z. (2012). Shuzi meiti xushi yanjiu [Study on digital media storytelling].
Communication University of China Press.
Multimedia Design Research From the Perspective 203 Wei, Y. (2017). Xin meiti jiaohu yishu [Interaction art of new media]. Chemical
Industry Press. Williams, A., Kabisch, E., & Dourish, P. (2005, September). From interaction to participation: Configuring space through embodied interaction. In International Conference on Ubiquitous Computing (pp. 287–304). Berlin, Heidelberg: Springer Berlin Heidelberg. Xu, P. (Ed.). (2020). Zhongguo xin meiti yishu jianshi [A brief history of Chinese
new media art]. Peking University Press. Zhang, W., & Fu, L. (2010). Design research on interactive behavior in multimedia works. Zhuangshi, (4), 104–105. Zhang, Y., & Li, X.-X. (n.d.). The development trajectory of digital narrative research
in China and the construction of an autonomous knowledge system . Xin meiti yu shehui [New Media and Society], 1–18.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Communication of Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-92-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional