Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evergreen: A Mixed Reality Transformation for Experiencing Plant Dyeing

1 Graduate School of Frontier Sciences, The University of Tokyo, Kashiwa, Chiba 2770882, Japan ↗ ↗
2 School of Software and Microelectronics, Peking University, Beijing 102600, China ↗
3 School of Computer Science, Sichuan University, Chengdu, Sichuan 610065, China ↗
4 School of Film, Xiamen University, Xiamen, Fujian 361005, China ↗
5 College of Arts, Xiamen University, Xiamen, Fujian 361005, China ↗
6 University of South California, Los Angeles, California 90007, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1003426

Abstract

Plant dyeing is a traditional process of dyeing with plants, which is natural, environmentally friendly and safe. However, in modern times, due to the widespread popularity of low-cost chemical dyeing and changeless conventional exhibition forms, plant dyeing is gradually losing people's attention. With an increasing urgent need to promote this sustainable ancient handicraft, the transformation and evolution of conventional plant dyeing exhibitions are of great importance, towards experiences that are helpful for the public to understand courses of plant dyeing, during which the advocated products bring people the aesthetic satisfaction and consumption desire. To enhance the effectiveness and improve the learning experience of exhibition activities, we applied MR (Mixed-Reality) technologies to construct an environment by merging the real and virtual worlds. Allowing participants to interact with the coexisting physical and digital objects, this MR environment sets up an interactive feedback information loop between the virtual world, real world and participants, intensifying the sense of reality in user experience. In this paper, we designed a transformation of promotion from traditional physical exhibitions to mix-reality ones, and implemented Evergreen, a MR interactive space that allows people to experience plant dyeing. As it was demonstrated at an art exhibition in Xiamen city, China, we conducted user tests on site to verify the effectiveness of this proposed transformation.

INTRODUCTION

Plant dyeing is a process of dyeing using various pigment-containing plant extracts in nature. It is natural, environmentally friendly and safe, which

© 2023. Published by AHFE Open Access. All rights reserved. 658

Evergreen: A Mixed Reality Transformation 659

is not available in modern chemical dyeing. Nevertheless, due to the wide- spread popularity of low-cost chemical dyeing in modern times, plant dyeing has gradually lost people’s attention. With the increasing contamination issues caused by chemical dyeing and declining exposure to the traditio- nal handicraft, the needs to promote this sustainable dyeing to people are becoming more and more urgent. In this paper, we investigated the method to create a system for promoting the concept of plant dyeing through MR and Human-Computer Interaction (HCI) technology. Plant dyeing refers to a series of traditional courses, for the purpose of dyeing with plants, resulting in nearly zero harmful consequences on both human bodies and ecology. Although exhibition of plant dyeing products has been conside- red as one of the most important promotion activities, which provides a platform for the public to understand the products and processes of plant dyeing, by previous interview and investigation, it was revealed that conven- tional plant dyeing exhibitions are faced with various limitations such as product displays in single form, less understandable courses introduction, and unpleasant attached experiences (e.g., smell and stain of pigment), which in many cases, determining that conventional exhibitions are no longer enough to satisfy participants’ experience as well as their aesthetic needs.

For several decades, MR (Mixed-Reality) has been in continuous develo- pment both theoretically and practically, it refers to a range of technologies that enable users to interact with a combination of virtual and physical obje- cts in a real-world environment. There are several different types of MR, including Augmented Reality (AR), Virtual Reality (VR), and Hybrid Rea- lity. Due to advances in hardware and software in recent years, MR has been improved into more widely available and popular technologies applied in gaming, education, training, design, and entertainment. They can also be used in a variety of industries, such as healthcare, manufacturing, and retail, becoming an aroused general interest.

With a purpose of improving conventional exhibitions effectiveness and learning experience, MR technologies were applied to construct an environ- ment merging the real and virtual worlds, where physical and digital objects coexist and allow users to interact simultaneously. This MR environment sets up an interactive feedback information loop between the virtual world, real world and participants, to enhance the sense of reality of user experie- nce. In this paper, we focused on the innovative application of MR in plant dyeing exhibitions to design a transformation of promotion from traditio- nal physical exhibitions to mix-reality ones, and we implemented Evergreen, a MR interactive space that allows people to experience plant dyeing. The design thinking of Evergreen is elaborated from three major interaction secti- ons, which convey the concept of plant dyeing, attracting people to approach and appreciate this ancient handicraft. Evergreen was implemented using the Unity3D platform, somatosensory interaction technology and illumination techniques. Additionally, we demonstrated it at an exhibition in Xiamen city, China, and conducted user tests to verify the effectiveness of this proposed transformation.

USER EXPERIENCE IN PLANT DYEING EXHIBITION Plants dyeing is a process in which natural plants such as flowers, tea and medicinal materials are used to make dyes and dye fabrics (see Figure 1), whose origins date back 3,000 years (Druding, 1982), is a time-honored han- dicraft technique documented in ancient writings in many parts of the world. Compared with chemical dyeing, plant dyeing has better biodegradability, and the resources consumed and pollutants discharged in the production process are far less than chemical dyeing, which is more energy saving and environmental protection (Devi, et al., 2013). In addition, plant dyes are non-toxic and harmless, and will not affect the user’s health (such as skin, respiratory system, etc.), making it safer for the human body (Elsalam et al., 2008) (Chengaiah et al., 2010). In addition, the color of the fabric dyed by plants is pure and soft, emitting the natural fragrance of plants, which can increase the aesthetic level of the product and bring users more sensory enjoy- ment (Koren, 1993). Plant dyeing conveys a healthy and sustainable lifestyle (Schmidt-Przewo´zna, 2009). It is a dyeing process worth promoting. Howe- ver, due to the low cost of chemical dyeing and the ease of mass production, plants dyeing has been on the decline in the modern market, and even many young people have not heard of it. With the environmental pollution cau- sed by chemical dyes becoming more and more serious (Kant, 2011) (Padhi, 2012) and the popularity of the concept of environmental protection in recent years, the plants dyeing industry has begun to revitalize (Kadolph, 2008), and the demand for publicity of related processes and products is also increasing rapidly.

In order to promote plant dyeing to the public, the exhibition of plant dyeing products is an important platform. People can learn about the pro- cess of plant dyeing and get close to its products at the exhibition. However, through the investigation and analysis of traditional plant dyeing exhibitions, we find that static display is the mainstream and this one-way inculcation of information transmission is difficult to arouse the interest of participants, nor can it form a deep memory (see Figure 2). Especially for those who do not know much about plant dyeing, due to the single way of presentation, participants cannot interact with the exhibits, and it easily leads to a sense of distance between participants and the exhibition content. This virtually raised the threshold of the exhibition, and the visitors were mostly from the plant dyeing industry, thus unable to achieve the ideal promotion effect.

Figure 1: Plant dyeing, an ancient traditional handicraft.
Original PDF Figure 1:

Evergreen: A Mixed Reality Transformation 661

Figure 2: Conventional exhibition of plant dyeing.
Original PDF Figure 2:

As for the technological process of plant dyeing, most of the exhibitions only provide the introduction using words and images, which cannot allow participants to truly experience the process of plant dyeing, and it is difficult to get a real and profound experience. For those exhibitions that can provide real dyeing experience on site, the participants who are novice at plant dyeing are not familiar with the complex process and dyeing skills, which leads to an unsatisfactory dyeing effect and experience. Secondly, the dying process requires the extraction of pigment from a large number of plants. Therefore, a field experience usually consumes a lot of dyeing materials, and the waste of dyeing materials is relatively serious due to the participant’s unfamiliar ope- ration. In addition, the actual plant dyeing process is time-consuming and complex, and the limited time in exhibitions often fails to allow participants to observe the real process and dyeing outcome. What’s more, the complica- ted dyeing operation may even make them feel bored, which is not conducive to their visiting experience.

PROSPECTS OF TRANSFORMATION Defined as a particular subset of Virtual Reality (VR) related technologies (Kishino et al., 1994), Mixed Reality (MR) superimposes virtual objects on the real world, allowing users to sense the virtual scene and real world simultaneously. Mixed reality technology has a wide range of applications in entertainment and interactive arts, as well as engineering and medical infor- matics. Because of the inherently interdisciplinary attributes (Ohta et al., 2014), MR technologies have aroused general interests to scientists in the fields of computer graphics, computer vision, human-computer interaction, and multimedia technologies. Although participant-observers of MR have no need to fully immerse themselves in the virtual environment like VR, they could still obtain the experience that exceeds the boundaries of physical rea- lity. Benefiting from its intuitive manipulation of virtual elements, Mixed Reality expedites a high-bandwidth communication between users and inte- raction context (Billinghurst et al., 1999), and that advantage on information dissemination certainly arose researchers’ interests to apply MR in museum exhibitions (Hughes et al., 2004), hybrid tourism (Debandi, et al., 2018) and video games entertainment (Herbst et al., 2008), while studying the experi- ences that Mixed Reality provide for interaction participants (Trunfio et al., 2020).

Human-Computer Interaction (HCI) focuses on the communication betw- een users (the human) and machines (the computers), to facilitate effective and efficient information exchange. While originally concentrated on com- puter technology and especially the design of graphic user interfaces and usability of software, HCI has completed a meteoric rise and became a ubi- quitous interdisciplinary subject emerging computer science, human factor engineering, and cognitive sciences. Responding to the demands of upleveling attraction and enlarging user groups, researchers have started to elaborate multifarious interactive applications for museum exhibitions and cultural heritage protection (Maye et al., 2014). For instance, earlier before, Sparacino et al. attempted different types of interactive applications in museum space, providing visitors with more unforgettable and fun learning experiences than traditional exhibits (Sparacino et al., 1999). Along with the development of HCI technologies and theories about interactive exhibition, researchers explored how different technologies influence visitors and what can be the best scenarios (Hornecker, 2010). And recently, there appears more studies that illustrate the ability of HCI technologies in museums to create strong connections between participant and exhibition contents. Cárdenas Gasca et al. reported their explorations of depicting sensitive narratives in an immer- sive environment through AR technology, to enhance the connection between participants and victims (Cárdenas et al., 2022).

DESIGN THINKING IN EVERGREEN Looking forward to transforming from the traditional exhibition, it is vital to provide a more vivid and intuitive introduction, consisting of refined relevant knowledge and enriched learning experience. Hence, we abstracted and sum- marized the actual courses of plant dyeing, designing an experience model (see Figure 3) for the participants. The first original plant dyeing handicraft on the top layer includes detailed and trivial working procedures, which indi- cates its tremendous manual value but establishes a boundary preventing most beginners from further practicing this tricky handwork. On the second layer, conventional courses are abstracted and summarized into three major sections: collecting plant materials from nature; extracting pigment from col- lected plants; and dyeing textiles with extracted pigment. On the third layer, transformed experiences serves as an autotelic process for participants to receive information from the provided interaction sections, which were desi- gned based on the extraction and digitalization of conventional plant dyeing courses. Through the process of acquiring transformed learning experiences provided by the mixed-reality interaction, we supposed that participants are able to conceptually understand the vague procedures of plant dyeing. As for the bottom layer, to start with the rough impression and comprehension gene- rated during the process of interaction, participants are expected to restore relevant information and enhance the former learning by themselves with imagination, to ultimately obtain relatively expanded recognitive knowledge, deepening the existing impression and understanding about plant dyeing.

Compared to artisans conventionally searching for herbs and collecting plants from natural environments, the participants would firstly select plants

Evergreen: A Mixed Reality Transformation 663

Figure 3: Participant experience model.
Original PDF Figure 3:

in the digital virtual forest. In the second step, as every kind of plant can be processed to produce a specific variety of colors, participants are allowed to alternate one color from the selected plant, corresponding to the pigment extraction course in traditional handicraft. When finishing color selection, the moment participants wave their hands from virtual plants towards the physical clothes, dyed colors and patterns are going to be rendered and pro- jected onto the model in white dress that was displayed in a real exhibition space. During the experiences of interaction above, the participants could efficiently understand the workflow of plant dyeing to a general extent, and get an impression of its natural aesthetic as well as environmental friendliness. By a process of imaginative restoration after interaction, the understanding about plant dyeing would be spontaneously expanded, helping participants per se to realize various valuable qualities of plant dyeing, from perspectives of lifestyle, heritage, harmony etc.

To provide a learning experience that exceeds existing exhibitions while connecting to an ancient traditional handicraft, we designed Evergreen, an interactive space where virtual and reality are embedding, connecting and mutually enhancing. The design of space is as shown in Figure 4, the par- ticipant would face the physical clothing (A), the display screen (B), the somatosensory device Kinect (C) and plant samples for exhibition (D), while the projector (E) is in the corner. The physical clothing would be projected with various colors and patterns by the projector. And the somatosensory device tracks the movements of the participant, meanwhile the virtual forest is displayed on the screen. Five different kinds of vegetation used in plant dyeing are in the virtual forest for selection.

When the participant is standing in the interactive area and makes specific body movements, the virtual forest responds by moving the indicator cursor along with real-time movement of the participant’s right hand. At the position of plants, a grabbing action can be performed, then the plant will be picked up, and when the participant releases hand, the plant will immediately fall. At the moment the participant waves hand to dye, the projector projects the

Figure 4: Conceptual design picture of evergreen.
Original PDF Figure 4:

corresponding colors and patterns to the physical clothing according to the selection of plant and color.

IMPLEMENTATION Based on design thinking, we used Unity3D engine and Kinect 2.0 to imple- ment Evergreen. The project structure is shown in Figure 5. The whole project is realized in the Unity3D engine. Kinect recognizes the participant’s posture and gesture information, and controls the interaction with different objects in the virtual scene, as well as the changes of colors and patterns on the physical clothing. We created two cameras in Unity to capture the virtual scene and the clothing effect, respectively. The virtual scene was presented on a screen, and the clothing effect was projected by the projector onto the real clothing in the space to present the visual effect of virtual and real combination.

Figure 5: Installation structure of evergreen.
Original PDF Figure 5:

Evergreen: A Mixed Reality Transformation 665

Figure 6: Demonstration of evergreen in exhibition “Echo-Image”.
Original PDF Figure 6:

RESULTS

We brought Evergreen to the exhibition “Echo-Image”, which attracted a lot of participants to experience (see Figure 6). Through questionnaires and interviews, we collected feedback from some participants on Evergreen’s experience and conducted statistical analysis. The participants are between 15 and 25 years old, and they are young people who are familiar with MR and other new media interaction technologies. They are also the target users for us to promote plants dyeing through Evergreen.

According to the statistical results of the questionnaire, more than 90% of participants did not know about the plants dyeing before the experience, while more than 80% of the participants thought that Evergreen helped them understand the concept of plants dyeing. More than 70% of participants thought that Evergreen effectively conveys the “natural and environmental protection” characteristics of plant dyeing. For Evergreen’s interactive expe- rience, more than 75% of participants thought the interaction process was clear and easy to understand, 90% of participants thought that the clothes projection was novel and interesting. More than 80% of participants thou- ght that the interactive form is reasonable and easy to understand. For virtual scenarios and interface design, more than 75% of participants thought the scenarios are beautiful and the interface layout is reasonable. In summary, Evergreen received ideal feedback from the young participant at the exhibi- tion. It can effectively attract people who are not familiar with plants dyeing to participate in the experience and convey the concept and characteristics of plants dyeing during the experience.

Some participants suggested that the plant models in the virtual scene were not realistic enough, the patterns of projection were not quite delicate and there was a certain delay in gesture recognition. In the next step, we will improve the refinement of the image elements and make more realistic scenes and models. In addition, the Kinect recognition and control algorithm will be optimized to achieve more responsive feedback during interaction.

DISCUSSION

In this research, we investigated and proposed a transformation of plant dyeing exhibition, from conventional physical learning to digital interactive experiences, with a purpose of improving the effectiveness of promotion

activities of this traditional handicraft. In the 4-layer experience model, the excessive amount of information contained in conventional courses of plant dyeing were refined and summarized into 3 major sections: plant collection, pigment extraction and dyeing. In the next layer, transformed digital intera- ctive experiences were designed correspondingly, consisting of virtual plant plucking, digital color selection, and visual projection onto physical clothes. Based on the learning experiences, the participants are expected to launch an autotelic imaginative restoration process, to spontaneously realize values of plant dyeing in various aspects from a widened perspective, further under- standing aesthetics, harmony spirit and natural lifestyle behind an ancient handicraft. Ideas above implemented by MR (Mixed-Reality) technologies have composed Evergreen, an interactive space which allows participants to conceptually experience plant dyeing through coherent playful activities with both virtual (plants in digital forest) and physical objects (dressed model in real world). Demonstrating Evergreen on the exhibition activities held in Xia- men city, we conducted user tests by questionnaire surveys and interviews, in terms of which the effectiveness of our transformed interactive learning experiences about plant dyeing were preliminarily verified. With suggested expectation about improvement on somatosensory controlling, Evergreen is optimistically anticipated to provide more copious content and respon- sive experiences in the future, and implication about advocating endangered traditional intangible heritage such as ancient handicraft.

REFERENCES

Abd Elsalam, O., Barakat, G., & Abdulwahah, H. (2008). Association between materials used in children’s-clothes and contact dermatitis. Journal of Applied Sciences Research, Volume 4 No. 10, pp. 1155–1165. Billinghurst, M., & Kato, H. (March, 1999). Collaborative mixed reality. In

Proceedings of the first international symposium on mixed reality. pp. 261–284. Cárdenas Gasca, A. M., Jacobs, J. M., Monroy-Hernández, A., & Nebeling, M.

(2022, June). AR Exhibitions for Sensitive Narratives: Designing an Immersive Exhibition for the Museum of Memory in Colombia. In Designing Interactive Systems Conference. pp. 1698–1714. Chengaiah, B., Rao, K., Kumar, K., Alagusundaram, M., & Chetty, C. (2010). Medicinal importance of natural dyes-a review. International Journal of PharmTech Research, Volume 2 No. 1, pp. 144–154. Debandi, F. et al. (2018) “Enhancing cultural tourism by a mixed reality applica-

tion for outdoor navigation and information browsing using immersive devices,” IOP Conference Series: Materials Science and Engineering, Volume 364 No. 1, pp. 012048. Devi, Meena and Ariharan, VN and Prasad, N. (2013). Annato, eco-friendly and potential source for natural dye. International Research Journal of Pharmacy, Volume 4 No. 6, pp. 106–108. Herbst, I., Braun, A. K., McCall, R., & Broll, W. (September, 2008). TimeWarp:

interactive time travel with a mobile mixed reality game. In Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. pp. 235–244.

Evergreen: A Mixed Reality Transformation 667

Hornecker, E. (January, 2010). Interactions around a contextually embedded system.

In Proceedings of the fourth international conference on tangible, embedded, and embodied interaction. pp. 169–176. Hughes, C. E., Smith, E., Stapleton, C. B., & Hughes, D. E. (November, 2004).

Augmenting museum experiences with mixed reality. In Proceedings of KSCE 2004 pp. 22–24. Kadolph, S. (2008). Natural dyes: a traditional craft experiencing new attention.

Delta Kappa Gamma Bulletin, Volume 75 No. 1, pp. 14. Kant, R. (2011). Textile dyeing industry an environmental hazard. Scientific research publishing Koren, Z. (1993). The colors and dyes on ancient textiles in Israel. Colors from

Nature: Natural Colors in Ancient Times, Eretz-Israel Museum, Tel-Aviv, Volume 15 No. 31, pp. 47–65. Maye, L. A. et al. (2014) “Interactive exhibitions design: what can we learn from cultural heritage professionals?”, Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, pp. 598–607. Milgram, P., & Kishino, F. (1994). A taxonomy of mixed reality visual displays.

IEICE TRANSACTIONS on Information and Systems, Volume 77 No. 12, pp. 1321–1329. Ohta, Y., & Tamura, H. (2014). Mixed reality: merging real and virtual worlds.

Springer Publishing Company, Incorporated. Padhi, B. (2012). Pollution due to synthetic dyes toxicity & carcinogenicity studies and remediation. International journal of environmental sciences, Volume 3 No. 3, pp. 940–955. Schmidt-Przewo´zna, K., & Zimniewska, M. (2009). Natural dyeing plants as a sou-

rce of compounds protecting against UV radiation. Herba Polonica, Volume 55 No. 3, pp. 311–318. Sparacino, F., Larson, K., MacNeil, R., Davenport, G., & Pentland, A. (September, 1999). Technologies and Methods for Interactive Exhibit Design: From Wireless Object & Body Tracking to Wearable Computers. In ICHIM. pp. 147–154. Susan C. Druding, (1982). Dye History from 2600 BC to the 20th Century. Susan C.

Druding. Trunfio, M., & Campana, S. (2020). A visitors’ experience model for mixed reality in the museum. Current Issues in Tourism, Volume 23 No. 9, pp. 1053–1058.

Cover of the proceedings volume Ergonomics In Design

Explore the complete digital volume

Ergonomics In Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-53-7

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →