Luo Hong, Kang Xinyan, Li Sheng, Zhou Yangmengliu (2023). Studying Tian Hock Keng's architectural heritage through the lens of interaction through digital design. In: Usability and User Experience. AHFE Open Access, vol. 110, pp. 475–481. https://doi.org/10.54941/ahfe1003222
The history of human civilization has formed a large number of precious architectural heritage, which reflects a country's achievements in engineering technology, social development, culture, art, etc. This architectural heritage also covers the concepts of architectural design planning. However, with the changes in the environment and the influence of human activities, cultural heritage is inevitably eroded and destroyed. The distribution of architectural cultural heritage is wide and large, and the resources for architectural protection are given priority to projects with higher value, and it is impossible to cover all. Under the background of the digital age, the digital preservation and dissemination of cultural heritage have gradually become an inevitable trend. Digital cultural heritage technologies supported by technologies such as virtual reality, augmented reality, and 3D modeling provides rich means for digital recording, preservation, display, and inheritance of cultural heritage, and expand the connotation of traditional cultural heritage protection.Virtual reality technology has been widely used in the protection and dissemination of architectural cultural heritage. As a national key cultural relics protection unit in Singapore, THK has a long history and rich ancient architectural resources. The research aims to digitally preserve THK from inevitable physical damage by creating 3D interactive models, which can restore the historical, cultural, and aesthetic value of THK's architectural heritage. With breakthroughs in perceptual interaction technology in recent years, the mode of natural interaction in VR has more possibilities. This research mainly focuses on the application of natural interaction modes based on eye and hand movement tracking on architectural cultural heritage dissemination. Meanwhile, it also explores the application in the dissemination of architectural cultural heritage and summarizes the interactive designing strategy. As the eye movement and hand movement tracking modules are applied to the virtual interactive display of THK, we have the following objectives. Firstly, we will analyze the characteristics of information dissemination which is in various forms in natural interaction. Moreover, we will validate design strategies for natural interactions with experiments, optimizing the user experience of the audience who are experiencing the interaction. Particularly, the 3D interactive mode can interactively transport users back in time. Although both the aesthetic elements and the architectural space have evolved through time, users can still interact with them. To experience the history and culture, target users can watch and interact with THK digital architecture models on their mobile phones from anywhere. This study can improve the impact of information spreading and make helpful explorations of the digital dissemination method of cultural heritage, as well as have positive effects on the distribution of spatial structural information and historical and cultural information about architectural heritage.
The history of human civilization has formed much precious architectural heritage, reflecting a country’s achievements in engineering technology, social
development, culture, art, etc. This architectural heritage also covers the concepts of architectural design planning. However, with the changes in the environment and the influence of human activities, cultural heritage is inevi- tably eroded and destroyed. The distribution of architectural cultural heritage is wide and large, and the resources for architectural protection are given pri- ority to projects with higher value, and it is impossible to cover all. Under the background of the digital age, the digital preservation and dissemination of cultural heritage have gradually become an inevitable trend. Digital cultural heritage technologies supported by virtual reality, augmented reality, and 3D modeling provide rich means for digital recording, preservation, display, and inheritance of cultural heritage, and expand the connotation of traditional cultural heritage protection.
DIGITIZATION OF ARCHITECTURAL HERITAGE In the development of virtual reality (VR) application systems for scholars and experts, a variety of information from different sources is integrated into the virtual space, and this information is closely related to the digital stru- cture of architectural heritage, providing researchers with a powerful visual research tool (Huang Xin-yuan, 2017). Another application of VR in the field of architectural heritage protection is to provide users with a complete sensory experience of historical scenes in an immersive interactive experie- nce, through the highly realistic visual restoration of architectural heritage (Isabelle Verhulst, 2021). The main purpose of its development is to disse- minate the aesthetic value and spiritual value of architectural heritage to the public and to popularize the structural characteristics of ancient buildings and the wisdom of traditional creation.
Several cases apply VR technology to the virtual restoration technology of ancient buildings. They have strong R&D teams and advanced software tech- nology. For example, William Jepson’s team at the University of California, Los Angeles, their virtual representation of ancient Jerusalem is now displa- yed at the Davis Demonstration Center in Israel (Yan, J, 2004). Alessandro’s team from the University of Geneva, Switzerland, performed a virtual resto- ration of the badly damaged Hagia Sophia (Zhang, L, 2009). In the virtual restoration project of architectural heritage, the Chinese Academy of Scie- nces proposed an improved algorithm for a multi-level cache, which played a key role in the digital Dunhuang project (Zhang, J, 2002). The National Key Laboratory of Zhejiang University has played a key role in projects such as the Digital Great Wall and the Digital Yuanmingyuan (Zhang, M.J, 2003). China’s first work based on VR technology is “The Palace Museum, Empe- ror’s Palace” (see Figure 1). It uses computer technology to build a realistic 3D model simulation scene of the Palace Museum, and tourists can watch the Palace Museum from different perspectives through intelligent remote control devices (Jiang, L.W, 2016).
Natural Interaction in Architectural Heritage Natural interaction is a human-computer interaction method that improves the interactive experience and the efficiency of information dissemination.
Studying Tian Hock Keng’s Architectural Heritage Through the Lens of Interaction 477
Figure 1: The Palace Museum The Emperor’s Palace Virtual Scene. (From Palace Museum, 2016).
In the natural interaction, the natural behaviors of the user include body movements, gesture movements, voice, and other behaviors. The interactive subject communicates with the computer system through these methods to achieve the purpose of information exchange. Humans use innate skills to communicate directly with digital information content through natural inte- ractions (Julius Pettersson, 2020). In the process of natural interaction, the methods used by the users to communicate with the computer system include gaze, hand movements to touch, grasp and manipulate an object, and verbal instructions to the system (Yan Yukang, 2019).
Visual attention is a major way of studying human cognition. In the immer- sive virtual space interaction process, humans obtain external information through vision, and the eye-tracking module in the VR headset continuously obtains the user’s focal point of sight. In this area, the input of interactive instructions is realized and the information output by the system is obtained, to complete the whole process of interactive behavior. In addition, the most concise and natural interaction process is completed through the movement of the eye gaze point, staring, blinking, and other actions (Zhang, L, 2021). In 2018 Japanese VR startup FOVE released the industry’s first eye-tracking VR headset. Meanwhile, Tobii, an eye-tracking developer from Sweden, has begun licensing its technology for consumer VR products. Global research firm Markets and Markets believes that eye-tracking market capitalization will reach $1.4 billion by 2023, in part due to its prominent role in VR products (Eric Kuerzel, 2018). The operation of human hands can complete various complex actions, realize precise manipulation of objects, and at the same time transmit rich information through hands. Using the action hands without wearing any equipment is one of the most natural human-computer interaction methods to complete the interaction between people and the vir- tual world. In the immersive virtual space, the user completes the interactive instruction input by grasping, touching, and manipulating with bare hands (Huang Pei-de, 2021). In 2010, the emergence of the somatosensory peri- pheral 3D camera Kinect pushed the development of gesture recognition (Feng, Z, 2012). LEAP introduced a somatosensory device that can capture hand movements at close range, bringing a new direction to human-computer interaction (Leap Motion, 2015). The Dunhuang Research Institute applied Leap Motion-based somatosensory interaction technology to the Dunhuang
Art Gallery, giving visitors a sense of immersion during their visit (Dading, L, 2015).
Chinese Temple Architectural Heritage Tian Hock Keng is the oldest Chinese temple in Singapore (see Figure 2). According to the record of the Establishment of Tian Hock Keng (THK) Inscription in 1850, the construction of THK started in 1839 and was finally completed in 1842. The construction period lasted 3 years (DU Nan, 2010). In architectural history, it inherits the obvious features of the large-scale arch- itectural system of ancient Chinese palaces and temples (LI Li, 2022). Over the past 100 years, Tian Hock Keng (THK) has undergone many restorati- ons. In 1906, the Hokkien Guild Hall donated money to repair the THK. In 1940, the Hokkien Guild Hall built halls on both sides of the temple and church building. In 1998, it cost more than 4 million Singapore dollars to complete this huge restoration project which lasted for three years. Tian Hock Keng won the UNESCO Asia-Pacific 2001 Cultural Heritage and Anci- ent Architecture Award from the United Nations after extensive restoration (Tan, 2005). With natural disasters and the passage of time, although there are many restorations and a lot of capital investment, it is still difficult to reproduce the extreme scene at that time in terms of architectural structure, space decoration pattern, and color gloss.
NATURAL INTERACTION IN THE DIGITIZATION OF TIAN HOCK KENG The goal is to design a virtual reality system for architectural heritage based on natural interaction and to apply virtual reality (VR) interaction technology to assist in the generation of immersive experiences of architectural heritage. The theoretical basis will start with the documents of Tian Hock Keng (THK), especially the images and information before and after each restoration. The research group will collect real photos and analyze the decorations and details in these photos of the THK, to provide a picture basis and texture reference for the subsequent digitalization process of THK. Through on-site mapping and hand-painted exploration, the research group applied tilt photogramme- try to reconstruct the shape of the palace, the wooden frame structure, the roof decoration, and the plaque in 3D with high precision and restored the
Studying Tian Hock Keng’s Architectural Heritage Through the Lens of Interaction 479
original appearance of Tian Hock Keng’s architectural heritage in Unity with high precision. After the basic structure and scale of the building model were completed, further information correction was carried out and photo mate- rials collected on site were used as realistic mapping, while attention was paid to restoring authenticity by using the same daylight period. The resea- rch group focused on HTC Vive’s eye-tracking technology and hand motion tracking in the architectural heritage interaction experience. Setting up eye- tracking interaction with information in an immersive architectural scene was able to reduce the interaction path for the user. After entering the VR system, users can use the handle to achieve virtual scene observation, where the green dot on the screen is the focus of the user’s vision in the virtual scene, it can help the user and the proposed heritage scene to complete the information interaction experience.
Eye Tracking Interaction Design Interactions regarding eye tracking are set up in the architectural heritage virtual reality system, including gaze, gaze interruption, and default states. When the user starts to enter the virtual scene, the icon will be activated, the user’s eyes will be in the range of the icon, the animation effect of the virtual system will be activated, and the eye tracking will be activated after the gaze is maintained for a period to obtain information interaction. When the user’s eyes are out of the range of the icon, it is a gaze interruption state; the default state is the default mode after the user’s gaze interruption. In the specific application, the key user’s eyes remain on an icon for too long, activating and obtaining detailed information about the construction of this icon and obtaining audio related to the historical context of Tian Hock Keng (THK).
Hand Motion Tracking Interaction Design In the past, complex hand movements were difficult to display in virtual rea- lity (VR) scenes, and there was a lag in the coordination of people, objects, and scenes, which affected the interaction between people and information. The hand tracking module in the architectural heritage virtual reality system is mainly based on the Tian Hock Keng (THK) building 3D model. Users can use hand tracking to perform different behaviors such as clicking, gra- sping, nudging, and stacking in different architectural scenes. At the same time, when clicking on the icons and completing each task according to the guidance, the system will display the corresponding image and text informa- tion, and play pleasant music as positive interactive feedback to guide the user to continue the interactive activities. At the same time, the architectu- ral heritage virtual reality system incorporates a voice mode, with different voice messages for each architectural scene, and matching voice prompts will appear when the user uses eye tracking and hand tracking to avoid the boring feeling brought by large text messages.
CONCLUSION
This topic takes traditional architectural heritage Tian Hock Keng (THK) as the research object, firstly analyzes the characteristics of virtual reality in
natural interaction in various forms of communication, carries out 3D digi- tal reconstruction of THK architectural heritage through 3D modeling and VR technology, optimizes the user experience of virtual reality system, helps users to be able to experience the history and culture of THK in an immersive virtual scene, and provides digital communication of cultural heritage. These technologies can restore and design architectural features, and interactively display architectural heritage through VR, to deepen people’s understanding of architectural art. On the other hand, the publicity and display of archite- ctural scenic spots can also be carried out in the form of online tourism, which can bring a better VR experience to people limited by time and economic con- ditions, deepen the cognition of the value of architectural art, and make the cultural and historical value of architectural heritage better protected and inherited.
REFERENCES
Chang, Q, Jiang, T. Y, Chen, C, et al. (2016). Reflection on the Fundamental Category of Heritage Architecture. Heritage Architecture, Volume 1. pp. 44–61. Eric Kuerzel. Eye tracking leads gaming technology to the world of augmented rea-
lity and virtual reality. China Electronic Business (Basic Electronics), Volume 3. pp. 39–41. Huang, P. D, Wang, L, Xiao, Z, et al. (2021). Rese- arch on Natural Interaction Method of Virtual Hand in VR Environment. Machinery Design & Manufacture, Volume 7. pp. 268–271. Isabelle Verhulst, Andy Woods, Laryssa Whittaker, et al. (2021). Do VR and AR Ver-
sions of an Immersive Cultural Experience Engender Different User Experiences? [J]. Computers in Human Behavior, Volume 7 No. 10. Ji, C, Li, Y. N. (2021). Digital Creative Design of Pingyang New Year Wood-block Prints Based on Scene Theory [J]. Packaging Engineering, Volume 9. pp. 388–395. Jiang, L. W. (2016). Design and implementation of interactive display in virtual
ancient architecture. Beijing: Beijing University of Technology. Kathryn Smith, Mark Roughley, Samantha Harris, et al. (2020). From Ta-Kesh to Ta- Kush: the Affordances of Digital, Haptic Visualisation for Heritage Accessibility. Digital Applications in Archaeology and Cultural Heritage, Volume 8 No. 10. Leap Motion. Leap Motion product [EB/OL]. (2014-05-1) [2023-01- 28].http://www.leapmotion.com/product. Li Dading, Yang Mengqin, Bo Longwei, et al. (2015). A preliminary investiga-
tion on the application of somatosensory technology in the interactive display of Dunhuang art. Dunhuang Research, No. 153(05). pp. 115–119. Li, J. J, Liu, J, Pan, W. J, et al. (2020). VR System Experience Design Driven by Mental Model in Ancient Village. Packaging Engineering, Volume 1. pp. 262–268. Li, L, Zhang, H. (2022). Authority Produced in Architectural Space: Taking Tianfu
Temple in Singapore as Example. Journal of Huaqiao University (Philosophy & Social Sciences). Volume 1. pp. 29–36. Liu, Y. X, Yin, J. (2021). Digital Cultural Creative Design of Taohuawu New Year Woodblock Printings Based on Cultural Translation. Packaging Engineering, Volume 4. pp. 12. Miao, L. (2018). Exploration of Virtual Reality Technology Applied in Narrative
Design of Museum[J]. Packaging Engineering, Volume 2. pp. 15-18.
Studying Tian Hock Keng’s Architectural Heritage Through the Lens of Interaction 481
Mu, X. D. (2017). Research on the development strategy of museum cultural creation in the context of “Internet +”—Taking the Beijing Palace Museum as an example. News Res. Guide Volume 8. pp. 251–252. Richard Laing. (2020). Built Heritage Modelling and Visualisa- tion: the Potential to
Engage with Issues of Heritage Value and Wider Participation[J]. Developments in the Built Environment, Volume 6. p. 10. Zhang, L, Zhao, W. X, Qiu, Y, et al. (2021). Communication Mode of Architectural Cultural Heritage Based on Natural Interaction: Taking the Jinlian Bridge as an Example. Packaging Engineering, Volume 11. pp. 20-25. Zhang, M. J. (2003). Virtual Reality System. Beijing: Science Press. pp. 5-8. Zhang, S. X. (2020). Research on Traditional Culture Communication in Mobile
Terminal Games: a Case Study of 100 Scenes in Jiangnan. New Media Research, Volume 6 No. 23. pp. 34–36.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Usability and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-86-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional