Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

An Installation on Immersive Dining of Image and Food

Department of New Media Art, Taipei National University of the Arts, 69121 Taipei, Taiwan ↗
Publication year
Digital Object Identifier10.54941/ahfe1001810

Abstract

This research studied the immersive projection technology based on the dining experience of a set meal to show the dining atmosphere, reshape the cultural memory of culinary arts, and highlight the connotation of image and taste. The immersive installations are planned in a dining space with a square table to accommodate eight people. In addition, it is matched with a wall projection to present a sense of a fully immersive environment. The use of infrared cameras and visual recognition results allows the participants to see the presentations. The immersive experiences are in three areas: tabletop, tablewares, and walls. The tabletop shows the abstract images related to the meal; the tablewares interact with dining actions; the surrounding walls of the dining space present the consequence animation from the tabletop. The findings of setting up the projection and detection devices related to the three essential areas to support the immersive experiences in a square-shaped structure space are stated in this paper. The research result describes creating the image related to the food taste, traceability, and surrounding environment.

INTRODUCTION

This project cooperates with the campus restaurant, Da Vinci, to create digital artwork related to three dishes and discuss how to coordinate the immersive dining experience with technology and specific space. The relevant topics learned from this experiment are image recognition, computer vision, algori- thms, and projection techniques. For the art aspect, the stories on the dishes presented with immersive animation, video, sound, and music; interact simul- taneously by the diners’ motion to stimulate the five senses experience of dining.

First, this paper was conducted through the literature survey, and sum- marizing the technical-related “immersive dining” data. Second, develop the “immersive” environmental installations plans in which images are proje- cted on the table and wall, using uninterrupted image connection techniques. Finally, the three dishes are the interactive media to trigger sound, light, video, and audio by dining ware motions. The camera detects both shapes and posi- tions of tablewares on the table, and the images project accurately on the aiming surfaces. Overall, The results suggest integrating the image and food

© 2022. Published by AHFE Open Access. All rights reserved. 300

An Installation on Immersive Dining of Image and Food 301

data reading process performed by computer vision to identify and calculate appropriately in the installation on immersive dining.

IMMERSIVE DINNING EXPERIENCE Digital media technologies are not only affecting what we see but also chan- ging our culinary experience. Several experiences of the technology with existing cases: for example, to enhance the flavor of meals; to provide enter- tainment pleasure when dining; not to mention, to aid in healthy eating. In the Digital Technology and Table Experience journal of Spence, different pathw- ays already exist for digital technology to the table: One is that a restaurant or bar may provide an audio-visual experience in the space of their diners/cu- stomers; Another is the presentation of the stories of the ingredients through diners’ mobile phones on the dining table. For example, use AR technology to enhance the sound of chewing or change the color of the food scene, use QRcode to interact with food, and use interactive technology to control the multi-sensory atmosphere at the dining table. (Spence & Piqueras-Fiszman, 2013) No matter what type of new media arts technology is showing on the table, it intends to use digital media technology to create a multi-sensory experience atmosphere in the dining environment, also known as “immersive dining experience.”

It is important to note that not all immersive dining experiences contain a storyline, nor do they necessarily have actors and characters, especially when presented digitally. These character stories, sights, and sounds bring culinary art to life, telling stories or interacting with guests as they dine. In these environments, food is not only a delicacy in itself but a tool (Gingerline) to enhance an established fantasy world.

Therefore, this research explores creating an immersive environment that refers the cases as follows: In the Ginza district of Tokyo, teamLab has created an immersive new sensory interactive restaurant that provides diners with a unique dining expe- rience. This immersive space perfectly integrates luxury ingredients, artistic utensils, and real-time projection media art technology. The walls and dining tables of the restaurant are projection curtains. The digital visual effects, immersive images, lighting atmosphere, and changing scenes replace tradi- tional decoration, giving diners a new “sensory cooking.” The projected art presentations are affected by the food and tableware on the table, creating a stable. The changing dining environment is used to show the changing seasons in Japan (teamLab, 2017).

“TREE by NAKED Yoyogi Park” changes performances and menus. The fusion cuisine of East and West integrates with light sculpture projection, VR, induction, lighting, smoke, aroma, and music to create a space where diners can enjoy a new culinary experience (Toyoshima , 2021).

Sublimation restaurant in Ibiza, Spain, is a new dining concept created by a team of Michelin-starred chefs, engineers, magicians, designers, architects, choreographers, and screenwriters for nearly two years. This “food show” is carefully choreographed for diners to experience the five senses stimula- tion far beyond the taste buds. With the help of audio-visual technology, the

Figure 1: Left: 3 area of immersive installation, Right: Live photo of installation.
Original PDF Figure 1:

tabletop and surrounding walls will show different scenes. In the distance, the flowers blooming in the Alps, and the projected images are constantly chan- ging. Through the systematic environmental control equipment arranged by the engineer, the temperature, humidity, and even the smell in the room will change with the theme switching (Sublimotion, 2014).

Dinner Time Story’s “Le Petit Chef” is a multi-sensory pop-up dining experience supported by 3D projections that immerse diners in culinary adventures around the world. True to its name, Le Petit, the little chef at TT Liquor in Shoreditch, introduces diners to a bit of chef who is Marco Polo’s travel guide. The more than six-course gourmet dinner experience takes par- ticipants from the Silk Road to France, making it a theatrical experience as the dining event. To further enhance the experience of tasting a variety of cuisines worldwide, Le Petit Chef offers cocktails designed to complement each cuisine (In The Vintage Kitchen, 2018).

THE INSTALLATION

1. Environment Device Configuration

The content of the immersive installation takes a square table that can accom- modate eight people as the main dining area. In addition, it is matched with a wall projection to present a sense of environmental wrapping. With an infrared camera and visual recognition results, the dinners can see interactive images in front of their dining area. The images show in three areas: the theme area, the interactive area, and the background area. The theme area covers the whole square dining table area, the interactive area is the particular dining area in front of the dinners, and the background area is the surrounding walls of the dining space. A projection and detection device is erected directly above the dining table by a truss structure to support the projection and detection device. In addition to the downward projection in the center, eight horizontal outward projections are facing the surrounding walls, and there is a camera above the user to detect responsible for the interaction of meal behavior. The image of background area is 6304X600 dpi, using (Immersive Display PRO) to split into 8 projectors, and one projector for theme area and one projector for interactive area.

An Installation on Immersive Dining of Image and Food 303

Figure 2: Serving time table and photos of immersive dining.
Original PDF Figure 2:

2. Experience Modes

Use the process of serving and dining to provide ambient audio and video effects and generate video interaction in the dining behavior to provide an immersive dining experience. They are as follows: With real-time calculation processing, the results can be output to the inte- ractive area, the theme area, and the background area. The interactive area is where the sensors detect shapes of tablewares used and movements of the diners changes of the position of the tablewares.

1) Detection: The sensors determine the eigenvalues of the shape and posi- tion of the plate, the wine glass, and the corresponding projection position. The system will project three levels of image changes in the form of textures, and the primary purpose is to display the layered sense of the ingredients appropriately. In addition, the story images will use the surrounding walls as the main narrative space. 2) Processing: Distinguish the eigenvalues of the shape and position of the dinner plate and wine glass, and perform the conversion formula calcula- tion. For example, the images accurately displayed on the dish are aligned by another extended image in real-time, even the glass placed out of the area. Analyze the data by identifying the detected shape of the dish and glass to activate various interactive settings in real-time. 3) Output: Image changes for each dish during the meal with ambient lights, light, video, and sound connected to the surrounding walls. For example, a seafood dish presents an image of the ocean.

An infrared camera (Figure 3) senses shapes and the positions of the dish through the algorithm of Yolov5 (Figure 4), which is deep learning through artificial intelligence so that the projection system can recognize the dish’s position and project it at the appropriate positions.

THE DIGITAL CONTENT DESIGN

1. The Story Board 2. Interaction and Animation Design

Table 1. Story Boards for the three dishes.
Original PDF Table 1.

An Installation on Immersive Dining of Image and Food 305

Figure 3: Infrared camera.
Original PDF Figure 3:
Figure 4: Yolov5.
Original PDF Figure 4:
Table 2. Interaction design and animation for the three dishes: Drink : juice Appetizer: salad Main course: seafood, chicken, vegetables
Original PDF Table 2.

CONCLUSION AND DISCUSSION This research project took an empirical performance in the 2021 Kuandu Art Festival in Taipei, Taiwan. 78 shows were held in the one-month exhibition, and about 600 people participated in this dining experience. There are three shows every night, with a maximum of 8 people in each show. This immer- sive dining exhibition went viral and was enthusiastically participated by the public. 44 effective questionnaires feedback from the dinners participated in this exhibition, and the results showed that the 30–39-year-olds were the majority, the 40–49-year-olds were the second, and the 20–29-year-olds were

the third. The age attribution is slightly different from our original expecta- tion that the younger group is the leading group. Part of the reason is the consumption power. This experience requires payment, and the ticket price is equivalent to public art events.

There were no significant differences by gender. In terms of overall sati- sfaction, more than 90% were satisfied with the dining + digital art perfor- mance. The satisfaction ratio of audio-visual effects and taste is relatively high. However, The satisfaction has slightly decreased about the mealtime. The reason is that the mealtime following the storyline to complete in the digital art performance, but the individual dining speed is different. Eight people are dining together simultaneously; the three dishes have to change after the set time is up. The service process is mainly about serving dishes and taking away dishes, and there is not much controversy in recognizing simple actions. The degree of agreement on the relationship between images and dishes is not high, and everyone has a different understanding of arti- stic creation. However, it does not affect the satisfaction with this immersive dining experience.

ACKNOWLEDGMENT

Thanks to the sponsor on this research project. This work was partially funded by the Ministry of Science and Technology (MOST 110-2410-H-119- 006-). Photos courtesy by Anpis Wang.

REFERENCES

Fly, E.-n. (n.d.). Immersive Display PRO. Retrieved from Immersive Display PRO: https://fly.elise-ng.net/immersive-display-pro/ Gingerline. (n.d.). What is immersive dining? Retrieved 12 7, 2020, from Gingerline:

https://www.gingerline.co.uk/what-is-immersive-dining In The Vintage Kitchen. (2018, 1 31). Dinner Time Stories: An Extra- ordinary Evening with Le Petit Chef. Retrieved from In The Vintage Kitchen: https://inthevintagekitchen.com/2018/01/31/dinner-time-stories-an-ext raordinary-evening-with-le-petit-chef/ Spence, C., & Piqueras-Fiszman , B. (2013, 3 11). Technology at the dining table.

Flavour. Sublimotion. (2014). Hard Rock Hotel Ibiza Sublimotion. Retrieved from hospitality-on.com/: https://hospitality-on.com/en/worldwide-hospitality-awards /hard-rock-hotel-ibiza/sublimotion teamLab. (n.d.). MoonFlower Sagaya Ginza. Retrieved from www.teamlab.art: http s://www.teamlab.art/zh-hant/e/sagaya/ Toyoshima , K. (2021, 9 16). Tree by Naked in Yoyogi is serving vegan lunch with

digital art. Retrieved 9 20, 2021, from Time Out: https://www.timeout.com/toky o/news/tree-by-naked-in-yoyogi-is-serving-vegan-lunch-with-digital-art-091621 Ultralytics. (n.d.). YOLOV5. Retrieved from YOLOV5: https://pytorch.org/hub/ultr alytics_yolov5/

Cover of the proceedings volume Affective and Pleasurable Design

Explore the complete digital volume

Affective and Pleasurable Design

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-17-9

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →