Megumi Umino, Ryota Yajima, Wonseok Yang (2024). Design of home appliance operation sounds based on the metaphorical nature of sound. In: Interdisciplinary Practice in Industrial Design. AHFE Open Access, vol. 144, pp. 157–167. https://doi.org/10.54941/ahfe1005128
Digitized products allow users to operate intuitively using visual Metaphor on a screen with a GUI, such as images and icons. Multimodal interaction, traditionally focused on visual information, is increasingly incorporating auditory elements like vibration and sound. This shift is driven by the growing prevalence of screenless devices and voice-activated operations. Unlike visual information, auditory information can be received in any direction. Therefore, Sound Interaction will continue to increase in various products used in daily life. However, the monotonous electronic sounds used in numerous products often have similar tones and no meaning. Therefore, users often cannot properly recognize the meaning or status of multiple products when using them, which leads to confusion in determining which function is activated in a product. We adapted the Metaphor technique, commonly used in graphical user interfaces (GUI), for auditory information. By employing a variety of real-world sounds in our design, we aim to provide intuitive auditory cues that enhance user interaction with home appliances. The purpose of this research is to see how the concept of “Sound Metaphoricity” can be utilized in the design of sound for home appliance operation with respect to human-object interaction. After selecting home appliances and organizing the main functions of the current products, abstract images associated with the product function names and outlines were extracted, and sensitivity evaluation experiments using sound and analysis were conducted. As a result, we were able to clarify the factorial characteristics of the sound that acts as a Metaphor in the design of operation sound.
With the development of artificial intelligence (AI) and ICT, home appliances used in our daily lives have become increasingly Internet of Things-oriented, and the concept of interaction between people and objects has shifted from “objects to be controlled” to “relationship formation.” In the interaction between people and objects, the forms of information expression can be divided into verbal and non-verbal. In addition, each type of information has its own meaning and can be divided into three categories, such as “under- standing the state of the system,” “understanding what the user should do,” and “understanding the use results”(Miyahara et al., 2017). This information
can be perceived through various sensory organs–mainly the visual, audi- tory, and tactile–depending on the type of object. In human interactions with objects, information representations can be divided into verbal and nonver- bal cases. In addition, each type of information has its own meaning and can be divided into three categories, such as “understanding the state of the sys- tem,” “understanding what the user should do,” and “understanding the use results” (Miyahara, 2017). This information can be perceived through vari- ous sensory organs–mainly the visual, auditory, and tactile–depending on the type of object. Traditionally, this has been achieved through the sense of sight; however, comprehending the meaning of an action or situation in places that the user or person cannot see is difficult. In contrast, compared to vision, hearing has no spatial constraints; therefore, information can be received even if the recipient is some distance away from the object or faces a com- pletely different direction (Shoji, 2016; Morimoto et al., 1998). Therefore, as home appliances become smarter and visual information becomes increas- ingly unavailable (Nishiyama, 2020), designing home appliances with a focus on interaction not only by visual information but also by auditory informa- tion is necessary. Currently, home appliances emit various types of sounds, such as “use sound,” which indicates that the device has been operated by the user, “operation sound,” which indicates that the device is working, and “function sound,” which informs the user about the operating contents and status of the device (Namba et al., 1995; Namba, 2000); however, no uni- fied design index has been defined for any of these sounds. Therefore, it has been suggested that some of the sounds used in reality may not fulfill the role of “conveying a message” and may convey incorrect information (Wake et al., 2003). In interface design that appeal to the sense of sight, the use of Metaphor that apply object and concept that exist in the real world to the design has enabled more reliable information transfer, such as increasing familiarity with new devices and facilitating the learning of device operations (Hosoya, 1995; Kusumi et al., Hosoya, 1995; Kusumi et al., 2002). In this research, we focus on the symbolic nature of sound (Akedo, 2013) and define “Sound Metaphoricity” as the ability of a sound to evoke a specific event in the listener and induce an emotion, and we will then discuss the possibil- ity of utilizing the Metaphorical nature of sound in the interaction design of auditory information.
LITERATURE REVIEW ON SOUND DESIGN AND METAPHOR A literature review was conducted to understand sound design methods that utilize the sensory aspects of sound, and also to understand how Metaphor have been utilized in the past when examining the Metaphorical nature of sound, respectively.
Sound Design Methods That Utilize the Sensory Aspects of Sound Yamauchi and Nomura (2016) examined the process of materializing the “EV-like sound,” a sound that is unknown and for which different people form different images. In their research, they attempted to understand the impression of “EV-like sound” by imagining a non-acoustic event (image
Design of Home Appliance Operation Sounds Based on the Metaphorical Nature of Sound 159
or situation) that they perceive as “EV-like,” administering a questionnaire survey asking about the impression they receive from the event, and collect- ing and analyzing the responses. The subjective evaluation experiment using adjective pairs used in this research can be widely adapted to the design of unknown sounds in general, because it is possible to interpret the results in terms of timbre factors. Other research have attempted to make soundscape design more systematic by confirming the correlation between impression evaluation words comprising adjectives and the perception of environmen- tal sounds by people in their daily lives (kawai et al., 2004). Kondo (2020) examined the process of embodying sounds from abstract concepts using fac- tors obtained through subjective evaluation experiments using the SD method and factor analysis as sound evaluation indices. From these results, it is evi- dent that the method of verbalizing and embodying abstract concepts using adjectives is useful for sound design.
The Role of Metaphor in Interfaces Metaphor are considered useful for improving the harmony between peo- ple and objects (Fujimoto and Chen, 1994). Metaphor in interfaces play an important role in two aspects: “to support users’ cognitive processes by designing tasks, tools, and operations in devices and systems” and “to pro- vide in manuals and education (Kusumi, 2002).” By utilizing Metaphor when designing an interface, the constraints that the interface imposes on the user as concerns expression and communication methods can be better aligned with those that the user already knows (Kinoe and Anzai, 1987). Previ- ously, when personal computers were first becoming popular, applying the deskwork Metaphor to users who were proficient in the deskwork domain but novices in the computer domain facilitated the use of the knowledge they already possessed and enabled them to learn to operate efficiently. The results of the literature review indicate that the application of the deskwork Metaphor to users who are proficient in the deskwork domain but novices in the computer domain facilitates the use of the knowledge they already have and enables them to learn to operate efficiently.
From these literature reviews, it was assumed that applying Metaphor to the operating sounds of home appliances would make it easier to operate them using sound cues, even if the functions were cumbersome. This research aims to use motion sound as a cue for the user to determine the optimal function by referring to the sound design methods used in previous research and utilizing other sounds in the real world in the design of motion sounds for home appliances.
RESEARCH METHOD In this research, we assumed the design of home appliances that are given artificial sounds during operation. First, home appliances were selected for the experiment, from which current products of the target home appliances were surveyed to extract the main functions and abstract images for each function. Next, based on the extracted functions and words, we selected several sounds to be used in the experiment and conducted a subjective
evaluation experiment using the sounds. Finally, we conducted sensitivity evaluation experiments and analysis using the sounds, and discussed the pos- sibility of utilizing the metaphorical nature of sound in the interaction design of auditory information.
Figure 1: Research flow.
Figure available in the original PDF paper.
RESEARCH ANALYSIS AND RESULT Selection of Appliances to be Used in the Experiment
In this research, we conducted an experiment on the interactions between the sounds emitted by home appliances and users. Therefore, we investi- gated a series of usage scenarios for four types of home appliances that emit “use sound,” “operation sound,” and “function sound” on a daily basis: a microwave oven, washing machine, air conditioner, and circulator. In this research, the “circulator,” which does not limit the scenes in which interac- tion occurs, is the subject of the experiment to verify the type of sound that can appropriately utilize Metaphoricity and induce the optimal interaction.
Survey the Current Products of the Target Home Appliance and Extract the Main Functions From the top 30 circulator models in the sales ranking (Kakaku.com, 2023), we investigated the main functions, focusing on the operation panel part. We focused on “air blowing mode,” which has a variety of functions, and excluded from the survey those functions such as “power,” “swing,” and “timer,” which are simple switching functions and only produce a sound sig- nal. As a result of the survey, the top five main functions were identified as “continuous,” “rhythm,” “clothes dryer,” “forced agitation,” and “sleep” (Table 1).
Table 1. Survey results of the main features of existing circulators (N = 30).
Table available in the original PDF paper.
Function Name Number Rhythm 15 Clothes Dryer 11 Sleep Mode 6 Forced Stirring 4 Warm Air 2 Circulation, Auto Airflow, Baby Mode, High Power Mode, Turbo Direct, Fan, Heater
Design of Home Appliance Operation Sounds Based on the Metaphorical Nature of Sound 161
Extract Words That Represent Images and Situations Evoked by the Function Because this survey examines the utilization of the Metaphorical nature of sound, deriving a specific sound that has a structural connection to one of its main function names is necessary. Therefore, to obtain the words recalled from the main functions, a questionnaire survey was administered to imag- ine the non-acoustic events felt from each of the extracted main function names and to ask about the impressions received from them, referring to pre- vious research. From the questionnaire, words representing the images and situations summarized in Table 2 were obtained for each function.
Table 2. Words representing images and situations extracted from function names (N = 10).
Table available in the original PDF paper.
Function name Overview of Functions Image/Situation Consecutive Operates at the air volume set using the air volume button
Comfortable, Nature, Wave, Regular
Constant, Monotonic, Automatic, Mechanical Rhythm Reproduce a wind that is close to nature by changing the strength. It sends out natural wind by changing the air volume at a certain rhythm.
Sun, Hot Air, Dry, Drying
Clothes Dryer The strength of the wind is adjusted depending on the direction, so that the speed of the wind hitting the laundry remains almost constant. Dry clothes evenly even when drying indoors The clothes drying mode automatically operates with the air volume and angle suitable for drying programmed. The machine automatically swings its head up, down, left and right, allowing air to blow directly onto the laundry from various angles, allowing it to dry quickly.
Forced Stirring Stir the air in the room to reduce temperature differences. Stirring means stirring.
Quiet, Wind chimes, Quiet sounds at night, Owl
Strong Wind, Typhoon, Mixing, Turning Sleep Mode The wind is light in the front range, becomes strong in the upward direction, and gradually changes in strength in between. Direct wind from the front is gentle, and when facing upward, it can be applied to a wall to create indirect wind or create convection of the air in the room. The wind is gradually reduced over time.
Select Sounds to be Used in the Experiment Based on the Extracted Functions and Words A table of images and situations (Table 2) extracted from the names of the air-blowing functions was used to select the sounds for later experiments. To examine the influence of the Metaphoric nature of sound, the sounds used in this research are based on objects and concepts that exist in the real world, and are expected to have a structural connection with the circulator, such as “natural sound” and “environmental sound” categories. To ensure objectivity in the selection of sound samples, we asked five students to view Table 2 and thereafter select a sound sample from among the multiple “natu- ral sound” and “environmental sound” samples that were considered to have strong associations with each word. Consequently, 16 sounds were selected, as shown in Table 3. The length of all sounds was set to 10 s, the volume level was standardized to 50 dB, and mp3-standard sound samples were prepared for use in the experiment.
Table 3. 16 selected sounds.
Table available in the original PDF paper.
Draft Strong wind Wind chime Stormy rain Stream Sugarcane field Bamboo forest Rice field Bird (lark) Frog Cicada Noise Bell cricket Owl Howler Bonfire
Subjective Evaluation Experiment Using Selected Sounds Aiming to acquire factors that can be used as indicators when design- ing a sound that can evoke each function of a circulator, a subjective sound evaluation experiment was conducted on 30 male and female in their early 20s. In parallel with the subjective evaluation, the degree of confor- mity between the audible sound and functional image was also evaluated. Research participants were asked to listen to 15 pairs of adjectives, selected with reference to previous research (Kondo and Yamanaka, 1998; Inoue and Kobayashi, 1985; Kuwata, Hochin and Nomiya, 2010), in the experimen- tal environment shown in Figure 2, in a room that was sufficiently quiet compared to the sound sample. The following procedures were used in the experiment:
1) Research participant listen to some sound. 2) Research participant responded to their impression of the sound on a 5-point scale from −2 to 2 on the evaluation sheet. 3) Research participant checks any matches with a specific functional image on the sheet related to the congruency verification. 4) The above procedure is repeated a total of 16 times with different samples and in random playback order.
Design of Home Appliance Operation Sounds Based on the Metaphorical Nature of Sound 163
Figure 2: Experiment environment.
Using all the data from the sensibility evaluation sheets obtained through the experiment, the maximum likelihood method was applied to extract the three factors, and factor analysis was performed by applying varimax rota- tion to the three factors. The factor-loadings matrix is shown in Table 4. Focusing on factor loadings with absolute values of 0.4 or higher, the following interpretations were made:
The contribution rate of Factor 1 was approximately 44.6%, which is con- sidered an important factor accounting for approximately half of the total, indicating that characteristics related to cleanliness and transparency have an important influence on the operating noise of air-conditioning appliances.
Table 4. Factor analysis of 15 pairs of adjectives.
Based on the results of the above factor analysis, a further cluster anal- ysis using Ward’s method was conducted, and the 16 sound samples were
classified into three groups as shown in Figure 3. This indicated that spe- cific function names and their images could be explained by the two factors obtained from the subjective evaluation.
Figure 3: MAP based on analysis results.
Experiments to Verify the Usefulness of Sound Metaphoricity An experiment was conducted to verify the validity of the factors obtained from the subjective sound evaluation experiment and analysis as well as to verify the influence of the Metaphoric nature of sound. This time, we focused on the “sleep” function among the circulator functions and performed the verification. For validation, we used 8 sound samples–2 of each of 4 types for each factor feature–from the sound samples used in the sensitivity eval- uation experiment (Table 5). Two sounds with high intelligibility factor and low intensity factor (i.e., factorial features with high “good night” function congruency) will be designated as Group A, and the other 6 sound samples as Group B.
Table 5. Types of sounds used in verification.
Table available in the original PDF paper.
Sound Sample Clarity Factor Impressive Factor Bird (lark) Stream High High Bell cricket Wind chime High Low Stormy rain Howler Low High Draft Strong wind Low Low
Design of Home Appliance Operation Sounds Based on the Metaphorical Nature of Sound 165
The experiment was conducted on 7 students using the following procedure.
1) Explain to research participants the purpose of this research and the classification of circulator functions. 2) Have research participants listen to one sound and rate it on a 5-point scale from −2 to 2 as to whether it is appropriate as a sound to be given to the “good night” function. 3) The above procedure was repeated a total of eight times.
A 2 sample T-test with no correspondence was conducted on the data obtained from the experiment (Table 6). The results show that there is a sta- tistically significant difference between the means of Group A and Group B, since p<0.05, suggesting that the mean of Group A is larger than that of Group B. This indicates that the interpretation of the factors obtained in the sensitivity evaluation experiment is valid, and that Group A is more appropriate as the operating sound for the “good night” function.
Table 6. Results of T-test.
Table available in the original PDF paper.
Variable 1 Variable 2 Mean 2.571429 1.571429 Variance 0.369048 0.59127 Observations 7 7 Pooled Variance 0.480159 Hypothesized Mean Difference 0 Degree of freedom 12 t 2.699862 P(T<=t) One-side 0.009657 t Critical One-side 1.782288 P(T<=t) Two-side 0.019314 t Critical Two-side 2.178813
CONCLUSION
In this research, we examined the possibility of utilizing the Metaphorical nature of sound in the design of operating sounds for home appliances that target circulators. First, after organizing the main functions of existing circu- lators, images for each function were extracted using words, and subjective evaluation and verification experiments were conducted using sounds that evoke objects and concepts that exist in the real world. From the analy- sis of the experimental results, two factors, “clarity” and “power,” were obtained. Additionally, a parallel research was conducted on the correspon- dence between words representing images and situations extracted from the names and outlines of each function. Consequently, among the main func- tions, “rhythm” exhibited no correlation or common relationship among the selected samples, while “drying clothes” was a sample sound that was “not powerful and not clear” and “forced agitation” had “powerful and unclear”
sample sounds, and “good night” had “not powerful and slightly clear” sam- ple sounds in common. This indicates that a particular function name and its image may be explained by a factor obtained through factor analysis using the results of the subjective evaluation of the sound. Furthermore, from an experiment to verify the validity of the obtained factors, the T-test based on the results of the verification experiment significantly revealed that the factor characteristic of the sound appropriate for the “good night” function was “less powerful and slightly clear,” indicating that the interpretation of the factors was also valid. Therefore, it was assumed that by clarifying the type of sound that acts as a Metaphor for a product, based on the relationship between the associative image of the product and the factorial characteris- tics of the sound, it would be possible to design an auditory information interaction appropriate for the product and its function. However, because personal experiences and preferences significantly influence the establish- ment of Metaphor, we should consider conducting evaluation experiments using a wider range of adjective pairs that consider these factors. In addition, the validity of the factors obtained should be verified for all functions in addi- tion to the “good night” function. We predict that future experiments with larger sample sizes, focusing on home appliances in addition to circulators, and exploring the Metaphorical nature of sound in more detail will provide a method for designing sounds for a wider variety of products.
REFERENCES
Akedo Shin-ya (2013). Application of the Soundscape Design for the Auditory Signal.
Japanese Society for Science of Design, 60(4): 35–44. Fujimoto Hideo and Chen Lian-Yi (1994). Composite Effects and Application of Metaphor in Human Interface. Transactions of the Japan society of mechanical engineers. C., 60(580): 380–385. Hosoya Tamon (1995). Auditory Sounds and Product User Interface. Special issue
of Japanese Society for the Science of Design, 3(2): 53–58. Inoue Massaki and Kobayashi Toshinobu (1985). The Research Domain and Scale Construction of Adjective-Pairs in a Semantic Differential Method in Japan. The Japanese journal of educational psychology, 33 (3): 253–260. Kakaku.com, Fan/Circulator Popular Selling Ranking, https://kakaku.com/kaden/fan/, 10.2023. Kawai Kenji, Kojima Takaya, Hirate Kotaroh and Yasuoka Masahito (2004). Per-
sonal psychological evaluation structure of environmental sounds: Experiments of subjective evaluation using research participants’ own terms. The journal of the acoustical society of japan, 60(5): 249–257. Kinoe Yousuke and Anzai Yuichiro (1987). User Interface and Cognitive Models.
Journal of the Japanese Society for Artificial Intelligence, 2(2): 141–149. Kondo Masao and Yamanaka Toshimasa (1998). The Characteristics of Inevitable Sound from Products by the Kansei Evaluation. Proceedings of the Annual Conference of JSSD, 45 (0): 316–317. Kondo Takashi, Hara Seiji, Sakaguchi Motoyasu, Torii Kenji, Tsuruta Tatsunori,
Yosioka Koichi and Miyazawa Masaya (2020). Engine Sound Design by Kansei Engineering - Case Research of “Woody sound” Design for New Engine Sound. Transactions of Japan Society of Kansei Engineering, 19 (1): 1–10.
Design of Home Appliance Operation Sounds Based on the Metaphorical Nature of Sound 167
Kusumi Takashi (2002). The Role of Metaphor in User Interface Design : Going from the Desktop to VirtualSpace and Returningte Language. SPECIAL ISSUE OFJSSD, 10(1): 64–73. Kuwata Kazuya, Hochin Teruhisa and Nomiya Hiroki (2010). Research on the Esti-
mation of the Semantic Harmony of Sound and Video Clips in the Audiovisual Material. Transactions of Japan Society of Kansei Engineering, 9 (3): 493–501. Miyahara Ai, Arai Hidefumi, Kawahara Kenta, Yamazaki Yuka, Nakanishi Miwa (2017). Development of a user experience evaluation method applicable to the design process of home appliances. Bulletin of Japanese Society for the Science of Design, 64 (3), 3_61-3_70. Morimoto Kazunari, Nishimura Takeshi, Kurokawa Takao and Oshibe Naokatsu (
1998). Effects on FeeHngs of Computer OperationAccording to Sound of a Sound Icon. The journal of Japanese Society for Science of Design, 45(1): 39–46. Namba Seiji (2000). Improvement of Recognizability of Home Appliance Function Sounds. Toshiba review, 55(7): 63–67. Namba Seiji, Sasaki Akira and Hosoito Tsuyoshi (1995). Research of Sound Signal
Interface. Toshiba review, 50(6): 471–474. Nishiyama Takashi (2020). The Present and Future of Smart Home Technology - Including the SoS Perspective-. journal of the Society of Instrument and Control Engineers, 59(12): 957–960. Shoji Hiroko (2016). Functional Aspects and Affective Aspects of Auditory Sig-
nals for Home Appliances. The Journal of the Institute of Electrical Installation Engineers of Japan, 36 (2): 88–91. Wake Sanae H., Okada Yoshihiko and Asahi Toshiyuki (2003). Designing AuditorY Signalson Human lnterface. Bulletin of Japanese Society for the Science of Design, 49 (5): 41–50. Yamauchi Katsuya and Nomura Takuya (2016). Developing Method Describing
Impression of Unknown Sounds through Non-acoustic Events. Journal of Society of Automotive Engineers of Japan, 70 (7): 61–66. Yamauchi Katsuya, Takada Masayuki and Iwamiya Shinichiro (2003). Functional imagery and onomatopoeic representation of auditory signals. The journal of the acoustical society of japan, 59(4): 192–202.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Interdisciplinary Practice in Industrial Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-20-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional