Kana Uenoyama, Wonseok Yang (2026). Interaction Design Using Physical Properties. In: Interdisciplinary Practice in Industrial Design. AHFE Open Access, vol. 222, pp. 100–110. https://doi.org/10.54941/ahfe1007750
In product design, controlling surface texture through the three CMF (Color, Material, Finish) elements is emphasized as a method for achieving differentiation and enhancing sensory value. Similarly, in GUI design, there is a movement to visually reproduce material qualities, such as the effect produced using glassmorphism, while striving to balance aesthetics and legibility. However, most efforts remain limited to reproducing "appearance," with insufficient exploration of expressions rooted in the physical properties inherent to real-world materials—such as weight, friction, and hardness/softness. Movements and reactions based on physical properties are intuitively understandable, which makes it easier to connect state changes and cause-and-effect relationships to bodily sensations. Therefore, this research explored the potential of interaction expressions based on physical properties. A card-sorting task was conducted with 30 students using 30 material cards (displaying material names and photos) and 9 touch-interaction cards. The participants selected materials matching each interaction and provided reasons for their choices. Beyond tallying selection rates, reasons were labeled based on prior research to compare differences across interactions. The results indicate that frequently referenced material properties varied by interaction, with the input temporal structure and motion characteristics likely being the primary switching factors. For instant inputs, such as taps, immediate responsiveness and deformation were emphasized, whereas for sustained inputs, such as long taps, holding, and state changes over time were referenced. For continuous movement actions such as dragging, swiping, and scrolling, roughness and friction were associated as control cues. For two-point operations such as pinching and stretching, continuum deformation and tension were evoked. For rotation, changes in friction resistance were relatively important.
As product shapes and functions become increasingly homogeneous, product design increasingly emphasizes techniques that leverage CMF to control surface textures, aiming to achieve differentiation and enhance the Kansei value (Tamai, 2023). Similarly, in GUI design, approaches such as glassmorphism seek to recreate specific material textures visually. By ensuring legibility while imparting emotional impressions, such as transparency, they attempt to balance usability and aesthetics. However, this reinterpretation of materiality has primarily remained confined to “visual” expression, with expressions based on the physical properties of real-world materials—such Received February 19, 2026; Revised April 6, 2026; Accepted April 26, 2026; Available online July 20, 2026
Interaction Design Using Physical Properties 101 as weight, friction, and hardness/softness—still insufficiently explored. Therefore, this research focused on the physical properties inherent in real- world objects. Movements and reactions based on physical properties are intuitively understandable, making it easier to connect state changes and causal relationships with bodily sensations (Ishii and Mizutani, 2011). Therefore, the employment of physically grounded expressions, such as visual and dynamic behaviors in GUI, contributes to the understanding and prediction of interactions. Asano et al. pointed out that material and surface textures are as, if not more, important than color and shape in forming impressions (Asano et al., 2010). Furthermore, Masui et al. demonstrated that sliders using the elasticity of rubber bands as a metaphor can facilitate fine-tuning values and the selection of items even within a limited screen space, showing how physical properties can be leveraged in interaction design (Masui et al., 1995). Furthermore, Apple’s Liquid Glass achieves interactions that convey the material’s physical properties by combining realistic glass rendering with fluid deformation in response to user actions (Apple, 2025). Building on these prior studies and UI design trends, this research explored the potential of interaction expressions based on physical properties.
Potential of CMF Expression in Digital Environments With a Focus on Texture As seen in recent examples such as Liquid Glass, UI integrating optical expression with dynamic deformation in response to user actions has emerged, treating materiality as a central element of the UI experience (Apple, 2025). This represents a movement to reorganize the value creation traditionally handled by CMF in the product domain, primarily as “Material and Finish” design within software. It can be understood as the “CMF-ization of digital” technology. This trend aligns with insights from existing interface research. Several studies have demonstrated that visual quality influences the perceived usability and satisfaction. The relationship between beauty and perceived usability (Tractinsky et al., 2000) and multifaceted metrics for measuring the visual beauty of the web (Moshagen and Thielsch, 2010) substantiate that design elements affecting “appearance”—including color and finish—can directly impact experiential value. Furthermore, comparative studies of UI stylistic approaches have examined how differences between skeuomorphism and flat designs can affect exploration efficiency and cognitive processes (Gross et al., 2014) (Burmistrov et al., 2015) (Spiliotopoulos et al., 2018). However, these findings have primarily focused on the evaluation of appearance and visual cues. Therefore, this research focuses on “Material and Finish,” which are considered to significantly influence operational understanding, and explores the potential for interaction expressions rooted in physical properties.
Textural Perception as a Property Estimation Based on Kinetic Cues Texture perception research has shown that people can infer not only material categories from images but also physical and functional properties such as hardness, roughness, and wetness. Fleming organized material perception
102 Uenoyama and Yang as “inference about diverse and variable objects,” arguing that vision infers materiality and properties from cues like reflectance properties (see Figure 1) and microstructure (Fleming, 2017). It is known that the perception of glossiness depends on multiple cues, such as highlight shape, contrast, and lighting conditions (Chadwick and Kentridge, 2015). Furthermore, recent years have clarified that mechanical properties are estimated not only from the static appearance but also from temporal cues during deformation and motion. For example, the integration of shape, motion, and optical cues has been shown to enable the estimation of an unknown object’s rigidity (Schmidt et al., 2017). For fluids, viscosity is said to be constantly estimated from shape and motion features (van Assen et al., 2018). Reports also indicate that the deformation speed and amount during pressing contribute to softness judgments (Ujitoko and Kawabe, 2022). Attempts have been made to reproduce viscosity perception using deep learning models and link it to explainable features (van Assen et al., 2020). These findings indicate that cues for texture may be distributed not only in “appearance” but also in “motion,” suggesting scope for designing dynamic GUI behaviors as cues for physical properties. Specifically, temporal characteristics such as velocity, delay, and damping can contribute to the estimation of physical properties, thereby making the manipulation of dynamic parameters a promising research avenue.
Figure 1: Schematic diagram of reflective properties (Fleming, 2017).
Predictable Interaction Representation Using Physical Properties In interaction research, direct manipulation has been understood as a design principle for “continuously manipulating visible objects and receiving immediate feedback,” forming the foundation for understanding and a sense of control (Shneiderman, 1983). Within this lineage, precision manipulation interfaces utilizing the elasticity of rubber bands have been proposed to embed metaphors of physical properties into GUI components (Masui et al., 1995). Furthermore, animated transitions provide cognitive cues that support an understanding of state changes (Heer and Robertson, 2007). However, it has also been reported that motion in UI designs using shape changes can lead to misunderstandings of affordances or system states (Tiab and Hornbæk, 2016), necessitating that the design of dynamic behavior ensures “semantic consistency.” Thus, while dynamic behaviors rooted in physical properties are effective as cues for understanding state changes, systematic knowledge of
Interaction Design Using Physical Properties 103 the parameters that aid in expressing friction, inertia, and elasticity, as well as the interactions for which they are effective, remains insufficient. Therefore, this research exploratively examined the correspondence between physical property representations and their potential effectiveness in representative interactions, presenting foundational perspectives for design.
RESEARCH METHOD In this research, we conducted research using the following procedure to explore the potential for interaction expressions based on the physical properties inherent in materials and textures. (see Figure 2)
Figure 2: Research flow.
Literature Review for Selecting Materials and Substances to Be Used in Experiments In selecting materials and textures for the experiments, we conducted a review of prior research that dealt with a balanced range of materials across categories and organized texture perception dimensions. Notably, Nagano et al., surveyed multiple studies on research trends concerning the composition of texture perception dimensions for tactile textures and provided a comprehensive overview of these findings. This integrated analysis concluded that five fundamental material qualities exist regarding the physical properties of materials: “Fine roughness (Rough/Smooth),” “Macro roughness (Uneven, Bulky),” “Hardness (Hard/Soft),” “Warmness (Warm/Cold),” and “Friction (Moist/Dry, Sticky/Slippery)” (Nagano et al., 2011). In this research, using these five dimensions as a guide, we selected 30 types of materials and textures found in nature and everyday life (see Figure 3) and conducted experiments.
Figure 3: The 30 material samples used in the experiment.
104 Uenoyama and Yang An Exploratory Experiment to Gather Ideas for Applying Physical Properties to Interaction Design
This experiment aimed to explore and collect ideas on how the physical properties of materials can be utilized in interactions. The participants were 30 undergraduate and graduate students majoring in design. The method employed card sorting, and stimuli included 30 material cards (see Figure 3) labeled with “material name + photo” and 9 interaction cards (see Figure 4). These interactions, which are used in mobile interfaces, referenced four design guidelines: Google Material Design, Apple Human Interface Guidelines, Microsoft’s Touch Input Guide, and SAP Design System.
Figure 4: The 9 interactions used in the experiment.
Figure available in the original PDF paper.
In the experiment, photographs served as examples to represent materials, and the participants were instructed to focus on the general properties of the materials themselves rather than the photographs. The number of materials selected for each interaction was not restricted. Participants were asked to select “materials applicable to each interaction” from the material cards and articulate “which properties they felt could be applied” as their rationale, thereby extracting material properties.
Results of the Experiment
1) The Selection Rate for Materials First, we calculated the selection rate for materials deemed “likely to fit” in each interaction. The selection rate is the number of material selections divided by the total number of participants. The top three materials are shown in Table 1. Long taps tended to result in selections of clay and sponge, while pinches showed a tendency toward fabric-based materials like silk and cotton.
Table 1: Top 3 material selection rates for each interaction.
Table available in the original PDF paper.
TOP 1 TOP 2 TOP 3
Tap Water (63.3%)
Sponge (53.3%)
Clay (50.0%)
double Tap Water (60.0%)
Honey (36.7%)
Olive oil (33.3%)
long Tap Clay (73.3%)
Sponge (60.0%)
Honey / Cork (40.0%)
drag Silk (53.3%)
Cotton / Leather (43.3%)
Sand (40.0%)
swipe Newspaper (53.3%)
Silk (46.7%)
Sand (43.3%) (Continued)
Table 1: Continued.
Table available in the original PDF paper.
TOP 1 TOP 2 TOP 3
scroll Silk (40.0%)
Glass / Stainless / Wrap (36.7%)
Olive oil / Marbie (33.3%)
pinch Silk (76.7%)
Cotton (63.3%)
Leather (46.7%)
stretch Silk (56.7%)
Rubber band (53.3%)
Cotton (46.7%)
rotate Silk (46.7%)
Cotton (36.7%)
Pottery (33.3%)
2) The Dispersion and Convergence Tendencies in Material Next, to understand the dispersion and convergence tendencies in material selection, we calculated the frequency distribution of selection rates (see Table 2), average number of selections, total selection rate of the top three materials, difference between the first and second places, and concentration (strength of the overall material bias) (see Table 3). The results revealed that long taps, pinches, and stretches—which exhibited high concentration and occupancy rates for the top three materials— showed strong convergence toward a few materials. Conversely, drags— which have a high average number of selections but low concentration and occupancy rates for the top three materials—tended to be more dispersed. Furthermore, while many interactions showed a tendency for numerous materials to be distributed in the low-selection-rate range—with a few top materials exhibiting high selection rates—the distribution shape differed across interactions. For example, in the action of scrolling, the prominence of the top materials was smaller, and the mid-range materials tended to cluster.
Table 2: Frequency distribution of selection rates.
Table 3: A average number of selections, total selection rate of the top three materials, difference between the first and second places, and concentration.
Table available in the original PDF paper.
Concentration
Interaction Average Number of Selections
Total Selection Rate of the Top Three Materials
Difference Between the First and Second Places Tap 6.1 1.67 0.10 0.09
Double Tap 5.5 1.30 0.23 0.05 Long Tap 5.2 1.73 0.13 0.13
Consideration of Which Physical Properties are Effective Referencing findings from prior studies (Nagano et al., 2011) (Motoyoshi, 2012), we categorized participants’ stated reasons for material selection into 17 labels. The results indicate that frequently referenced material properties varied by interaction, with the input temporal structure (Instantaneous or continuous) and motion characteristics (continuous motion, two-point manipulation, etc.) likely being the primary switching factors. The following sections present considerations for each interaction type.
Tap Water (63.3%), sponge (53.3%), and clay (50.0%) exhibited the highest selection rates. Labeling results for selection reasons showed frequent mentions of “shape/state change” and “responsiveness.” Water, which had the highest selection rate, may have been influenced by its ability to evoke immediate reactions, such as ripples or fluctuations, to short-duration inputs. Both sponge and clay deform or are left with marks when pressed, clearly indicating “the tap was pressed.” Thus, responsiveness and instantaneous state changes serve as primary cues for taps, indicating that expressions emphasizing immediate reactions as material cues are effective.
Double Tap Water (60.0%) ranked the highest, followed by honey (36.7%) and olive oil (33.3%). Labeling results for selection reasons showed frequent mentions of “shape/state change” and “responsiveness.” We considered that the top ranking of water is related to how its response—arising from two inputs— is easily perceived as a continuous ripple pattern. Furthermore, viscous materials such as honey and olive oil may readily evoke the concept of
Interaction Design Using Physical Properties 107 “residual effect”—where the reaction persists when input intervals are short. Therefore, we concluded that in double-tap interactions, beyond single, immediate responses, temporal overlap and the persistence of reactions serve as effective clues for material properties.
Long Tap Clay (73.3%) and sponge (60.0%) took second place, followed by cork and honey (40.0%) tied for third. Additionally, the labeling results for selection reasons showed a particularly high number of descriptions related to “shape/ state changes.” The concentration of top selections for clay and sponge was considered to be due to the sustained press operation requiring “holding” or “continuation of state over time.” The clay was retained its pressed shape visually, whereas the sponge maintained its indented state. Honey was considered an effective metaphor for time-dependent input, evoking slow deformation owing to its viscosity. For long taps, temporal physical properties— such as transitions accompanying sustained input, delayed changes, or states evolving with prolonged contact—may reinforce the meaning of the operation more effectively than instantaneous responses.
Drag The silk (53.3%), cotton, leather (43.3%), and sand (40.0%) contents were the highest. Labeling results for selection reasons showed particularly frequent descriptions related to “shape/state changes,” followed by descriptions related to “roughness/smoothness.” These materials readily evoke changes in tactile sensation during continuous operations such as friction, compliance, and deflection, suggesting that the operation of “pressing and moving” was recalled. Conversely, dragging had the highest average selection count (7.7 items and lowest concentration (0.019). This suggests that dragging is a multifaceted operation that is easily substituted by multiple physical property cues, rather than converging on specific materials.
Swipe Newspapers (53.3%), silk (46.7%), and sand (43.3%) ranked highest. Labeling results for selection reasons showed numerous descriptions related to “roughness/smoothness,” followed by “shape/state change” and “associations from actions.” The reasons cited for selecting newspapers and silk were that they readily evoke metaphors for everyday actions, such as brushing or flipping. Furthermore, sand suggested that the motion of particles moving when brushed could be easily recalled, potentially aligning with the “brushing in one direction” action of swiping. Therefore, it was considered that, during swiping, the movement of materials consistent with the operational image and the resistance derived from friction are likely to be referenced as cues.
Silk (40.0%), glass, stainless steel, wrap (36.7%), olive oil, and marble (33.3%) were ranked the highest. Labeling results for selection reasons showed numerous descriptions related to “roughness/smoothness,” followed by descriptions related to “associations from operation” and “friction sensation.” The low prominence of the top materials and the lowest combined selection rate of 1.133 for the top three materials suggest that scrolling is difficult to converge on a single material property. However, the reasons for selection included descriptions such as glass and stainless steel having smooth surfaces, wrap providing a sense of friction, and silk conveying fluidity. This led to the interpretation that scrolling is an operation supported by multiple cues enabling “continuous movement.” Consequently, it was concluded that smoothness during continuous movement and resistance changes that affect the ease of stopping are critical factors for scrolling.
Pinch Silk (76.7%), cotton (63.3%), and leather (46.7%) ranked the highest. The combined selection rate for the top three materials was 1.867, which was the highest value, and the concentration was also high at 0.131. This suggests a relatively strong convergence toward a few materials, making material metaphors easier to clarify. Additionally, labeling results for selection reasons showed numerous descriptions related to “shape/state change,” followed by descriptions related to “associations from actions.” This result suggests that the action of pinching is an operation centered on two points: “pinching” and “bringing together,” making it easy to evoke the behavior of wrinkles forming on the surface of relatively soft materials such as cloth or leather. Therefore, a pinch can be considered an interaction expression rooted in material properties through the representation of material deformation and bringing together.
Stretch Silk (56.7%), rubber bands (53.3%), and cotton (46.7%) ranked highest, with a relatively high concentration of 0.128. Furthermore, labeling results for selection reasons showed numerous descriptions related to “shape/ state change.” The reason rubber bands ranked high was likely because their direct physical property of elongation was used as a metaphor for the stretch operation. Furthermore, the simultaneous selection of fabric-based materials such as silk and cotton suggested that the behavior of tension arising from pulling contributed to the understanding of the expansion operation. Therefore, it was concluded that, in stretch operations, resistance changes conveying elongation and tension, along with expressions of deformation accompanying expansion, are effective interpretations of material properties.
Interaction Design Using Physical Properties 109 Rotation
Silk (46.7%), cotton (36.7%), and ceramic (33.3%) were ranked the highest. Labeling results for selection reasons showed numerous descriptions related to “shape/state changes,” followed by descriptions related to “friction sensation.” As rotation involves turning around two points, we considered both slipperiness and changes in resistance to be simultaneously important. The prominence of fabric suggests recall of slippage and compliance during insertion. The high ranking of ceramic materials indicates recall of the rotational action itself, along with the tactile feedback and frictional resistance experienced when rotating hard surfaces. Therefore, for rotation, incorporating friction-derived cues—such as resistance, which increases or decreases with the rotation amount or gradual catching—into the operational feedback is effective.
CONCLUSION
This research suggests that physical cues are not uniform but tend to vary depending on the temporal structure and motion characteristics of the input. For instantaneous inputs such as taps and double taps, immediate state changes such as ripples and deformations, along with responsiveness, serve as primary cues. For sustained inputs, such as long taps, holding actions, time- dependent transitions, and delayed changes were more readily referenced. For continuous movements, such as dragging, swiping, and scrolling, the resistance derived from roughness or friction was found to be readily recalled as a control cue. For two-point operations such as pinching and stretching, deformation and tension were strongly evoked, suggesting that material metaphors become relatively clearer. This aligns with the observation that movements and reactions based on physical properties support an understanding of state changes and causality (Ishii and Mizutani, 2011). This indicates that assigning appropriate material behaviors according to the characteristics of the operation can aid in understanding and predicting interactions. The “effective physical property cues for each manipulation characteristic” identified in this research are considered fundamental guidelines for designing expressions rooted in physical properties.
REFERENCES
Apple. (June 9, 2025) Apple introduces a delightful and elegant new software design.
Apple Newsroom (Press Release): https://www.apple.com/newsroom/2025/06/ apple-introduces-a-delightful-and-elegant-new-software-design/ Asano, C.M., Asano, A., Li, L., Fujimoto, T. (2010) Morphological Texture Manipulation for The Evaluation of Human Visual Sensibility. Kansei Engineering International Journal, 10(1), pp.11–18. Burmistrov, I., Zlokazova, T., Izmalkova, A., Leonova, A. (2015). Flat Design vs
Traditional Design: Comparative Experimental Study. In J. Abascal, S. Barbosa, M. Fetter, T. Gross, P. Palanque, M. Winckler (Eds.), Human-Computer Interaction – INTERACT 2015: 15th IFIP TC 13 International Conference, Bamberg, Germany, September 14–18, 2015, Proceedings, Part II. Lecture Notes in Computer Science, vol. 9297, pp.106–114.
110 Uenoyama and Yang Chadwick, A.C., Kentridge, R.W. (2015) The perception of gloss: A review. Vision
Research, 109(Part B), pp. 221–235. Fleming, R.W. (2017) Material Perception. Annual Review of Vision Science, 3, pp.
365–388. Gross, S., Bardzell, J., Bardzell, S. (2014). Skeu the Evolution: Skeuomorphs, Style, and the Material of Tangible Interactions. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI ’14), Munich, Germany, 16–19 February 2014. ACM, pp. 53–60. Heer, J., Robertson, G. (2007) Animated Transitions in Statistical Data Graphics.
IEEE Transactions on Visualization and Computer Graphics (Proc. InfoVis), 13(6), pp.1240–1247. Ishii, M., Mizutani, G. (2011) Intuitive GUI design using viscous liquid representation.
In: Proceedings of the Annual Conference of the Japanese Society for the Science of Design (JSSD), 58th, pp. 304–305. Masui, T., Kashiwagi, K., Borden IV, G.R. (1995) Elastic Graphical Interfaces for Precise Data Manipulation. In: Katz, I., Mack, R., Marks, L. (Eds.), CHI 1995 - Conference Companion on Human Factors in Computing Systems. Association for Computing Machinery, pp.143–144. Moshagen, M., Thielsch, M. T. (2010). Facets of visual aesthetics. International
Journal of Human-Computer Studies, 68(10), pp. 689–709. Motoyoshi, I. (2012) Perceptual mechanisms of visual material perception. Journal of the Institute of Image Information and Television Engineers, 66(5), pp.338–342. Nagano, H., Okamoto, S., Yamada, Y. (2011) Research Direction in Structuring
of Tactile Dimensions of Material Textures. Transactions of the Virtual Reality Society of Japan, 16(3), pp. 343–353. Schmidt, F., Paulun, V.C., van Assen, J.J.R., Fleming, R.W. (2017) Inferring the stiffness of unfamiliar objects from optical, shape, and motion cues. Journal of Vision, 17(3):18, pp.1–17. Shneiderman, B. (1983) Direct Manipulation: A Step Beyond Programming
Languages. Computer, 16(8), pp. 57–69. Spiliotopoulos, K., Rigou, M., Sirmakessis, S. (2018). A Comparative Study of Skeuomorphic and Flat Design from a UX Perspective. Multimodal Technologies and Interaction, 2(2), 31. Tamai, M. (2023) CMF (Color Material Finish) Design. Hyomen Gijutsu, 74(11),
pp. 549–556. Tiab, J.S.S., Hornbæk, K. (2016) Understanding affordance, system state, and feedback in shape-changing buttons. In: Proceedings of the 34th Annual CHI Conference on Human Factors in Computing Systems (CHI 2016). Association for Computing Machinery, pp. 2752–2763. Tractinsky, N., Katz, A. S., Ikar, D. (2000). What is beautiful is usable. Interacting
with Computers, 13(2), 127–145. Ujitoko, Y., Kawabe, T. (2022) Perceptual judgments for the softness of materials under indentation. Scientific Reports, 12, Article 1761. van Assen, J.J.R., Barla, P., Fleming, R.W. (2018) Visual Features in the Perception of
Liquids. Current Biology, 28(3), pp. 452–458. van Assen, J.J.R., Nishida, S., Fleming, R.W. (2020) Visual perception of liquids: Insights from deep neural networks. PLoS Computational Biology, 16(8).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Interdisciplinary Practice in Industrial Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-98-4
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional