Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Neural decision-making application of Internet product interaction design based on EEG

1 Zhejiang Business Technology Institute, No. 1988, Jichang Road, Haishu District, Ningbo 315012, Zhejiang, China ↗
2 Yiwu Industrial and Commercial College, No. 2, Xueyuan Road, Yiwu 322000, Zhejiang, China ↗
Publication year
Digital Object Identifier10.54941/ahfe1002755

Abstract

The rapid development of information technology and the rapid growth of "Internet +" have brought many new challenges to human factors science. At present, in the practice of information product design that emphasizes user experience, new human factors measurement methods and tools are urgently needed to evaluate the effect of product interaction design more accurately and objectively, and to provide theoretical support and basis for product design and decision-making. Different from traditional human factors research methods, neural measurement overcomes the subjective bias of questionnaires, scales and other methods, and can dynamically grasp the user's cognitive process of products with millisecond precision in the process of users' cognition of products. Design practice provides the basis and guidance for neurological decision-making. On the basis of the comprehensive evaluation of the neural mechanism, neural decision-making and emotional experience in the product interaction design, an interaction design scheme in line with neurocognitive science is proposed. The established indicator threshold ranges are compared to determine the range of user experience; longitudinal comparison refers to comparing with the plan before design intervention to test whether the optimization effect is significant. With the help of neural evaluation indicators, the design scheme is finally determined.Misoperations during the use of Internet products are usually caused by factors such as inattention and identification errors. Therefore, it is very important to study the influence of various design elements on the components of the attentional neural response, including latency, amplitude, and response position. Focus on two cognitive neural components N2 and LPP, N2 is a cognitive neural response component of early attention, LPP is a neural component related to attention after full cognition, and reflects the user's emotional changes, which is an important part of emotional experience. Cognitive Neural Indicators. Existing studies have shown that product decision-making is related to the P300 component in cognitive nerves. In interaction design, the P300 amplitude corresponding to easily recognizable icons is greater than the P300 amplitude corresponding to difficult-to-recognize icons.This paper firstly summarizes and sorts out the research progress and related achievements of domestic and foreign scholars in the past five years from the aspects of cognitive ability research of product interaction design and neural decision-making technology of product interaction design scheme. Secondly, it puts forward a detailed review of the research objectives and technical routes of neural decision-making in interaction design, and summarizes the current application fields of neural decision-making in Internet product interaction design based on EEG. The source of electrical data and other issues, and the prospect of EEG neural decision-making in the field of Internet product interaction design research.

INTRODUCTION

Internet products have penetrated into all aspects of people’s work, educa- tion, health, communication and entertainment (Biswas et al., 2016); Online information search, reservation, shopping and other apps can not only bring convenience to users, but also help users improve work efficiency. The “digi- tal 2018” Internet Research Report shows that the total number of Internet users around the world has exceeded 4 billion, and the number of domestic Internet users has increased rapidly, reaching 772 million as of January 2018. Studying the interactive design of Internet products and improving the user experience of products will help to increase the social welfare of Internet users and improve work efficiency and knowledge dissemination efficiency. At pre- sent, the interaction design of Internet products increasingly emphasizes user experience, and there is an urgent need for new human factors measurement

© 2022. Published by AHFE Open Access. All rights reserved. 374

Neural Decision-Making Application of Internet Product Interaction Design Based on EEG 375

methods and tools to more accurately and objectively evaluate the effect of product interaction design, so as to provide theoretical support and basis for product design and decision-making. Neural measurement is different from traditional human factors research methods. It overcomes the subjective devi- ation of questionnaire, scale and other methods, and can dynamically grasp the user’s cognitive process of the product in the millisecond accuracy, so as to provide neurological decision-making basis and guidance for design practice.

Users’ cognition of products includes perception, attention, memory, thin- king and other aspects, especially attention, working memory and spatial perception have an important impact on the learning and use of products. The above cognitive factors indirectly affect the user’s sense of control and overall feeling of the product, and ultimately affect the user’s perception of the product user experience. However, the mechanism of cognitive ability affecting network behavior is complex. The existing research methods pay more attention to the behavior research results (such as usability research), which is difficult to reveal the dynamic process of user cognition, the brain cognitive mechanism of users in the process of interacting with products, and it is impossible to use the theoretical results to guide the design. Therefore, the research on neural decision-making of Internet product design is of great significance to the interaction design of Internet products.

RESEARCH STATUS OF EGG IN PRODUCT DESIGN The research of product interaction design at home and abroad is divided into two main directions: one is the research on the cognitive ability of product interaction design, and the other is the neural decision-making technology of product interaction design scheme. The following reviews the research of the two technologies at home and abroad.

Research on Cognitive Ability of Product Interaction Design “Cognitive ability” includes verbal information, intellectual skills, cognitive strategies, attitudes and motor skills]. Over the past 20 years, there have been “cognitive models” of “reading ability”, “cognitive ability of” working space “,” cognitive ability of “reading ability” and “cognitive ability of” visual abi- lity “. The most systematic and in-depth research on cognitive ability in the field of human-computer interaction belongs to virtual cognitive modeling. For example, the models soar, act and epic combined with psychology and artificial intelligence, the simplified GOMS (goals, operators, methods, and selection rules) of these models, the models ease and cogtool dedicated to improving interactive inclusiveness, and guide considering the ability limita- tions related to disability and aging. Funded by the EU horizon 2020 research and innovation project, Greek researchers have recently developed a new vir- tual user model that can simulate the mild cognitive impairment of the elderly, and will develop a cognitive prediction model. Most of these studies have established complex virtual user model prototypes from the technical point of view, but they are still unable to understand the dynamic brain cognitive process of users using products. Due to the large gap between theoretical

376 Yang and Hua

discovery and design practice, the transformation of Ergonomics application has not been completed.

Cambridge University has explored the user’s cognitive ability and the inclusive evaluation of product design, and believes that the cognitive abili- ties that have a direct impact on design include: working memory, long-term memory, mental model, planning and problem-solving ability, language and communication ability. From the perspective of product use, neerincx and others defined the cognitive ability of users as intelligence, memory, langu- age and reading ability. In the I-design interdisciplinary project funded by the British Society for engineering and material research (EPSRC), Cambridge University conducted research on the sensory, motor and cognitive abilities of British adults in order to provide designers with comprehensive user data reference. However, limited by conditions, this study only collected several easily measured “cognitive ability” data based on psychology, which is not closely related to human-computer interaction, and has not yet formed intera- ctive cognitive ability knowledge for designers. How to transform user data into designer knowledge is a major “challenge and opportunity” in design research.

There are also studies involving cognitive ability in China. Rao Peilun of Tsinghua University pointed out that cognitive aging generally includes perce- ptual aging, memory aging, attention aging and intelligence aging [26]. Zhang Hongbing of Shanghai University proposed that multi-channel user interface design is more suitable for users with cognitive impairment. The applicant’s research team sampled people aged 50-80 in urban and rural China (with an average age of 64), collected user cognitive ability data mainly through self-report, and found that there were significant differences in the use of information products between urban and rural users. The study also found that it is necessary to develop new cognitive tests that are more relevant to design and increase objective cognitive experimental measurements. Howe- ver, cognitive ability is complex and difficult to measure. The measurement of cognitive ability has not been involved in the new round of “survey of basic ergonomic parameters of Chinese adults”.

Neural Decision Technology of Product Interaction Design Scheme Research Contents The concept of neural management and neural design was first put forward by Professor Qingguo Ma of Zhejiang University in China. Through 10 years of practice, it provides a lot of theoretical basis for the development of neu- ral design. At present, many neural management teams at home and abroad have carried out research on product neuro-measurement from different per- spectives, and preliminarily achieved the following results: a) the cognitive process of product interaction is complex. Neuro-measurement research divi- des it into early components and late components, and relevant research has been carried out from different stages. Qingguo Ma research team of Zheji- ang University (2011-2014) found that N1 was related to early attention, P2 responded significantly in evaluating aesthetics, and there was a significant difference in the amplitude of beauty and nonbeauty. In addition, N2 is a

Neural Decision-Making Application of Internet Product Interaction Design Based on EEG 377

good measurement index in terms of innovation, N400 in terms of familia- rity, etc. Due to the complex composition of cognitive factors in the process of product interaction, there are many corresponding cognitive neural indi- cators, and they are still in the exploratory stage. At present, there is no complete index system for practical application reference. The latest resea- rch not only focuses on the neural response components, but also studies the bands related to the neural activity, such as theta band and delta band, which are found to be related to neural activities such as product memory and clas- sification. b) Multiple ERP components related to decision-making can be used to evaluate the response level of user experience. Guo Fu (2016) of Nor- theast University and Chengqi Xue (2016) of Southeast University found that the classic components of P300 related to cognition respond in the process of product appearance experience and interface interaction. ERP components related to cognition mostly belong to late components, which have measure- ment significance for interactive interface recognition, navigation recognition and so on. c) For the LPP component related to emotion, Jin Jia et al. (2014) found in the research of warning signs that users will have a large LPP ampli- tude for words with strong warning. Compared with existing studies, it is found that the modified component is helpful to judge the emotional state after users’ information understanding. d) The analysis of resting EEG data shows that theta band of parietal region is an important index of cognitive load monitoring, and time-frequency analysis is the main analysis method of this kind of data. In addition, cognitive laterality analysis can also reflect the cognitive state of interaction process.

RESEARCH ON NEURAL DESIGN APPLYING IN PRODUCT INTERACTION DESIGN Research Contents

Through neural measurement, we will study the ERP components and neural mechanism of users in the process of product interaction, build a neural measurement index system to evaluate product interaction design, and put forward neural measurement standards to provide basis for design decision-making. The research contents are as follows: The attention neural mechanism of product icon shape, color and other design elements. Mis-operation in the use of Internet products is usually cau- sed by factors such as inattention and recognition errors. Therefore, it is very important to study the influence of various design elements on the compo- nents of attention neural response, including latency, amplitude and response position. This study focuses on two cognitive neural components: N2 and LPP. N2 is a cognitive neural response component of early attention. LPP is a neural component related to attention after full cognition, and reflects the emotional changes of users. It is an important cognitive neural index of emotional experience.

Neurocognitive mechanism of product interaction design decision. Pre- vious studies have shown that product decision-making is related to P300 components in cognitive nerve. In interaction design, the amplitude of P300

378 Yang and Hua

corresponding to easy to identify icons is greater than that of P300 cor- responding to difficult to identify icons. However, the interaction design of Internet products is an overall process, and user identification is also an overall comparison process. Therefore, whether P300 in the interaction design decision-making process can effectively represent decision-making information needs to be explored in this study.

Research on emotional experience of product interaction design. The com- position of user experience is very complex. It is generally divided into hedonic experience and practical experience. The interactive design attri- butes related to hedonism include design aesthetics, emotional experience, physiological experience, cognitive experience, etc. each experience attribute corresponds to one or more neurometric indicators. Some studies have only involved some ERP components related to aesthetics, emotion and cognition, and mostly discussed the availability and sensitivity of this indicator from the theoretical level, There is still a lack of relevant research on user experience design practice. Therefore, this study takes the emotional experience as the research object to explore the changes of users’ emotional experience in the process of product interaction.

Technology Roadmap This study adopts the method of combining empirical research and the- oretical research. The empirical research adopts the research methods of questionnaire, in-depth interview, behavior record and EEG experiment. Based on information communication theory, interaction design theory, Situ- ational Cognition Theory and Wilson information behavior model, according to the research objectives and contents, the research is divided into two modu- les, corresponding to a number of research contents respectively. The overall framework of the technical route is shown in Figure 1, and each module contains several tasks. The technical route focuses on two aspects: intera- ctive design neural decision-making and Ergonomics application. They are interdependent. Interactive design neural decision-making originates from the theory of cognitive psychology and serves design; Ergonomics applica- tion aims at improving the accessibility of the interactive interface, and the effect of Ergonomics application is tested through various indicators of neural decision-making. The technical route is as follows:

In the design of neural decision-making part, firstly, study the attention neural response mechanism of various elements of interactive design, pay attention to the amplitude, latency and response area of attention neuroco- gnitive components, compare the impact of different design elements on neu- ral components, and provide ergonomic application suggestions; Secondly, the neural response component of the overall decision-making of interactive design scheme is studied to provide a theoretical basis for Ergonomics appli- cation decision-making; Thirdly, the emotional body of product interaction design scheme is studied.

Test the neural response component, and provide the decision-making basis of user emotion for Ergonomics application by judging the user’s emotional response. Finally, a set of effective neural decision-making and

Neural Decision-Making Application of Internet Product Interaction Design Based on EEG 379

Figure 1: Technology roadmap.
Original PDF Figure 1:

ergonomic application methods for interactive design of Internet products are formed.

Ergonomics application part: firstly, carry out literature research around Situational Cognition Theory, interaction design theory and Wilson infor- mation behavior theory, and then collect and sort out the commonly used app interaction cases in China; According to the neuroscience basis of neural decision-making, it provides guidance for design practice. Design and deve- lop a set of interactive design scheme in line with neurocognitive theory, carry out neural decision-making evaluation on the scheme, and vertically compare the advantages and disadvantages of the scheme, so as to provide a method for Ergonomics application.

Experimental Method Design This study uses EEG experimental method and combined with technical route to do the following experiments: A. design EEG experiments in which attributes affect the neurocognitive mechanism of attention. The experi- ment plans to adopt a series of basic design neural experiments to carry out EEG experiments on interactive icons, colors, aesthetics, etc. in the design practice, so as to explore the guidelines of design elements in line with the neurocognitive mechanism of attention. b. EEG experiment of overall neural decision-making of interactive design scheme. The experi- ment intends to adopt the classic oddball experimental paradigm, take the excellent interaction design scheme as the target stimulation material and the ordinary scheme as the background stimulation material, observe the role of P300 EEG component in the decision-making process, and explore the application of this component in the decision-making of interaction design scheme. c. Neural evaluation experiment of emotional experience in

380 Yang and Hua

interactive design. The experiment takes the influence of operation fluency on emotional experience as the experimental goal, and measures users’ emo- tional experience through simple operation. In addition, for the problems found in the experiment, feed back to the designer to make a new round of design scheme. d. Interaction design scheme, ergonomic behavior experiment and EEG experiment. The experiment adopts behavioral experiment and EEG experiment paradigm to observe the user’s subjective feedback and EEG feedback on the design scheme. Through the subjective and objective joint evaluation, it provides the basis for the decision-making of design scheme.

Key Technology Frequency domain analysis and time-frequency analysis technology of EEG signals. General EEG ERP experiments mainly focus on time domain analysis, which mainly discusses the latency, amplitude and response distribution, and less studies the frequency domain and time-frequency information. Freque- ncy domain analysis takes the electrode as the research object to analyze the power distribution. Time-frequency analysis is a combination of the two. In this study, the comprehensive consideration of neural measurement indicators requires not only time-domain information, but also frequency-domain infor- mation. Therefore, the comprehensive evaluation of neural measurement indicators is one of the key technologies of this study.

Multiple comparison test technique of repeated measurement analysis of variance. EEG experimental data is large and there are many collection points, so it is easy to appear false and significant in statistical analysis. Mul- tiple comparison and correction technology can reasonably avoid this risk, but under what conditions and which correction technology is used is a key technology. Through long-term accumulation, our research team has a good grasp of greenhouse geisser correction technology and Bonferroni multiple test technology.

RESEARCH TRENDS The traditional design adopts the perceptual evaluation method to evalu- ate the product design and interactive interface. Perceptual evaluation is the explicit result of neural evaluation, but perceptual evaluation is easy to be affected by social approval effect in life. Sometimes it will express distor- tion and mislead product development and decision-making. Design neural evaluation is an implicit measurement of user experience, which does not directly require users to provide subjective feedback. In the process of using or watching, the EEG data of users are analyzed through event-related poten- tials to understand the real experience of users. Design neural evaluation is more objective and accurate, but the difficulty lies in the selection and analysis of ERP components. This study systematizes the ERP components of user experience measurement and explores targeted measurement indica- tors, which is a preliminary preparation for design neuro-evaluation and is of great significance to the development of design neurology.

Through the research on the brain response of APP shape, color and gra- phic complexity, this paper puts forward the neural design guide of APP

Neural Decision-Making Application of Internet Product Interaction Design Based on EEG 381

icon to help users quickly and accurately find the required app among the many apps in mobile phones, and use the neural response characteristics of attention to reduce the search time and improve efficiency. Eliminate app icon mis-selection and mis-operation. Summarize the neural response characteristics of icons, refine the common problems in icon design, and provide a research basis for design intervention. Take the interface layout, attention of navigation and semantic neural response mechanism as the research object, improve the interaction efficiency of product interface, and eliminate the phenomena of unclear key information, difficult to find, chaotic navigation space and lost navigation. On the basis of neural mea- surement, using the way of design intervention, this paper deeply analyzes the key neural response indicators affecting interaction performance, puts forward the dynamic response model of users’ cognitive Internet product interaction, develops interaction design methods and tools, and enriches the connotation and extension of interaction design. Measure the emoti- onal response of users in the use of Internet products, and take it as an important index to evaluate Internet products. Establish the neural response index related to emotional experience as an important basis for design decision-making.

CONCLUSION

This study studies interaction design decision-making from the perspective of cognitive nerve, re-examines the causes of interaction design failure, conden- ses common problems, brings cognitive factors such as neural mechanism, neural decision-making and emotional experience into ergonomics resea- rch, and constructs interaction design direction and interaction experience decision-making method serving design. From the perspective of neuro- science, it expands the connotation of interaction design and provides a theoretical basis for eliminating the cognitive impairment of interaction design. In addition, the basic framework of neural decision-making in pro- duct interaction design is established, and the neurocognitive indexes suitable for Internet product interaction design decision-making are explored, which provides a theoretical basis for design decision-making. By integrating the methods of experimental measurement and qualitative research, this paper finds the mapping relationship between user cognition and interactive beha- vior by setting interactive tasks related to design, uses the method of design intervention to detect the availability and pertinence of typical interactive task design schemes, and comprehensively explains the mechanism of design elements affecting user neurocognition. In the aspect of neural decision- making of design scheme, the method of combining neural measurement and behavior experiment is adopted to make the decision-making result more accurate and objective. The successful implementation of this project resea- rch will play a positive role in promoting human factors research to serve the design and application, and help the design of Internet information products better meet the needs of users. It has both theoretical innovation and good application prospects.

382 Yang and Hua

ACKNOWLEDGMENT

This project was sponsored by grant No. LGG19G010001 from the Natural Science Foundation of Zhejiang province.

REFERENCES

Biswas, P., Robinson, P., & Langdon, P. (2012). Designing inclusive interfaces through user modeling and simulation. International Journal of Human- Computer Interaction, 28(1), 1–33. Biswas, P., Schmutz, S., & Sauer, J. (2016). The influence of age in usability testing.

Applied Ergonomics, 52, 291–300. Castilla, D. & Garcia-Palacios, A. (2016). Effect of Web navigation style in elderly users. Computers in Human Behavior,55(3):909–920. Clarkson, P. J., & Coleman, R. (2015). History of Inclusive Design in the UK. Applied

ergonomics, 46, 235–247. Czaja, S. J., & Lee, C. C. (2007). The impact of aging on access to technology.

Universal Access in the Information Society, 5(4), 341–349. Hurtienne, J.,Horn, A. M. & Langdon, P. M. (2010). Facets of Prior Experience and Their Impact on Product Usability for Older Users[J]. Designing Inclusive Interactions, 38, 123–132. Langdon, P., Johnson, D., Huppert, F., & Clarkson, P. J. (2015). A framework for

collecting inclusive design data for the UK population. Applied ergonomics, 46, 318–324. Lohse, G. L. (1997). The role of working memory on graphical information processing. Behaviour & Information Technology, 16(6), 297–308 Ma Q., Hu L., & Wang X. (2015) Emotion and novelty processing in an implicit

aestheticexperience of architectures: evidence from an event-related. NeuroRe- port, 26:279–284. Mankoff, J., Fait, H., & Juang, R. (2005) Evaluating accessibility by simulating the experiences of users with vision or motor impairments. IBM Systems Journal, 44(3), 505–517. Nielsen, J., & Levy, J. (2010). Measuring Usability Preference Vs Performance.

Communications of the ACM, 37(4):66–73. Prensky M. (2001) The scientific evidence behind the Digital Native⣙s thinking changes, and the evidence that Digital Native-style learning works [J]. Digital Natives, Digital Immigrants. pp: 1–6. Waller, S. D., Langdon, P. M., & Clarkson, P. J. (2010). Using disability data to

estimate design exclusion. Universal access in the Information Society, 9(3), 195– 207. Wang X., Huang Y., & Ma Q. (2012) Event-related potential P2 correlates of implicit aesthetic experience[J]. NeuroReport, 23:862–866.

Cover of the proceedings volume Human Interaction & Emerging Technologies (IHIET 2022): Artificial Intelligence & Future Applications

Explore the complete digital volume

Human Interaction & Emerging Technologies (IHIET 2022): Artificial Intelligence & Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-44-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →