Fenglin Gao, Fei Xu (2023). The Sound Nest project: mobile Application design for auditory cognitive training of stressed people in the post epidemic era. In: Ergonomics In Design. AHFE Open Access, vol. 77, pp. 798–806. https://doi.org/10.54941/ahfe1003440
Under the COVID-19 epidemic and the economic downturn, people's pressure is increasing daily. Many people in the workplace are troubled by the distraction caused by excessive pressure. This paper aims to study the effective methods for stressed people to improve their concentration in the post-pandemic era. It also introduces the design and development process of an Android system-based mobile app - "Sound Nest." With auditory cognitive training as the core, we will conduct nonmusical auditory training for stressed people from three aspects: sound frequency, spatial orientation, and rhythm. We use SteamAudioVR to build a 3D virtual sound field and, with the guidance of visual interface orientation, to help improve our concentration. This software has three training modules. Based on the auditory attention training (AT) of Adrian Wells, it constructs single-source orientation training and multi-source track training modules. Based on the binaural auditory beats training of James · D · Lane · A, build a binaural beat training module. 3D virtual sound field orientation is constructed based on the ASMR Map of Emma L. Barratt. This application conducted sampling tests on some stressed people and received good feedback, which initially entered clinical validation.
Due to the high infection rate and a certain degree of mortality caused by the COVID-19 epidemic, many countries have psychological problems including anxiety, depression, behavioral burnout, and lack of concentration (Yıldırım and Solmaz, 2020). During the epidemic, workplace staffs were also under great pressure and anxiety because of “no boundary between work and life at home”, “material shortage”, “family conflicts”, “fragmentation events” and other problems. Relieving pressure has become a new demand of design.
To relieve anxiety and stress, modern treatment mainly relies on drugs, physical equipment and mindstory cognitive training. The research scope of this paper belongs to the third category. Mindstory cognitive training is derived from Neuroplasticity (Garlick, 2002). Scientific evaluation and
The Sound Nest Project: Mobile APP Design for Auditory Cognition Training 799
systematic training are used in training to improve people’s attention abi- lity, perception ability, memory, thinking ability, emotional regulation ability, cognitive flexibility, etc. This research is a set of auditory cognitive training system based on the basic goal of cognitive training, aiming at alleviating the pressure of social groups, and based on the sound space positioning technology.
As early as 1894, Urbantschitsch initiated the method of hearing training in Innsbruck, which was initially studied to restore the hearing function of people with hearing impairment. In 1952, Henk C. Huizing, a professor of audiology and phonology at Groningen University in the Netherlands, used a conventional metronome to conduct listening training for the sub- jects, so as to activate the “recognition ability” of human senses and help the patients slowly recover their cognitive function. This kind of therapy was widely concerned in the society at that time (Huizing, 1952). In the 1960s, Guy Bernard, a French otolaryngologist, designed a training program to treat hearing disorders such as depression, dyslexia, tinnitus and autism, and put it into clinical use later (Madell and Rose, 1994). Since 1990, auditory training has been studied and applied in the United States. The research on the effecti- veness of using hearing for autism and other related psychological disorders had begun to take shape at that time, but the professionalism of its treatment effect was questioned by some experts. In 1998, James D Lane A, professor of the American Department of Psychiatry and Behavioral Sciences, propo- sed binaural rhythm training, and initially found that sound has the effect of influencing brain waves (Lane et al., 1998). Since 2010, a large number of papers on the relationship between EEG and sound have been published, and sound perception training has been presented to the public. It has become a new research topic on how sound can generate benign brain waves and form brain focus. It can be seen from this that, from the initial sound training aimed at treating hearing impairment to the sound training aimed at emoti- onal regulation and concentration improvement, hearing training is slowly developing towards the spiritual field.
At present, the cognitive function training is an effective method for pre- venting distraction. In auditory cognitive function training, AT (Attention Training) and binaural rhythm training have good clinical performance.
CONTENT AND GENERATION PRINCIPLE Training Method
As a method of auditory cognitive training, AT (Attention Training) was pro- posed by Professor Adrian Wells of the Clinical Psychology Department of Manchester University in 1990 (Wells, 1990). It has produced good curative effect for the symptoms of inattention caused by excessive pressure. AT can help trainees adjust their cognitive style of attention. If there are unpleasant feelings or physical discomfort during AT practice, the trainees should simply treat them as noise and do not distract or try to analyze the sound. Trainees should selectively pay attention to these sounds in turn, firstly recognize them, and then focus on them. The trainees will quickly shift their attention from one sound source orientation to another, and switch back and forth from
800 Gao and Xu
multiple sound source orientations. The training distracts attention by paying attention to as many different sounds and directions as possible at the same time, and the trainees improve their concentration by constantly identifying the direction of sound source and the type of timbre.
The binaural rhythm training is a training therapy mentioned by James D Lane A, a professor of American Department of Psychiatry and Behavioral Sciences, in an article published in 1998 (Lane et al., 1998). In the article, the professor believes that the binaural auditory rhythm is a perception pheno- menon. When two tones with slightly different frequencies are presented to the two ears, the listener will perceive a single tone whose amplitude changes between the two tones. The article shows that when the listener listens to the binaural rhythm, the EEG activity detected by the equipment may affect the state of consciousness, thus changing the cognitive and mental state of the per- son. This technology can be used to control attention and change emotional feelings. Then, in 2018, Professor Garcia-Argibay Miguel of the Department of Behavioral Sciences of UNED University in Spain tested listeners who recei- ved binaural rhythm training from 2007 to 2017, and the results confirmed that binaural rhythm training did have a positive effect on controlling atten- tion and changing emotions (Garcia-Argibay et al., 2019). In order to achieve the training effect of improving concentration, AT focus training needs to build the 3D sound field required for sound source orientation on the mobile terminal and create non-musical timbre materials. The binaural rhythm trai- ning needs to try the different audio rhythm and sound feeling of the two ears. To this end, we have carried out research on virtual sound field, sound, sound source orientation and visual correspondence, and audio beat.
Sound Field Research In an article by Professor Emma L. Barratt of the Department of Psychology of Swansea University, it was pointed out that the areas most strongly sti- mulated by sound orientation were the back of the head, scalp, and back of the neck, which gradually extended to the shoulders and back (Barratt and Davis, 2015). Therefore, in the author’s system, the sound directional stimu- lation mainly focuses on the head to shoulder and upper torso. In order to improve the effect of sound orientation on concentration in AT, the 3D virtual sound field is built in the application terminal to strengthen the perception of sound orientation.
3D virtual sound field refers to the use of VR technology to simulate the sound from different directions and angles heard in the real world in vir- tual applications. Its construction is crucial, which determines the difference between this application and general applications. In traditional applications, it is very difficult to simulate the sound field. Android Studio is not perfect for virtual space simulation and is not commonly used in general applica- tions. The author borrows the space technology of Unity engine and uses the Steam AudioVR space sound effect solution of Valve Company to com- plete the construction of 3D sound field. Steam Audio can integrate ordinary audio, Ambiosonic audio and SOFA, and use ray-tracing method to better restore the sound change effect caused by the dynamic changes of characters
The Sound Nest Project: Mobile APP Design for Auditory Cognition Training 801
Figure 1: Sound source position test.
and scenes in VR scenes. The 3D sound field is composed of user position (AudioTo), sound source position (AudioFrom) and motion track. The sound source moves and transforms in space according to a specific track, and the user feels the sound source stimulation from different directions in the center of the sound field (Fig. 1).
Frequency Research The form of sound in real space is far from the visual expression of people’s daily contact. Sound itself is a 3D form of sound frequency. It also exists in 3D form in space. In our daily life, the visual expression of the sound we touch is the image generated by 2D collapse of the sound from one dire- ction, which is mostly the relationship between the amplitude and time of the sound, that is, the relationship between the decibel (volume) and time. This graphic expression is convenient. But it will hinder our understanding of sound. Just like everyone’s appearance is very different, it is a way for us to distinguish the characteristics of different people. Different sounds also have different characteristics. Locating the characteristics of sound is a way for us to find a specific sound. Using the spatial visualization atlas of sound and the sound feature location method of vibration frequency, amplitude and time, the author will analyze the collected sound, find the commonality betw- een sound, and provide supporting reference data for the subsequent sound production. In the article of Professor Garcia-Argibay Miguel, the effect of music assistance on hearing training was tested. The research shows that music melody will weaken the brainwave effect brought by auditory trai- ning (Garcia-Argibay et al., 2019). Therefore, in this system, we choose to create non-musical sound materials. Next work is to use the web crawler software to search audio on the music platform with keywords applicable to the focus area, collect audio comments, and set positive emotional keyword analysis for such comments. According to the score, select the audio with the
802 Gao and Xu
most pressure relief feeling, and use Praat (Phonetics software) to analyze the sound spectrum (Fig. 2).
By analyzing the frequency spectrum of audio with pressure relief feeling, we can find that the individual differences of these sounds are large on the spectrum chart, but there is a special commonality on the spectrum characte- ristic chart, that is, they all have frequencies of medium and high frequency bands and above (500-16KHz). Most of them have strong auditory sense in high frequency band. Therefore, the author summed up the audio fea- tures with a certain sense of pressure relief. That is, the sound at 0-30DB volume in the middle and high frequency band (0-3KHz), and the sound at 0-50DB volume in the high frequency band (5K-16KHz) have certain healing function (Fig. 3). When making audio, use the Pigments sound synthesizer to create sound audio that conforms to the audio characteristics, and use Insi- ght to monitor the spectrum of the produced audio in real time, so as to avoid audio that does not conform to the specifications during production. A total of 64 audio materials were produced. The production of audio is clo- sely related to the final effect. The production of a single sound source cannot be meaningless. The sound source must be placed under the 3D sound field for testing and adjustment, so that the single or multiple sound sources can move in the direction on the track of the program call, and the test focuses on improving the effect. In the follow-up test feedback, 18 types of audio were selected as the mobile audio source.
Figure 2: Audio analysis results by Praat.
Medium-high Frequency
High Frequency
High Frequency
High Frequency HZ Below 2000 Below 3000 Below 5000 9000-10800 DB 0-30 0-52 0-40 0-40
Figure 3: Audio frequency feature analysis.
The Sound Nest Project: Mobile APP Design for Auditory Cognition Training 803
Sound Orientation and Visual Correspondence EEG can record the bioelectric signals of brain neurons on the cerebral cortex with electrodes. Both visual and acoustic stimuli will affect EEG. Professor of Basel University of Psychiatry Mirjam Munch studied the effects of vari- ous color light stimuli on brain waves in high-density EEG with Arc-free ozone good xenon lamp (Munch et al., 2014). Li Jie, associate professor of Shanxi Medical University found that dual-brain synchronous resonance music can increase the duration of memory in the field of α Brain wave music. During interface design, visual guidance in line with sound perception will strengthen the impact of audio on brainwave, and thus improve the training effect (Li et al., 2012). Therefore, we designed the visual representation of the sound direction of the mobile terminal for many times, aiming to integrate the visual perception and functional training, and connect the users’ two sen- ses to achieve better training effect. The visual representation of the rail ring is painted with the Shader in Unity. The curve drawing of graphics is mainly used. Multiple circles are superimposed in the center of the picture, audio information is imported in the direction of the sound, and the value returned by the inverse tangent function is used as the RBG value of the change color.
Audio Beat Research In the binaural rhythm training, the voice heard by the trainees is the mid value of the frequency difference between the left and right vocal channels. This mid value, as the frequency of brain wave simulation after hearing binau- ral beat sound, will affect and regulate people’s concentration and sleep. In the study of the prefrontal cortex of the brain by Elif Kirmizi-Alsan, profes- sor of the Department of Physiology, Istanbul Medical College, it was found that the delta, theta, alpha, beta and gamma bands would have different effe- cts on the brain (Kirmizi-Alsan et al., 2006). At 0.01-4HZ, people’s sleep will be improved. Professor James D Lane A found that binaural beat audio at 16-24hz frequency can effectively improve the concentration of subjects (Lane et al., 1998). Therefore, in the module of building binaural beat trai- ning on the mobile terminal, focus on training the binaural beat material with a difference of 14.01-30HZ, use binaural beat material with a differe- nce of 0.01-4HZ in the sleep training, and recommend the trainees to conduct listening training for a long time.
SOFTWARE INTERFACE AND FUNCTION Information Framework
Following is the information framework tree of the App (Fig. 4).
APP Interface The APP is named of Sound Nest, and the main body is divided into four secti- ons, named: sound source location, sound track tracking, rhythm memory and discovery. Neomorphism is applied to the interface vision, which is desi- gned in the form of virtual keys, with pure color background to highlight the experience of quiet and healing. Among the four sections, the first three secti- ons are the training function section. The sound source location and sound
804 Gao and Xu
Figure 4: Information framework tree.
track tracking are based on the principle of AT, and the rhythm memory is generated based on the principle of binaural beat training. The discovery section is the explanation of the reference and theoretical basis of the whole software (Fig. 5).
Features The APP is equipped with three modes to guide users to conduct sound cognitive function training based on the principle of “from simple to deep”. There are sound source personalization options in the three sections, and the selected sound source has different attribute characteristics. There are speed
The Sound Nest Project: Mobile APP Design for Auditory Cognition Training 805
Figure 5: App four parts interface.
adjustment options in the two sections of sound source location and sound track tracking to select the sound change period. There are many choices for the direction of sound source location. The self-determined option is con- trolled according to the built-in gyroscope program of the phone. The user can rotate the phone to get the change of sound source orientation. There are two different categories of options in the sound track tracking: recommended and customized. Two tracks are set in the recommended options, and diffe- rent sound sources can be added separately for stacking; Customized tracks are added to the customized options. Through the arrangement and combi- nation of different tracks in the time slot, users can create track sequences that match their own auditory feelings (Fig. 6).
Figure 6: Interface of sound track tracking.
CONCLUSION
This paper studies and designs a cognitive training software with sound space positioning as the technical framework, aiming at the problem of distraction caused by increased social pressure after the COVID-19 epidemic. Based on the spectrum analysis of the decompression sound source, a number of sof- tware are used to produce the sound source that meets the specific needs, and
806 Gao and Xu
a certain rhythm and spatial layout are combined and distributed in the left and right ear channels. Finally, a set of operation software with visual guida- nce interface is completed, which is composed of three functional modules: sound source location, sound track tracking and rhythm memory, and has a positive effect on the user’s sound source spatial judgment and binaural beat perception. The software is currently in the stage of demo test, and we has organized a certain number of users to use it, and initially recorded the convergence items of interface operation smoothness and function selection, so as to further adjust the content and interface. The problem has been found is that there are certain cognitive difficulties in some customized function interfaces of the APP, so users usually choose the default function options as the basic choice. In the acquisition of special sound sources, it is necessary to further make or collect effective sounds to achieve better results. In terms of hardware, it is necessary to further test the matching effect of different ear- phones and sound source location programs to achieve better presentation quality of sound sources in space.
REFERENCES
Barratt E, Davis N.: Autonomous sensory meridian response (ASMR): a flow-like mental state. PeerJ (2015). Garlick D.: Understanding the nature of the general factor of intelligence: the role of
individual differences in neural plasticity as an explanatory mechanism. Psycho- logical Review, 109(1), 116–136 (2002). Henk C Huizing.: Auditory training. Acta Oto-Laryngologica, 41(100), 158–163 (1952). Jane R. Madell, Darrell E. Rose.: Auditory integration training. American Journal of
Audiology, 1(3), 14–18 (1994). Kirmizi-Alsan E, Bayraktaroglu Z, Gurvit H, et al.: Comparative analysis of eventrelated potentials during Go/NoGo and CPT: Decomposition of electrophysio- logical markers of response inhibition and sustained attention. Brain Research, 1104(1), 114–128 (2006). Lane J, Stefan J. Kasian, Justine E. Owens, et al.: Binaural auditory beats affect
vigilance performance and mood. Physiology & Behavior, 63(2), 249–252 (1998). Li J, An B, Cui W, et al.: Improving Effect of Alpha Brainwave music on Middle School students’ memory. Chinese Journal of Mental Health, 26(4), 283–286 (2012). Miguel Garcia-Argibay, Miguel A. Santed, José M. Reales.: Efficacy of binau-
ral auditory beats in cognition, anxiety, and pain perception: a meta-analysis. Psychological Research, 83, 357–372 (2019). Munch M, Plomp G, Thunell E, et al.: Different colors of light lead to different adaptation and activation as determined by hight-density EEG. Neuroimage, 101, 547–554 (2014). Wells A.: Panic disorder in association with relaxation induced anxiety: An attention
training approach to treatment. Behavior Therapy, 21, 273–280 (1990). Yıldırım M, Solmaz F.: COVID-19 burnout, COVID-19 stress and resilience: Initial psychometric properties of COVID-19 Burnout Scale. Death Studies, 1–9 (2020).
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Ergonomics In Design
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-53-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional