Zhiyuan Li, Zhaozhe Zhou, Yiyan Wang, Jingze Tian, Wenjun Yang, Yafeng Niu (2023). Enhancement Characteristics of Visual Stimulus Elements in SSVEP-BCI System. In: Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems. AHFE Open Access, vol. 69, pp. 954–964. https://doi.org/10.54941/ahfe1002921
SSVEP-BCI system has the advantages of few recording electrodes, short training time, strong anti-interference ability, etc. It is widely used in the brain-computer interface (BCI) field. However, this system also has the disadvantage of low recognition efficiency. Its stimulation interface will also cause visual fatigue for the users. Therefore, aiming at the optimization of stimulus elements in the SSVEP-BCI interface, this paper explored the impact of different numbers of auxiliary stimulus particles on the system recognition efficiency and user experience under the same stimulus area. Relevant ergonomic experiments and subjective evaluations were conducted. Ergonomic experiment results show that the number of auxiliary stimulus particles has a significant impact on the recognition efficiency of the system. When the number of auxiliary stimulus particles approaches infinity, the recognition efficiency is the highest. The subjective evaluation results show that the change in the number of auxiliary stimulus particles has a significant impact on the system interface usability index score, and the score is higher when the number is infinite. From the perspective of design ergonomics, this study explored the impact of the number of auxiliary stimulus particles on the efficiency of the brain-computer interface system and user satisfaction, the research conclusions have important guiding significance for optimizing and standardizing the design of the brain-computer interface.
Nowadays, with the continuous breakthrough of high-tech, more and more advanced human-computer interaction methods have been created, such as brain-computer interface, augmented reality, virtual reality, gesture inte- raction, eye control, etc. (Niu et al. 2023). This paper selects the BCI system based on Steady State Visual Evoked Potentials (SSVEP). SSVEP is a kind of periodic brain response induced by repeating visual stimuli (Gao et al. 2021). This response represents different characteristics according to the different flickering frequencies of visual stimuli. By encoding different functional semantics into visual stimuli of different frequencies, SSVEP- BCI enables subjects to generate steady-state visual evoked potentials with
Enhancement Characteristics of Visual Stimulus Elements in SSVEP-BCI System 955
different characteristics. Then the system translates the SSVEP signals into the control language for the machine. In that way, the human brain can control the machine (Chen et al. 2020). A series of typical experimental paradigms were proposed to improve the communication efficiency of SSVEP-BCI and the ability of the brain to induce stimulation.
From the ergonomic perspective of human-computer interaction, designing a user-friendly interface and appropriate stimulus elements can help alleviate the visual burden of subjects and improve the efficiency of system recogni- tion. However, for SSVEP-BCI using the screen to present stimulus elements, there is still a lack of unified and standardized interface design standards and ergonomics research (Niu, 2022). For example, in the BCI experimen- tal paradigm of mixed P300-SSVEP designed by Li et al. (2013), to enhance the subjects’ attention, four groups of stimulus element targets are set on the visual interface, and each group of stimulus element targets consists of the big target in the middle and eight smaller auxiliary targets around it; To enhance system performance, Kapgate et al. (2020) also design a central sti- mulus surrounded with smaller auxiliary stimulus particles to evoke SSVEP signals. This interface design combining the central stimulus element with the auxiliary stimulus particle is conducive to improving the concentration of the subjects and thus improving the recognition efficiency. However, its scienti- fic nature needs further research. There is no corresponding design standard for this interface design. For example, the appropriate number of auxiliary stimulus particles and the distance between the central stimulus element and auxiliary stimulus particles are unknown. For the impact of changes in the number of auxiliary stimulus particles on system identification efficiency and user experience, there is also a lack of ergonomics research. Both interface designs adopted the style of combining auxiliary stimulus particles with cen- tral stimulus. However, those designs were lack of ergonomic research and subjective user evaluation.
EXPERIMENTAL DESIGN 20 Southeast University students were recruited for this experiment. Their ages ranged from 22 to 26, with an average age of 23.5 and a standard devi- ation of 1.15. The naked eye or corrected vision of all subjects was above 4.8, and there was no eye or brain disease. In the experiment, an easyCap EEG cap was used to collect the EEG signals of subjects. O1 and O2 electrodes were used as EEG input channels. After the open-source tool OpenViBE (Renard et al. 2010) in the computer receives the EEG signal, it uses the CSP common space pattern algorithm to extract the feature of the signal. It also takes the amplitude at the fundamental frequency and the second harmonic of the stimulus frequency as the feature index. The final output is the instruction, which is fed back to the experimental stimulus interface. Among them, the experimental stimulus interface is developed and presented by Unity and communicates with the OpenViBE designer through the VRPN protocol.
The stimulus element consists of a circular central stimulus and some auxi- liary stimulus particles. The auxiliary stimulus particles maintained the same
Figure 1: Design of stimulus elements.
center distance as the central stimulus at a specific angle. Every particle was evenly distributed around the central stimulus. The number of auxiliary sti- mulus particles was set at five levels, namely four, eight, twelve, sixteen, and infinite. Those particles of five levels were arranged at 90
, 45
, 30
, 22.5
and 0
around the central stimulus. According to the design principle of the Windows interface, the central stimulus diameter was set to 150px (Ya-feng et al. 2022). To avoid premature visual fatigue caused by the excessive size of auxiliary stimulus particles, when the initial number of auxiliary stimu- lus particles is four, the radius of a single auxiliary particle was set to 25px. Accordingly, the area of a single auxiliary particle was 6255px2 and the total area sum of four auxiliary particles was 25005px2. When the number of par- ticles continues to increase to eight, twelve, sixteen and infinite, to avoid the influence of area factor on the experimental results, the total area of auxiliary stimulus particles was set to keep 25005px2. Therefore, when the number of particles is eight, twelve, and sixteen, the radius of a single auxiliary parti- cle was set to 17.7px, 14.4px, and 12.5px respectively. For infinite particles, because the auxiliary stimulus particles were connected into a ring, the width of the ring was set to 15.1px. The distance between the center of the auxiliary stimulus particles and the center of the central stimulus was set to 115px at each number level, as shown in Figure 1.
In the design of the experimental interface, to reduce the visual fatigue of the subjects, the interface background was set to dark gray (RGB 85, 85, 85). Zambalde et al. (2018) showed that the cross-arrangement has a better system efficiency than the square arrangement. Therefore, the four visual stimulus elements with different frequencies were located at the top, bottom, left, and right of the experimental interface. The center of the visual stimulus element was aligned with the center of the interface, with a horizontal dista- nce of 600px and a vertical distance of 300px, as shown in Figure 2. The visual stimulus element used the experimental paradigm of a simple flicker stimulus, according to the research of Chu et al. (2016), red can elicit a bet- ter SSVEP response, so the stimulus color of visual stimulus elements in the experiment was red (RGB 255, 0, 0) and black (RGB 0, 0, 0). The stimulation frequencies were: f1 = 8.57Hz, f2 = 10Hz, f3 = 12Hz, and f4 = 15Hz.
During the experiment, the subjects kept sitting upright and kept the whole person and the screen at the center line. The distance between the eyes and
Enhancement Characteristics of Visual Stimulus Elements in SSVEP-BCI System 957
Figure 2: Design of experimental interface (take particle number 8 as an example).
the screen center was about 65cm. The vertical visual angle of the subjects to the screen was about 25.9
, and the horizontal visual angle was about 44.5
. To avoid other EEG signals generated by unrelated actions affecting experimental signal processing, subjects were only allowed to blink, swallow saliva, or slightly rotate their heads.
The experimental process of one trial in the experiment was as follows: In step 1, the interface center presented “+”, the subject need to look at the interface center, and waited it automatically jumped to the next step after 1000ms. In step 2, the interface presented four visual stimulus elements, and an indicator arrow appeared in the center of the interface. The subject needs to focus on the target visual stimulus element according to the arrow indi- cation. The system recognition result would turn green after matching the target visual stimulus element, indicating that the recognition was successful. When the system was recorded, it would jump to the next step; an 8s time- out was set for each trial, and a circular countdown progress bar around the arrow prompted the remaining time of the trial. If the identification was unsuccessful within 8s, the system automatically jumped to the next step. No prompt would appear in the interface, and the system recorded the task as failed. Step 3: The interface presented a dark gray blank screen, which lasted for 1000ms. This step was used to eliminate the visual residue of the subject. The next trial proceeded until the experiment at that level was completed. After the completion of the experiment at each level, the subjects were asked to close their eyes and rest. If necessary, eye drops were provided until the end of the experiment. The experimental process of a single trial is shown in Figure 3.
EXPERIMENTAL RESULTS AND ANALYSIS The reaction time refers to the time from which a stimulus element was pre- sented to be recognized successfully by the subject in the BCI system in one trial. The number of failures refers to the number of recognition timeouts in one trial. This timeout would be recorded as a failure in the system. The ave- rage reaction time of four, eight, twelve, sixteen and infinite auxiliary stimulus
Figure 3: Experimental process.
particles in the experiment was 3.573s, 3.833s, 3.959s, 3.911s, and 3.400s respectively.
The recognition success rate refers to the ratio between the number of successful recognition experiments and the total number of experiments. The number of successful experiments for the four, eight, twelve, sixteen and infinite auxiliary stimulus particles was 335, 341, 334, 329, and 336. Correspondingly, the task recognition success rate of experiments was 83.75%, 85.25%, 83.5%, 82.25%, and 84%. The experimental data were analyzed by SPSS Statistics 26. The total expe- rimental sample size of the response time of the number of auxiliary stimulus particles at five levels was 2000. After removing the number of failures due to response timeout, the sample size of the successfully recognized data was 1675. Kolmogorov-Smirnov (K-S) test was used to test its normality. The data significance of the number of different auxiliary stimulus particles was less than 0.05, which did not conform to the normal distribution. At the same time, the absolute value of the kurtosis of the data normality test result was 0.627 less than 10 and the absolute value of the skewness was 0.119 less than 3, indicating that although the data was not normal, it was acceptable to be a normal distribution. The results of the homogeneity analysis of variance showed that the F value of the different number of auxiliary stimulus parti- cles for the recognition success time was 0.795, and the p-value was 0.528, which was greater than 0.05, meeting the prerequisite requirements of using ANOVA.
The variance analysis was used to study the difference between the number of auxiliary stimulus particles and the reaction time. The F value of the dif- ferent number of auxiliary stimulus particle samples for the reaction time is 6.425, and the p-value is 0.000 less than 0.01, all of which show significance. This means that the different number of auxiliary stimulus particle samples had differences in the recognition time, and multiple comparisons were nee- ded for analysis to understand the differences between different levels. The results of multiple comparisons are shown in Table 1.
Through LSD analysis, when the four auxiliary stimulus particles were compared with sixteen, the p-value was less than 0.05, and the difference was significant; Compared with four and twelve, eight and infinite, twelve
Enhancement Characteristics of Visual Stimulus Elements in SSVEP-BCI System 959
and infinite, and sixteen and infinite, the p-value was less than 0.01, and the difference was extremely significant; The p-values of four and eight, twelve and sixteen were all greater than 0.05, with no significant difference.
USER’S SUBJECTIVE EVALUATION To further explore the impact of different numbers of auxiliary stimulus par- ticles on user experience, this paper adopted the specification for usability evaluation of human-computer interaction. According to the standard of ISO 9241-11: 2018 Ergonomics of human-system interaction–Part 11: Usability: Definitions and concepts, this paper used three indicators: efficiency (reaction time), effectiveness (recognition success rate), and satisfaction. Because of the visual fatigue problem of the SSVEP-BCI system (Mouli and Palaniappan, 2017), the index of fatigue was added. The subjective evaluation adopted a 7-point Likert scale and a total of 20 questionnaires were collected. Each questionnaire had 20 questions, and a total of 400 questions. According to the calculation of G*Power, when the effect size = 0.4, α=0.05, the statistical test power of this subjective evaluation is 1, and the experimental data can support this study.
Through the analysis of the reliability of the questionnaire, the result sho- wed that its Cronbach’s α = 0.740, between 0.70 and 0.98. In the exploratory study, this coefficient is of high reliability. Therefore, the reliability of the sub- jective evaluation questionnaire is good. The results are of the reference value. Analysis of validity is conducive to determining whether the item design of the questionnaire is reasonable. Through factor analysis, it is verified that the KMO value of the questionnaire data is 0.603, greater than 0.6, indicating that the information content of the research item can be effectively extracted.
By taking the “number of auxiliary stimulus particles”as the grouping vari- able, Shapiro-Wilk (S-W) test was used to verify the normality of the scoring data of efficiency, effectiveness, satisfaction, and fatigue in user experience. The results are shown in Table 2.
The significance of the scores of the four subjective evaluations was mostly less than 0.05, indicating that the user experience scoring data did not con- form to the normal distribution. The Friedman test results showed that the
Table 2. Analysis results of the S-W test.
Table available in the original PDF paper.
Stimuli number Sample size Shapiro-Wilk test Efficiency Effectiveness Satisfaction Fatigue Four 20 0.002** 0.020* 0.037* 0.023*
significance of the four subjective evaluations was less than 0.05, as shown in Table 3, indicating that the change in the “number of auxiliary particles” had a significant impact on the scoring results.
The scores of efficiency, effectiveness, satisfaction and fatigue at different numbers of auxiliary stimulus particles are shown in Figures 4a, 4b, 4c, and 4d. In the efficiency score, when the number of auxiliary stimuli particles was twelve, the median score was 5; When the number was four, eight, sixteen, and infinite, the median score was 6 In the effectiveness score, when the num- ber of auxiliary stimuli particles was four, eight and twelve, the median score was 5; When the number was sixteen or infinite, the median score was 6. In the satisfaction score, when the number of auxiliary stimuli particles was twelve, the median score was 5; When the number was sixteen, the median score was 5.5; When the number was four, eight, and unlimited, the median score was 6. In the fatigue score, when the number of auxiliary stimuli was twelve, the median score was 4; When the number was eight or sixteen, the median score was 5; When the number was four or infinite, the median score was 6.
DISCUSSION
In terms of recognition success rate, as the number of auxiliary stimulus par- ticles increases, the recognition success rate fluctuates but is not significant, which is of little significance to the judgment of experimental performance. This may be because the recognition success rate is greatly affected by indi- vidual factors (Benda and Volosyak, 2020), which are related to the subjects’ experimental status, experimental environment, and other factors.
In terms of reaction time, when the number of auxiliary stimulus parti- cles was four and infinite, the average reaction time was shortened. It was
Enhancement Characteristics of Visual Stimulus Elements in SSVEP-BCI System 961
significantly lower than the reaction time of the number of auxiliary stimu- lus particles at the other three levels. This may be due to the change in the number of auxiliary stimulus particles in the experiment. The auxiliary sti- mulus particles and the central stimulus formed a different arrangement. The change of this arrangement may affect the subject’s attention to the stimulus elements, thus affecting the reaction intensity of the subject’s SSVEP signal. This is consistent with Tello et al.’s conclusion (2015) that the intensity of the SSVEP signal is significantly affected by the subjects’ attention. After analy- zing the experimental data, it can be seen that the change in the number of auxiliary stimulus particles had a significant impact on the average rea- ction time, which further proves the above conjecture. With the increase in the number of auxiliary stimulus particles, the reaction time increased first and then shortened. According to the mental rotation results of Aldridge and Flores (1988), the response time increases with the complexity of graphic stimuli. This may be the reason that when the number of auxiliary stimulus particles increased, the response time first rises and then shorten. The increase in the number of auxiliary stimulus particles in the experiment makes the sti- mulus elements have a higher image complexity, leading to a longer reaction time and a poorer system recognition efficiency; With the further increase of the number of auxiliary stimulus particles and their gradual connection into a whole ring, the image complexity of the stimulus elements decreased, the reaction time of the subjects shorten, and the system recognition efficiency improved.
Based on the above analysis of ergonomic experimental data, it can be seen that with the increase in the number of auxiliary stimulus particles, the rea- ction time of subjects to identify stimulus elements showed a trend of rising first and then shortening. When the number of auxiliary stimulus particles was infinite, the reaction time was the shortest and the success rate was high.
According to the subjective evaluation results, the change in the number of auxiliary stimulus particles had a significant impact on the subjects’ sco- res on the efficiency, effectiveness, satisfaction, and fatigue of the SSVEP-BCI system. As shown in Figure 4, the overall trend of the median scores of the four items is that with the increase in the number of auxiliary stimulus par- ticles, the median score decreases. When the number of auxiliary stimulus particles was twelve, the score was the lowest. With a further increase in the number of auxiliary stimulus particles, the median score increased, When the number was infinite, the median score reached the highest. The reasons for this result may be: (1) With the increase of the number of auxiliary sti- mulus particles, the complexity of graphic stimulus elements increased, the subjects’ attention was distracted, and the reaction time increased, leading to a decrease in efficiency and effectiveness scores; Then, after the auxiliary particles continued to increase to a certain number, the auxiliary particles gra- dually connected into a whole, the complexity of the graph decreased, and the efficiency and effectiveness scores were therefore improved. The research conclusions of Aldridge (1988) can support this assumption. (2) In this expe- riment, when the number of auxiliary stimulus particles with low-frequency flicker increased, the visual load of the subjects increased, leading to a decline
Figure 4: Box plot of four subjective scores of different numbers of auxiliary stimulus.
in the fatigue score and satisfaction score of the subjects. The research conclu- sions of Won et al. (2016) can support this assumption; Then, as the number continued to increase, the spacing between the auxiliary stimulus particles decreased. According to the Gestalt Integrity Principle (Rock and Palmer, 1990), it is easier for the subjects to treat it as a whole flashing stimulus ele- ment in their cognitive psychology, and the fatigue of the stimulus element also decreases. This may also be the reason why the fatigue degree and sati- sfaction score were optimal when the particles tended to become infinite into rings.
Through subjective evaluation, the study found that with the increase in the number of auxiliary stimulus particles, the user experience score showed a trend of first falling and then rising. When the number of auxiliary stimulus particles reached infinity, the user experience score of effectiveness, efficiency, satisfaction, and fatigue was the best.
CONCLUSION
Through the analysis of response time results, it can be seen that there is a significant difference between the response times of different numbers of auxiliary stimulus particles. When the number of auxiliary stimulus parti- cles is infinite, the response time of the experimental task is the shortest and the system recognition efficiency is the highest. Combined with subje- ctive evaluation, the change in the number of auxiliary stimulus particles has an impact on the user experience score. Corresponding to the ergono- mics experimental results, when the number of auxiliary stimulus particles reaches infinity, the user experience score is the best. Based on the above experimental results, the recognition efficiency of the system is the highest
Enhancement Characteristics of Visual Stimulus Elements in SSVEP-BCI System 963
when the number of auxiliary stimulus particles is infinite, which can be recommended.
The conclusion of this experiment can provide a certain parameter design basis for SSVEP-BCI interface design. It also helps to further improve the user experience when using the SSVEP-BCI system. There are some limitations in this study, and the following studies will continue to explore the ergonomics problems in terms of different arrangements of auxiliary stimulus particles and the contrast of brightness and color between auxiliary stimulus particles and central stimulus; In addition, specific operation tasks and application scenarios need further research.
ACKNOWLEDGMENT
This work was supported jointly by National Natural Science Foundation of China (No. 72171044, 71801037), Aerospace Science Foundation of China (No. 20200058069002), ZhiShan Scholar Program of Southeast University (2242022R40004).
REFERENCES
Aldridge, J. W. and S. A. Flores (1988). Random figures and mirror images with variable complexity, size, and orientation in Applesoft Basic. Behavior Research Methods Instruments & Computers 20(3): 313–316. Benda, M. and I. Volosyak (2020). Comparison of Different Visual Feedback
Methods for SSVEP-Based BCIs. Brain Sciences 10(4): 13. Chen, X. Y., Z. J. Cheng, C. M. Hu, T. F. Liang and Q. Liu (2020). Steady-state visual evoked potential and its applications in studying visual selective attention. Chinese Science Bulletin-Chinese 65(24): 2601–2614. Chu, L., J. Fernandez-Vargas, K. Kita and W. W. Yu (2016). Influence of Stimulus
Color on Steady State Visual Evoked Potentials. 14th International Conference on Intelligent Autonomous Systems (IAS), Shanghai Jiao Tong Univ, Shanghai, PEOPLES R CHINA, Springer International Publishing Ag. Gao, X., Y. Wang, X. Chen and S. Gao (2021). Interface, interaction, and intelligence in generalized brain–computer interfaces. Trends in Cognitive Sciences 25(8): 671–684. Han, C. C., G. H. Xu, J. Xie, C. Y. Chen and S. C. Zhang (2018). Highly Intera-
ctive Brain-Computer Interface Based on Flicker-Free Steady-State Motion Visual Evoked Potential. Scientific Reports 8: 13. Kapgate, D., D. Kalbande and U. Shrawankar (2020). An optimized facial stimuli paradigm for hybrid SSVEP+P300 brain computer interface. Cognitive Systems Research 59: 114–122. Li, Y. Q., J. H. Pan, F. Wang and Z. L. Yu (2013). A Hybrid BCI System Combining
P300 and SSVEP and Its Application to Wheelchair Control. Ieee Transactions on Biomedical Engineering 60(11): 3156–3166. Mouli, S. and R. Palaniappan (2017). Toward a reliable PWM-based light-emitting diode visual stimulus for improved SSVEP response with minimal visual fatigue. Journal of Engineering-Joe: 6. Niu, Y., X. Li, W. Yang, C. Xue, N. Peng and T. Jin (2023). Smooth Pursuit Study on
an Eye-Control System for Continuous Variable Adjustment Tasks. International Journal of Human–Computer Interaction 39(1): 23–33.
Niu, Y. F., J. H. Wang, J. C. Wu, C. Q. Xue and W. J. Yang (2022). Ergonomic Study on Element Size and Spacing of Brain Computer Interface Based on SSVEP. Journal of Electronics & Information Technology 44(2): 455–463. Renard, Y., F. Lotte, G. Gibert, M. Congedo, E. Maby, V. Delannoy, O. Bertrand
and A. Lecuyer (2010). OpenViBE: An Open-Source Software Platform to Design, Test, and Use Brain-Computer Interfaces in Real and Virtual Environments. Presence-Teleoperators and Virtual Environments 19(1): 35–53. Rock, I. and S. Palmer (1990). THE LEGACY OF GESTALT PSYCHOLOGY.
Scientific American 263(6): 84–90. Tello, R. J. M. G., S. M. T. Müller, A. Ferreira and T. F. Bastos (2015). Comparison of the influence of stimuli color on Steady-State Visual Evoked Potentials. Research on Biomedical Engineering 31(3): 218–231. Won, D.O., H.J.Hwang, S.Dahne, K.R.Muller and S.W.Lee (2016). Effect of higher
frequency on the classification of steady-state visual evoked potentials. Journal of Neural Engineering 13(1): 11. Xu, G., F. Zhang, J. Xie, Y. Li, C. Han and L. Li (2015). Brain-Computer Interface Paradigms and Signal Processing Strategy for Steady State Visual Evoked Potential. Journal of Xi’an Jiaotong University 49(6): 1. Ya-feng, N., L. Jin, C. Jia-qi, Y. Wen-jun, Z. Hong-rui, H. Jia-xin, X. Lang, W. Jia-hao,
M. Guo-rui, H. Zi-jian, X. Cheng-qi, Z. Xiao-zhou and J. Tao (2022). Resea- rch on visual representation of icon colour in eye-controlled systems. Advanced Engineering Informatics 52: 24. Zambalde, E. P., G. Jablonski, M. B. de Almeida and E. L. M. Naves (2018). Evaluation of the Target Positioning in a SSVEP-BCI. 26th Brazilian Congress on Biomedical Engineering (CBEB), Armacao de Buzios, BRAZIL, Springer. Zhang, X., G. H. Xu, X. Zhang and Q. Q. Wu (2018). A Light Spot Humanoid
Motion Paradigm Modulated by the Change of Brightness to Recognize the Stride Motion Frequency. Frontiers in Human Neuroscience 12: 11.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-45-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional