Mingyu Ai, Xiaonan Yang, Shuai Wang, Wanni Wei, Hongwei Niu, Haonan Fang (2024). Effects of different visual feedback mechanisms on eye-controlled interaction in a vibration environments. In: Usability and User Experience. AHFE Open Access, vol. 156, pp. 87–96. https://doi.org/10.54941/ahfe1005418
Eye-controlled interaction is a human-computer interaction technology that users can complete the interactive behavior by moving eyes to control equipment or devices, which has the advantages of human-centered, natural and direct. During the process of eye-controlled interaction, effective visual feedback mechanisms can help users accurately understand the operation process. A vital part of eye-controlled interaction is to confirm the user’s intent to select the target character, so it is important to study the effects of different visual feedback mechanisms under the selection task. However, there is a gap between lab environment and real-world application, that contain vibration environment. This study aimed to compare the performance of different visual feedback mechanisms in vibration environments. 20 participants are required to complete the selection task of time delay selection mechanism and selection sub-selection reconfirmation mechanism in three states: static, low vibration, and high vibration environments. The results indicate that the various visual feedback mechanisms have distinct impacts on selection accuracy. Regardless of the vibration environment, the accuracy of the time delay selection mechanism surpasses that of the selection sub-selection reconfirmation mechanism; Vibration exerts an adverse influence on visual fatigue, with more intense vibration leading to a shorter occurrence time of visual fatigue. The time delay selection mechanism is a widely favored visual feedback mechanism, offering a lower subjective workload.
Eye-controlled interaction, a kind of human-computer interaction, means that users can complete the interactive behavior by moving only through their eyes and the computer. It uses the eye movement information as the input to realize the control of the equipment or device and make users com- plete the task without contact in complex situations (Lv and Wang, 2012). As an eye-based interaction, it has its unique advantages compared with other interaction methods. First, it offers the prospect of reducing learning time and locating rapidly by providing a natural and direct means of pointing at a dis- played object on-screen. Second, relying on the eyes to complete the task, the
hands can cooperate to complete other operation, which is suitable for the operation scenario that requires synchronous processing of multiple tasks. Third, users with limited physical movement and motor impairment gener- ally retain good ocular motor control, and so devices based on eye-movement may be used by a larger range of those ‘special’ users (Huang and Cheng et al., 2021). Previous studies have shown that eye-controlled interaction is superior to mouse operation in time consumption and accuracy of selection tasks, and it can meet the operation requirements of special scenes (Bednarik and Gowases et al., 2009). Therefore, eye-controlled interaction has gradually come to the attention of the public and is more and more used in human-computer interaction. In the process of eye-controlled interaction, visual feedback can help users understand the operation progress of the system. The necessity of visual feedback is well-known, visual feedback is a meaningful cue for the user to know the current state of interactive systems and infer whether they are responding to the user’s action as the anticipation or not (Zhang and Feng et al., 2011). Users could accurately realize which target was “captured” by their gaze through explicit visual feedback, and avoid unexpected commands being activated accidentally (Istance and Spinner et al., 1996). According to previous research on eye-machine interaction interfaces, the whole process of eye-controlled interaction is divided into three stages: recognition, selection and triggering, A vital part of eye-controlled interaction is confirmming the user’s intent to select the target character, so it is important to study the effects of different visual feedback mechanisms under the selection task.
Selection in eye-controlled interaction refers to the whole process of the user focusing on an object on the interface and selects it. Traditional visual feedback mechanisms in selection tasks have two options: time delay selec- tion mechanism and selection sub-selection reconfirmation mechanism. The time delay selection mechanism uses gaze to make a selection of the target on the screen. The selection sub-selection reconfirmation mechanism is to transfer the gazing object to the pop-up subobject corresponding to the tar- get object. They also came to a conclusion that the satisfaction of marking menu with selecting sub-selection recon-firmation mechanism is low, but its fault tolerance is higher (Istance and Spinner et al., 1996). These studies are done in a static environment, which is somewhat different from real-world environment, that contain vibration environment.
Vibration can cause physiological discomfort, and cognitive and behav- ioral difficulties (Baker, 2013; Munafo and Wade et al., 2015; Salmon and Lenné et al., 2011). It may also affect human-computer interaction efficiency and performance. Ahmad et al. pointed out that vibration could result in erroneous user input unintentional or false selections. In the process of rec- tify those incorrect selections, e.g. doing the same task repeatedly, can tie up further of the user’s attention, which can increase the likelihood of task inef- ficiency (Ahmad and Langdon et al., 2018). In the process of eye-controlled interaction, the acquisition of eye movement information is affected by vibra- tion, resulting in inaccurate positioning, and long hours of interaction with the screen can lead to visual fatigue. However, in a new environment, a lot of selection tasks still need to be completed by users, different visual feedback
Effects of Different Visual Feedback Mechanisms on Eye-Controlled Interaction 89
mechanisms will also cause different effects in the vibration environment. This study aimed to compare the performance of time delay selection mech- anism and selection sub-selection reconfirmation mechanism in vibration environments.
METION Participants
Eligible participants were recruited if their uncorrected visual acuity (UCVA) or corrected visual acuity (CVA) of 1 and above, no eye disease, no percep- tual disorder, dyskinesia or mental impairment and other problems. A total of 16 participants completed our experiment. The age of the participants was mainly between 20–35 years old, and the average age is 23. During the exper- iment, the participants required not to wear contact lenses or false eyelashes, and they should ensure sufficient sleep on the day before the experiment to avoid overusing of eyes. In addition, the Ethics Committee of the university approved the present study.
Apparatus Three vibration environments were simulated on a six-degrees-of-freedom platform with ranges as: sway ± 19 cm, heave ± 19 cm, surge ± 19 cm, pitch ± 13
, yaw ±13
, roll ±13
(see Figure 1). In order to simulate the real environment and generate more disordered and unpredictable vibration, we set four sets of parameters in low/high vibration environment. The vibration range and period of each set of parameters are different. Each set of parame- ters has its own duration, and will be replaced by the next set of parameters after the duration.
The eye tracker (model: tobii spectrum 150) is used to record the eye movement data (see Figure 2), Participants interacte with it and complete the selection task. A program was written in python to select tasks. A computer application was developed in python for selection task.
Figure 1: The six-degrees-of-freedom platform.Figure 2: Tobii spectrum 150.
Design Our study with vibration, visual feedback mechanisms as the within-subjects factors: three vibration environments (static, low vibration, and high vibra- tion environments) and two visual feedback mechanisms (time delay selec- tion mechanism and selection sub-selection reconfirmation mechanism). The dependent variables included two objective measures (select accuracy and occurrence time of visual fatigue) and two subjective measures (preference rating and subjective workload). Select accuracy was calculated as the pro- portion of correct choice; a correct choice was recorded if the participants found the correct option and completed the selection within the prescribed time, occurrence time of visual fatigue was measured as the time when the participant visually fatigued and requested a pause in the current experiment. Preference rating was measured using the questionnaire, with a rating range from 1 (don’t like it very much) to 7 (like it very much). Subjective work- load was measured using the NASA-TLX scale, a commonly used subjective technique for work-load assessment
The NASA-TLX scale comprised six subscales for measuring mental demand, physical demand, temporal demand, performance, effort and frustration, with a rating range from 0 (absolutely no workload) to 20 (maximum workload) (Hart, 2006).
Materials and Tasks The experiment was carried out in a case of an interface of 24 in, and a res- olution of 1920 px × 1080 px. Different combinations of the foreground and background colours of the icons have different effects on user experi- ence, and icons composed of black and yellow more popular with partcipants (Shieh and Ko, 2005). Yellow icons appeared to have higher visual recogni- tion and legibility against dark backgrounds (Humar and Gradisar et al., 2014). Therefore, the background color of the experiment is black, and the icon is a 150 px × 150 px yellow square border with yellow English letters in the center. Sight dwell time for time delay selection mechanism is set to 1000 ms, the translucent white square will dynamically fill from the center to the periphery over time, the iron border changes from yellow to red when 1000 ms is reached. Triggering the selection sub-selection reconfirmation mech- anism requires first gazing at the iron about 300 ms, which has the same animation effect as time delay selection mechanism. Then the subobject (a yellow square with a size of 75 px × 75 px) will be popped up at 20 px on
Effects of Different Visual Feedback Mechanisms on Eye-Controlled Interaction 91
the right side. The visual attentional shift from the object to the subobject occurred within 500 ms, followed by gazing the yellow square for 200 ms, the red color will fill the subobject, and the yellow border will turn red to indicate selection. Two visual feedback mechanisms are shown in Figure 3 and 4.
Participants are required to complete the selection task of time delay selec- tion mechanism and sub-selection reconfirmation mechanism in three states: static, low vibration, and high vibration environments. At the beginning of the selection task, a letter required to be selected will be shown. After 6500 ms, 5 × 4 options will appear on the screen. Participants should use visual feedback mechanisms to find and successfully select the right letter option. Repeat this step to complete the selection task until subjective visual fatigue occurs, it’s failure if the selected letter is wrong or unsuccessful within the specified time. Repeat this step until participant’s visual fatigue.
Procedures Before the experiment, participants were given a detailed introduction to the selection task in the previous section, signed the consent form and fill in a personal information questionnaire, which includes name, age, gen- der and other information. Then participants were required to sit on chairs on six-degrees-of-freedom platform for calibration. They had to undergo a short training process until they understand how to complete the selec- tion task using two visual feedback mechanisms. It is important to note that participants will undergo 6 experimental sessions (3 vibration environ- ments × 2 visual feedback mechanisms). To ensure the accuracy of eye tracker data capture, calibration is required before each session. Additionally, they are required to complete preference and NASA-TLX questionnaire after the experiment, followed by a period of rest to alleviate visual fatigue. We recom- mend that participants close their eyes or gaze into the distance during this rest period. At the end of the experiment, each participant was given RMB 100 for their participation. The experimental procedure is shown in Figure 5.
Figure 5: Experiment procedure.
Effects of Different Visual Feedback Mechanisms on Eye-Controlled Interaction 93
RESULT
Repeated measures analyses of variance (ANOVAs), validated by Mauchly’s sphericity test, were performed to examine the effects of vibration environ- ments and different visual feedback mechanisms. The result of two-ways ANOVAS showed that vibrations environments had significant affected on occurrence time of visual fatigue, F(2,94)=6.232, P = 0.003<0.05, but had not significant differences on preference rating and subjective work- load, F(2,94)=1.028, P = 0.362; F(2,94)=0.240, P = 0.787; different visual feedback mechanisms had significant affected on preference rating and sub- jective workload, F(1,94)=44.508, P<0.001; F(1,94)=4.629, P = 0.034, however, different visual feedback mechanisms hadn’t significant affected on occurrence time of visual fatigue, F(1,95)=0.024, P = 0.877. vibrations envi- ronments and different visual feedback mechanisms had significant effects on select accuracy, F(2,94)=3.205,P = 0.045 and F(1,95)=24.969, respectively. At the same time, we also found that there were no significant interaction effects between the vibration environments and the different visual feed- back mechanism on the select accuracy, occurrence time of visual fatigue, preference rating ane subjective workload for selection task (all P>0.05).
Figure 6 illustrates the trends and comparisons of the accuracy of selection, occurrence time of visual fatigue, preference rating and subjective workload in response to variations in the different vibration environments under two visual feedback mechanisms: time delay selection mechanism and selection sub-selection reconfirmation mechanism.
Figure 6: The trends and comparisons of the (a) select accuracy, (b) occurrence time of visual fatigue, (c) preference rating and (d) subjective workload in response to varia- tions in the different vibration environments under two visual feedback mechanisms.
DISCUSSION
The visual feedback mechanism significantly influences the accuracy of selec- tion. Regardless of the vibration environment, the time delay selection mech- anism consistently outperforms the selection sub-selection reconfirmation mechanism in terms of selection accuracy. We posit that in densely populated menu interfaces, the learnability of the selection sub-selection reconfirmation mechanism is lower than that of the time delay selection mechanism. The selection sub-selection reconfirmation mechanism requires the final selection to be made by gazing on a sub-target, which is typically smaller in size and may consequently lead to decreased accuracy. Furthermore, inherent inac- curacies in eye-tracking devices are exacerbated in vibrational environments due to the increased movement of the human body, particularly the head. The intricacies of the selection sub-selection reconfirmation mechanism also render it more prone to erroneous operations and subsequent deselection, thereby resulting in lower selection accuracy compared to the time delay selection mechanism. The vibrational environment significantly influences the accuracy of selection. Under both mechanisms, accuracy of selection is lower in a vibrational environment. Vibrations can lead to errors in data acquisition by the eye tracker. Sudden vibrations can cause abrupt shifts in the user’s fixation point, thereby diminishing the accuracy of selection.
With the changing vibrational environment, the occurrence visual fatigue for both selection mechanisms exhibits a decreasing trend. Vibration on the flight deck increased subjective fatigue (Dodd and Lancaster et al., 2014), which may in turn exacerbate visual fatigue. Furthermore, as the intensity of vibrations becomes more pronounced, the occurrence of visual fatigue shortens. This results in the shortest onset of visual fatigue for both selection mechanisms in high vibration environment, measuring 444.3 s (time delay selection mechanism) and 448.1 s (selection sub-selection reconfirmation mechanism) respectively.
The user satisfaction with selecting sub-selection recon-firmation mecha- nism is low (Hou and Zhang et al., 2019), in consistent with previous studies, and within the three environments set in this experiment (static, low vibra- tion and high vibration environments), the time delay selection mechanism is favored by the subjects. In our analysis of subjective workload, we found that the selection sub-selection reconfirmation mechanism resulted in higher subjective workload in all three environments. We believe that the decrease in accuracy and usability will lead to an even greater subjective workload. These findings prove that the user pays more attention to the select accuracy and ease-using when using the eye-controlled interaction.
CONCLUSION
The experiment investigated the impact of vibration environments and visual feedback mechanisms on the select accuracy, occurrence time of visual fatigue, preference rating, and subjective workload in eye-controlled inter- actions. Generally, the time delay selection mechanism exhibited higher accuracy and preference, and lower subjective workload. As the intensity of vibration increased, the occurrence time of visual fatigue shortened, and the
Effects of Different Visual Feedback Mechanisms on Eye-Controlled Interaction 95
accuracy under vibration environments were lower than that under static. Vibration led to visual fatigue, but different visual feedback mechanisms did not significantly affect visual fatigue. Moreover, the impact of visual feedback mechanisms on accuracy was more significant than that of vibra- tion. Considering the comprehensive measurement indicators, the time delay selection mechanism was the superior choice in both static and vibrational environments. When selecting visual feedback mechanisms in different vibra- tion environments, the impact of visual fatigue can be less considered, with select accuracy, subjective workload, and preference being used as the cri- teria. This study provides theoretical support for selecting feedback forms in eye-controlled interactions in vibrational environments. This study pro- vides theoretical support for the selection of visual feedback mechanisms in eye-controlled interactions in vibrational environments.
ACKNOWLEDGMENT
The authors would like to thank the Foundation Strengthening Project (2022- JCJQ-JJ-0858).
REFERENCES
Ahmad, B. I. and P. M. Langdon, et al. (2018). Stabilising Touch Interactions in Cockpits, Aerospace, and Vibrating Environments, Cham, Springer International Publishing. Baker, W. D. R. (2013). The influence of whole-body vibration and postural support
on activity interference in standing rail passengers. Bednarik, R. and T. Gowases, et al. (2009). “Gaze interaction enhances problem solving: Effects of dwell-time based, gaze-augmented, and mouse interaction on problem-solving strategies and user experience.” Journal of Eye Movement Research 3. Dodd, S. and J. Lancaster, et al. (2014). “Touch Screens on the Flight Deck: The
Impact of Touch Target Size, Spacing, Touch Technology and Turbulence on Pilot. Hart, S. G. (2006). “Nasa-Task Load Index (NASA-TLX); 20 Years Later.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50 (9): 904–908. Hou, W. and B. Zhang, et al. (2019). Applicability Study of Eye Movement Menu
based on Analytic Hierarchy Process, Cham, Springer International Publishing. Huang, W. and B. Cheng, et al. (2021). “Ergonomics research on eye–hand control dual channel interaction.” Multimedia Tools and Applications 80 (5): 7833–7851. Humar, I. and M. Gradisar, et al. (2014). “The impact of color combinations on the
legibility of text presented on LCDs.” Applied Ergonomics 45 (6): 1510–1517. Istance, H. O. and C. Spinner, et al. (1996). Providing motor impaired users with access to standard Graphical User Interface (GUI) software via eye-based interaction. Lv, B. and H. Wang (2012). The Techniques of Eye Movement Operating System
Based on Human Eye Movements Characteristics. Frontiers in Computer Edu- cation. S. Sambath and E. Zhu. Berlin, Heidelberg, Springer Berlin Heidelberg: 949–956. Munafo, J. and M. G. Wade, et al. (2015). Subjective reports and postural performance among older adult passengers on a sea voyage. United Kingdom, Taylor & Francis. 27: 127–143.
Salmon, P. M. and M. G. Lenné, et al. (2011). “The effects of motion on in-vehicle touch screen system operation: A battle management system case study.” Trans- portation Research Part F: Traffic Psychology and Behaviour 14 (6): 494–503. Shieh, K. and Y. Ko (2005). “Effects of Display Characteristics and Individual Differ-
ences on Preferences of VDT Icon Design.” Perceptual and Motor Skills 100 (2): 305–318. Zhang, X. and W. Feng, et al. (2011). Effects of Different Visual Feedback Forms on Eye Cursor’s Stabilities, Berlin, Heidelberg, Springer Berlin Heidelberg.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Usability and User Experience
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-32-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional