Yamato Miyajima, Chun Xie, Itaru Kitahara (2024). Video Generation Method Unconsciously Gaze-Guiding for a Passenger on Autonomous Vehicle with Controlling Color and Resolution. In: Accessibility, Assistive Technology and Digital Environments. AHFE Open Access, vol. 121, pp. 9–18. https://doi.org/10.54941/ahfe1004609
While automobiles are an indispensable means of transportation in our daily lives, many people suffer from motion sickness. Various symptoms such as nausea, fatigue, and vertigo develop when people become motion sickness. There are several theories on the cause of motion sickness, including the sensory confusion theory, which states that the brain becomes confused due to discrepancies in information from sensory organs such as vision, vestibular sensation, and somatosensory perception, and the eye movement theory, which states that motion sickness is caused by optokinetic nystagmus. According to these theories, gaze movements are deeply related to the cause of motion sickness. Moreover, existing research has shown that motion sickness can be alleviated by looking at a particular object. Therefore, it is considered that motion sickness can be reduced by providing gaze guidance that encourages fixation. However, explicit gaze guidance that gives visual stimuli such as flashing lights may greatly disturb the user experience while riding in a car, so it is preferable to provide unconscious gaze guidance in which the user does not feel the intention of gaze guidance. This research aims to realize unconscious gaze guidance methods to reduce motion sickness. We proposed three types of unconsciously gaze-guided video generation methods. The first method is a resolution-controlled gaze guidance method. This method reduces the resolution by applying a smoothing process to the unguided areas and guides the gaze to the unprocessed high-resolution areas. The second method is the color-control method. This method is based on the idea that regions with high saliency in a visual saliency map are preferentially gazed at. This method guides the gaze by changing the hue and brightness of the image so that the saliency of the area to be guided becomes higher in the image. The third method is a combination of resolution-control method and color-control method. By combining these two methods, we hypothesized that we can complement the problems of both methods and achieve a stronger gaze guidance effect. We expected that unconscious gaze guidance is possible, since neither of these methods provides explicit visual stimuli. Experimental evaluations to verify the effectiveness of each method are conducted. In the experiment, we measured the gaze position while watching the generated video using an HMD with a gaze measurement function. We compared the success rate of gaze guidance between movies generated by the proposed method and movies with explicit stimuli that were shown to be effective in reducing motion sickness in existing studies. The gaze guidance rate of the movie generated with the resolution-control method was 53%, that of the movie generated with the color-control method was 34%, and that of the movie generated with the combination of the two methods was 70%. These results show that the combination of the two methods has the closest gaze guidance effect to explicit stimuli. We also conducted a questionnaire survey of the intensity with which participants felt they were being gaze guided while watching the movie. All participants felt the strongest intention of guidance with the explicit stimuli, whereas all participants felt less the intention with the proposed methods than with the explicit stimuli. Furthermore, the color-control was almost imperceptible to the intention. These findings indicate that there is a correlation between the strength of the intention of the guidance and the guidance effect. As the result, it can be considered that the guidance effect and naturalness are controlled by adjusting the degree of changes.
While automobiles enhance the convenience of daily transportation, many people suffer from motion sickness. Individuals prone to motion sickness may find themselves restricted from traveling or commuting to work due to symptoms such as nausea, fatigue, and dizziness. According to the sensory conflict theory (Reason and Brand, 1975), motion sickness is caused by con- fusion in the brain resulting from discrepancies in information from sensory organs, including vision, vestibular sensation, and somatosensory perception. The eye movement theory (Ebenholtz et al., 1994) suggests that the cause of motion sickness is optokinetic nystagmus, which is generated by a cycle of slow eye movements to follow a moving object and quick eye movements to reset the eye position. Optokinetic nystagmus is easily induced by objects with significant movement, such as nearby objects and oncoming cars, especially when observed from inside a rapidly moving vehicle.
Visually Induced Motion Sickness (VIMS) causes symptoms akin to those of traditional motion sickness. Since the sensory conflict theory and the eye movement theory are prevalent in explaining the mechanism of VIMS (Flanagan et al., 2002), strategies to mitigate VIMS are believed to also dimin- ish motion sickness. For instance, Webb and Griffin (2002) demonstrated that VIMS can be alleviated by focusing on a specific object within a visual scene. This research aims to alleviate motion sickness by guiding the user’s gaze towards an object while in a moving car. However, explicit gaze guid- ance using visual cues, like flashing lights, can significantly disrupt the in-car experience. We believe that subtle gaze guidance, which is less noticeable to the user, would be more appropriate.
There are two methods for subtly directing gaze: color control (Hagiwara et al., 2011, Takimoto et al., 2015) and resolution control (Hata et al., 2015). These techniques can guide the gaze without overt visual stimuli and poten- tially reduce motion sickness. This research will test the effectiveness of gaze guidance methods using resolution and color control on videos displayed on vehicle windows to determine their impact on motion sickness. Additionally, we aim to discover if combining resolution and color control can enhance the gaze guidance effect. Given that automated vehicles may use the wind- shield as a display, it is feasible to modify and present the view seen through the vehicle windows. Our proposed method leverages this feature and could offer a novel way to combat motion sickness.
RELATED WORKS Motion Sickness Reduction by Gazed Guidance
Several theories have been suggested to explain the causes of motion sick- ness, with the sensory conflict theory and eye movement theory identifying visual information and eye movements as primary factors. Visually Induced Motion Sickness (VIMS) and Virtual Reality (VR) sickness serve as instances of conditions precipitated by these elements. In the case of VIMS, research by Webb and Griffin (2002) has demonstrated that inertia and nystagmus can be mitigated by focusing on a stable point within the visual field. Similarly, Miura et al. (2018) found that this fixed-point gazing technique also reduces sickness in VR environments. Further investigation into whether it is more effective to maintain gaze on a specific object or in a specific direction has revealed that the former strategy is relatively more efficacious in alleviating sickness.
Drawing on this body of research, gazing at a specific object shows promise as a method for reducing motion sickness, given its shared causative factors with VIMS and VR sickness. However, traditional methods of gaze guid- ance, such as the establishment of a fixed viewpoint, tend to disrupt the user experience due to their explicit nature.
Gaze Guidance via Image Processing Gaze guidance through image processing can be categorized into two distinct methods: explicit and unconscious gaze guidance. The explicit method directs
Video Generation Method Unconsciously Gaze-Guiding 11
the user’s attention by presenting clear visual cues, such as fixed points or arrows within the image. Despite its effectiveness, this approach has been critiqued for feeling artificial and intrusive, as reported by Peet et al. (2012).
Conversely, unconscious gaze guidance subtly steers the user’s gaze without their conscious awareness, taking advantage of the viewer’s visual perception characteristics. Itti et al. (1998) proposed utilizing visual saliency, an index reflecting the likelihood of attracting a person’s gaze, to create a map that guides attention by integrating image aspects like luminance, hue, and edge direction.
Color control, as a practical application of visual saliency for gaze guid- ance, has been explored by prior research. Hagiwara et al. (2011) designed a method that alters an image’s hue and brightness to highlight specific regions, relying on the principle that areas with greater prominence on a visual saliency map capture attention more readily. Takimoto et al. (2015) further refined this approach by employing the Lab* color system for more natural color adjustments.
Another method involves resolution control, as investigated by Hata et al. (2015), who advocated for guiding the eyes naturally by degrading the res- olution in all but the target regions. Their findings indicate that this can effectively direct attention, although it may not work in texture-less areas.
The cumulative evidence from these studies suggests that gaze guidance can be effectively achieved without overt visual cues. This current research extends the application of these unconscious gaze guidance techniques to videos displayed on vehicle windows, aiming to compare their effectiveness with explicit guidance methods. Additionally, we seek to determine whether combining color control and resolution control can enhance the overall gaze guidance effect.
GAZE GUIDANCE IMAGE PROCESSING This research implements two unconscious gaze guidance methods, a resolu- tion control method and a color control method, and verify the effectiveness of the guidance. In addition, we propose a method that combines these two methods.
Gaze Guidance Method Using Resolution Control The resolution control method for gaze guidance builds upon the technique developed by Hata et al. (2015). This method involves a selective blurring process that reduces the resolution of non-target areas, thereby drawing the viewer’s gaze towards regions of higher resolution. The degree of blur is precisely controlled by adjusting the kernel size of the moving average fil- ter applied during the smoothing process. To ensure a gradual transition between areas of differing resolution, a Gaussian filter is implemented along the boundaries. An illustrative example of this gaze guidance technique is depicted in Figure 1. In Figure 1(a), the red car is designated as the focal point for gaze guidance. Figure 1(b) displays the outcome after processing, where the red car is depicted with high resolution while the surrounding area
is shown with reduced resolution. The intent behind this technique is to nat- urally direct the viewer’s attention to the red car within the high-resolution zone.
Figure 1: Gaze guidance processing results with resolution control. The red car in (b) is depicted with high resolution while the surrounding area is shown with reduced resolution.
Gaze Guidance Method Using Color Control The color control method for gaze guidance employs the Lab* color system, adapted from the approach by Takimoto et al. (2015). The process is illus- trated in Figure 2, which outlines the flowchart of the gaze guidance method. The first step involves selecting the region that requires attention (Figure 2: Select region). Subsequently, a visual saliency map is generated based on the Lab* color system (Figure 2: Saliency map). Using this saliency map, the L*, a*, and b* values of the original image are altered to enhance the visual saliency of the target area. This adjustment process is repeated iteratively until a predetermined termination condition is met. In the method proposed by Takimoto et al., the termination condition could be the average saliency in the target region, the maximum saliency value, or a specific number of iterations. For this research, we employ a fixed number of iterations as the termination condition to maintain consistency between frames and prevent flickering.
Figure 3 compares the imagery before and after the application of the color control gaze guidance. The red car, which is the focal point of the guidance, appears more vividly red following the process. Conversely, the saturation of regions outside the guidance target is diminished. This alteration not only increases the saliency of the red car but also decreases the saliency of the surrounding area, thereby indicating a higher likelihood that the gaze will be drawn to the red car.
Gaze Guidance Method Combining Resolution Control and Color Control Gaze guidance through resolution control is executed by modifying the edge components within the visual saliency map, while color control achieves this by altering the luminance and hue components. It is hypothesized that a more potent gaze guidance effect can be realized by simultaneously manipulating all three components—luminance, hue, and edge. To this end, a combined
Video Generation Method Unconsciously Gaze-Guiding 13
approach is employed where resolution control is applied subsequent to color adjustment on the image. Figure 4 illustrates the results of this integrated guidance processing technique. A comparison of the images, pre- and post- processing in Figures 4(a) and 4(b), reveals that the red car not only assumes a more vivid hue but also that the regions not subject to gaze guidance exhibit both reduced saturation and resolution. These outcomes indicate that the conjoint application of both methods enhances the effectiveness of gaze guidance beyond what is achievable when each is applied independently.
Figure 2: Flowchart of gaze guidance process using color control.Figure 3: Gaze guidance processing results with color control. In (b), the red car appears more vividly red. Conversely, the saturation of regions outside the guidance target is diminished.Figure 4: Gaze guidance processing results combining resolution control and color control. In (b), the red car appears more vividly red. Conversely, the saturation and resolution of regions outside the guidance target is diminished.
EXPERIMENT Experimental Procedure
To assess the efficacy of our unconscious gaze guidance methods, we con- ducted evaluation experiments that tracked the gaze of participants using a head-mounted display (HMD) with integrated gaze-tracking capabilities. The experimental stimuli comprised videos assembled from 10 omnidirectional images of car windows—5 videos per each of two driving scenes. Figure 5 and Figure 6 provides examples of the videos utilized in the experiments. The HMD used was the Meta Quest Pro by Meta, and the videos were presented at a resolution of 3840 by 2160 pixels with a frame rate of 30 frames per second. To measure the subjective experience of discomfort while watching the videos, participants were asked to respond to two questions on a 7-point Likert scale ranging from 0 (not at all) to 6 (extremely):
1. Did you perceive any image processing effects? 2. Did you detect any gaze guidance efforts?
Figure 5: The videos used in the experiment (scene1). The images in (b) through (e) are first-person views of the white vehicle on the left side of (a) with the respective processed omnidirectional images viewed through the HMD.
Video Generation Method Unconsciously Gaze-Guiding 15
Figure 6: The videos used in the experiment (scene2). The images in (b) through (e) are first-person views of the white vehicle on the left side of (a) with the respective processed omnidirectional images viewed through the HMD.
RESULTS
The experiment involved 10 male participants in their 20s, all with normal or corrected-to-normal vision and color vision. Figure 7 presents the gaze guid- ance rates observed when subjects viewed videos created by each method. The gaze guidance rate reflects the proportion of frames where the subjects’ gaze matched the targeted area. The cross (‘+’) in the figure indicates the mean of the gaze guidance rate. The combined resolution and color control method achieved the highest rate, surpassing the rates of the individual meth- ods. Notably, this combined approach reached 85% of the gaze guidance rate of the explicit gazing point drawing method, which is known for its effective- ness in reducing motion sickness—a considerable increase from the 30% rate observed with no processing. This implies that the combined method rivals explicit gaze guidance in effectiveness.
Figures 8 and 9 display questionnaire responses regarding the perceived image processing and the detectability of gaze guidance. The gazing point
drawing method was most strongly associated with unnatural image pro- cessing and intentional guidance. In contrast, the combined method reduced the perception of unnatural image processing and the detectability of guid- ance by 80% and 74%, respectively, indicating that it is less disruptive to the viewing experience.
Figure 10 details the gaze guidance rates across different scenes. Scene 1, characterized by fewer buildings, and Scene 2, with a denser urban environ- ment, did not significantly differ in the effectiveness trend of the combined method. However, within the resolution control method, Scene 1 exhibited a lower guidance rate than Scene 2, likely due to its larger areas lacking texture, which diminishes the guidance effect.
Figure 7: Results of gaze guidance rates.Figure 8: Results of survey on sense of unnatural image processing.
Video Generation Method Unconsciously Gaze-Guiding 17
Figure 9: Results of a survey on the sense of gaze guidance.Figure 10: Results of gaze guidance rates by scene.
CONCLUSION
In this research, we explored video generation techniques aimed at subtly guiding the gaze of passengers in autonomous vehicles to mitigate motion sickness. We developed three unconscious gaze guidance methods: one based on resolution control, another on color control, and a third that combines these two techniques. Our evaluation experiments indicated that the com- bined method enhanced the guidance’s effectiveness, achieving 85% of the effectiveness attributed to explicit gaze guidance, which is well-regarded for reducing motion sickness. Notably, the combined method lessened the
discomfort typically associated with image processing and overt gaze guid- ance. This research received partial funding from the JSPS Grant-in-Aid for Scientific Research (Grant Number 21H03484).
REFERENCES
A. Hagiwara, A. Sugimoto, K. Kawamoto. (2011), Saliency-based image editing for guiding visual attention, Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interaction, pp. 43–48. H. Hata, H. Koike, S. Yoichi (2015), Visual Attention Guidance Using Image Resolu-
tion Control, Journal of Information Processing, Vol. 56, No. 4, pp. 1152–1161. H. Takimoto, T. Kokui, H. Yamaguchi, M. Kishihara, K. Okubo. (2015), Image Modification Based on a Visual Saliency Map for Guiding Visual Attention, IEICE Transaction on Information and Systems, Vol. E98-D, No. 11, pp. 1967–1975. J. T. Reason, J. J. Brand. (1975), Motion sickness, Academic Press, London. L. Itti, C. Koch, E. Niebur. (1998), A model of saliency based visual attention for rapid
scene analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 11, pp. 1254–1259. L. M. Peet, M. Lalmas, V. Navalpakkam. (2012), On Saliency, Affect and Focused Attention, Proceedings of the SIGCHI Conference on Human Factors in Comput- ing Systems, pp. 541–550. M. B. Flanagan, J. G. May, T. G. Dobie. (2002), Optokinetic nystagmus, vection, and
motion sickness, Aviat Space Environ Med, Vol. 73, No. 11, pp. 1067–1073. N. A. Webb, M. J. Griffin. (2002), Optokinetic stimuli: motion sickness, visual acuity, and eye movements, Aviation, Space, and Environmental Medicine, Vol. 73, No. 4, pp. 351–358. N. Miura, H. Ujike, M. Ohkura. (2018), Influence of fixation point movement on
visually induced motion sickness suppression effect, International Conference on Applied Human Factors and Ergonomics, pp. 277–288. S. M. Ebenholtz, M. M. Cohen, B. J. Linder. (1994), The possible role of nystagmus in motion sickness: a hypothesis, Aviat Space Environ Med, Vol. 65, No. 11, pp. 1032–1035.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Accessibility, Assistive Technology and Digital Environments
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-97-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional