Helu Li, Xiaozhou Zhou (2022). Layout of Direct Input Interface Based on Virtual Space. In: Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems. AHFE Open Access, vol. 22. https://doi.org/10.54941/ahfe1001054
Immersive virtual environment has the potential to realize natural interaction in users' close interactive space. To explore the efficient layout of direct input interface suitable for hand-eye cooperation in virtual environment, through the freehand clicking experiment, the participants were required to quickly click the random targets in three different spatial depths. The results show that in the visual field of head mounted display (HMD), the depth has a significant impact on the arm movement time and clicking accuracy. It is suggested that the low-frequency and short-term interactive content can be placed in the left and upper right areas of the visual field, and the high-frequency and long-term content can be placed in the central and lower area with low consumption endurance. The down-right area is not suitable for quick click. This paper proposes an efficient direct input interactive layout in virtual space based on spatial dimensions and arm kinematics.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
INTRODUCTION
Virtual reality (VR) technology refers to the modern method of using computers to generate realistic visual, auditory, tactile and other multi-channel integrated virtual environment (VE). Using hands and bodies to manipulate virtual objects provides a natural interface to interact in three-dimensional (3D) space. Among them, gesture interaction can be divided into two types: non wearable and wearable (Borrego et al. 2018). The former does not need any media, and can use virtual hand or ray-casting technology to construct a virtual substitute for the user's hand in the VE, which is called freehand interaction (Mine et al. 1995). The interactive space in VE is within the reachable domain of an arm (Mclester et al. 2008). The interaction performance of this space depends on various ergonomic factors, e.g., the user's endurance, muscle strength, arm gesture, action frequency and comfort (Lubos et al. 2018). However, how the position of the virtual target within the reach of the arm affects the interaction performance is still a problem to be studied (Jude et al. 2014). Direct interaction with HMD is limited by perception, e.g., the accommodation-convergence mismatch, ghosting, double vision or distance misjudgment (Bruder et al. 2013. Chan, 2010). Fatigue is one of the biggest problems of gestures after long-time interaction, so it may not provide high performance when used for a long time, but this has not been fully considered in the interface layout design (Segen and Kumar, 1998).
A reasonable and efficient interface is the basis of human-computer free interaction. Based on the spatial dimension and arm kinematics characteristics, this paper proposes an efficient layout of direct input interface to improve the efficiency of hand-eye collaboration in VE. This work provides a deeper understanding of how spatial dimensions affect the interactive performance and comfort of direct input.
MATERIALS AND METHODS
Thirty participants (10 males and 20 females) aged between 21 and 25 years (M = 23.6, SD = 1.28) were recruited to participate in the experiment. All participants had normal vision or corrected to normal without any physical injury. All participants are right-handed.
A HTC Vive Pro HMD was provided, with a binocular resolution of 2880×1600px and a refresh rate of 90 Hz, supporting stereo vision to simulate the depth of field of view (FOV). This device was equipped with a LMC, which can capture the finger position and gesture, and ensure to always follow the participant's front interaction range.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
The experiment task is the freehand clicking in VE. The center of the participant's eyes is determined as the center of the reference frame in the virtual scene. The target randomly appears at the setting spatial position with the spatial depth as the radius. Targets are deflected so that it is perpendicular to the line of sight from the participant's eyes to the target. To ensure that the same target size is seen in each trial, the targets in the experiment are presented in the perspective, with a diameter of 5 °, as shown in Fig. 1. According to the results of binocular visual range and visual position in natural state, 19 horizontal positions are taken from - 45 ° to 45 ° and 13 vertical positions are taken from - 35 ° to 25 ° at an interval of 5 °. According to the comfortable range of the participants in the pre-experiment, three spatial depths (i.e. input distance) are determined: 350mm, 400mm and 450mm.
The 14 corners where the participants still had difficulties in clicking or perception with HMD. Therefore, 233 spatial positions are reserved, as shown in Fig. 2. To eliminate the hand occlusion problem and non fingertip mistouch problem, translucent material is assigned to the virtual hand, and a trigger ball is added to the index fingertip.
EXPERIMENTAL PROCEDURE The formal experiment included 699 trials. The presentation order of targets was balanced among participants on a Latin square basis. The experiment was divided into 21 blocks. Sufficient forced rests were set to ensure no fatigue effect. The total time lasted about 1 hour. Participants were required to quickly click on random targets at three spatial depths in a sitting position. Participants were introduced to the purpose and task requirements. Before
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 the formal block, participants were required to practice until they are observed to have achieved a stable interactive performance.
In a trial, the participant needs to straighten the head according to the gray sign with a fixed position to ensure that the head is in the same starting position. Next, the participant is required to place the virtual hand on the red target that appears to ensure that the hand is in the same starting position. Then the participant is required to quickly click the blue target that appears to disappear. Enter the next trial at an interval of 2 seconds. As shown in Fig. 3. After completing the experimental task, participants were shown nine spatial areas and asked to report preference scores. The experimental scenario is shown in Fig. 4.
Fig. 3. Experimental process design Fig. 4. Freehand clicking experimental scene
RESULTS
This study requires participants to quickly click on the target, so clicking offset is the main performance evaluation index. The correlation analysis results in Table 1 show that the correlation between clicking offset and depth is the highest, followed by movement time and distribution area, in which the correlation between depth, movement time and clicking offset is significant at the level of 0.01.
Table 1: Correlation between clicking offset and depth, movement time and distribution area Depth Movement Time Distribution Area
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
Significant .000 .000 .025
**Significance at 0.01 level *Significance at 0.05 level
MOVEMENT TIME Movement time refers to the time to complete a clicking action, i.e., the time from the virtual hand leaving the red button to clicking the blue target. One way ANOVA showed that the movement time was significantly affected by depth [F(2,20322)=119.549, sig=0.000<0.010], target distribution area [F(8,20316)=132.186, sig=0.000<0.01], horizontal arrangement angle [F(18,20306)=44.112,sig=0.000<0.01] and vertical arrangement angle [F(13,20311)=32.766,sig=0.000<0.01].
As shown in Fig. 5, it is a quartile contour map of movement time at three spatial depths. In general, the movement time required to click the target in the central area is shorter, while the time required to click the lower right area and upper left area is longer. With the increase of spatial depth, the performance advantage of the central area decreases. While the movement time performance of the lower right corner area becomes better, but the movement time of the upper left corner area deteriorates.
Fig. 5. Quartile contour map of movement time at three spatial depths CLICKING ACCURACY
The offset reflects the accuracy of clicking each position: the higher the offset value, the lower the clicking accuracy. The offset is defined as the standard offset of the distance between the clicking endpoint and the target center. The offset is in visual angle (°) to exclude distance offset caused by visual angle problem. A positive offset x indicates that the user hits the right side of the target center, and the positive offset y indicates that the user hits the top
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 of the target center. One way ANOVA showed that the absolute value of offset was significantly affected by depth [F(2,20322)=1462.160, sig=0.000<0.01], target distribution area [F(8,20316)=4.138, sig=0.000<0.01] and vertical arrangement angle [F(13,20311)=2.993, sig=0.000<0.01]. With the increase of depth, the overall target clicking offset increases, but some values different from the surrounding appear in the spatial position distribution. As shown in Fig. 6.
Fig. 6. Absolute value heat map of offset at three spatial depths One way ANOVA showed that the absolute value of offset in horizontal direction was significantly affected by depth [F(2,20322)=15.785, sig=0.000<0.01], target distribution area [F(8,20316)=805.116, sig=0.000<0.01] and horizontal arrangement angle [F(18,20306)=400.048, sig=0.000<0.01]. The heat map shows that the offsets are usually negative in the right half area, indicating that the actual clicking position in the right half area shows a left trend. The left half area is reversed. As shown in Fig. 7.
Fig. 7. Heat map of offset in horizontal direction at three spatial depths One way ANOVA found that the absolute value of offset in the vertical direction was significantly affected by depth [F(2,20322)=66.743, sig=0.000<0.01], target distribution area [F(8,20316)=550.035, sig=0.000<0.01], horizontal arrangement angle [F(18,20306)=6.696, sig=0.000<0.01] and vertical arrangement angle [F(13,20311)=378.186, sig=0.000<0.01]. The heat map shows that the offsets are usually
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7 negative in the upper half area, indicating that the actual clicking position in the upper half area shows a downward trend. The lower half area is reversed. As shown in Fig. 8.
Fig. 8. Heat map of offset in vertical direction at three spatial depths
CONCLUSIONS
In this study, the effects of the spatial area of the target on clicking movement time, clicking accuracy and user subjective preference are investigated. The results show that the area where the target appears has a significant impact on the three indicators. The overlapping area with fast movement time and high clicking accuracy is an efficient interaction area for hand-eye cooperation. For low-frequency and short-term interactive content, hand-eye coordination should be mainly considered. It is recommended to place the information content in the left and upper right areas of the visual range with short moving time. For high- frequency and long-term interactive content, hand-eye coordination and arm fatigue should be considered at the same time. It is recommended to place it in the central area with good performance of freehand interaction and the area slightly lower than the horizontal line of sight with low consumed endurance. The lower right area is not suitable for quick clicking operation. This work provides a deeper understanding of how spatial dimensions affect the interaction performance and comfort of direct input.
ACKNOWLEDGMENTS
This work has received funding partly from the National Natural Science Foundation of China (No. 71901061, 71871056).
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
REFERENCES
Borrego, A., Latorre, J., AlcaIz, M., et al. (2018). Comparison of Oculus Rift and HTC Vive: Feasibility for Virtual Reality-Based Exploration, Navigation, Exergaming, and Rehabilitation. Games for Health Journal, 7, 151--156 . Mine, M.R.. (1995). Virtual Environment Interaction Techniques. UNC Chapel Hill
Computer Science Technical Report, UNC Chapel Hill CS Dept, 1--18. Mclester, J., Pierre, P.(2008). Applied Biomechanics : Concepts and Connections. Science Life Sciences Biophysics . Lubos, P., Bruder, G., Ariza, O., Steinicke, F. (2016). Touching the Sphere: Leveraging
Joint-Centered Kinespheres for Spatial User Interaction. Symposium on Spatial User Interaction. ACM, 13--22 . Jude, A., Poor, G., Guinness, D. (2014). Personal Space: User Defined Gesture Space for GUI Interaction. In Proc. of ACM CHI, 1615–1620. Lubos, P., Bruder, G., Steinicke, F. (2015). Influence of Comfort on 3D Selection Task
Performance in mmersive Desktop Setups. Journal of Virtual Reality and Broadcasting (JVRB), 12(2), 15 pages . Bruder, G., Steinicke, F., Stuerzlinger, W.(2013). Touching the Void Revisited: Analyses of Touch Behavior on and Above Tabletop Surfaces. In Proc. of lN-TERACT,17 pages. Chan, L.W., Kao, H.S., Chen, M.Y., Lee, M.S., Hsu, J., Hung, Y.P. (2010). Touching the
Void: Direct-touch Interaction for Intangible Displays. In: ACM, 2625-2634. Boring, S., Altendorfer, M., Broll, G., Hilliges, O., Butz, A.: Shoot & Copy. (2007). Using mobile phones for accessing information on large displays. In Proc. OZCHI, ACM, 4 pages. Segen, J., Kumar, S.: Gesture VR. (1998). Vision-Based 3D Hand Interace for Spatial
Interaction. Proceedings of the 6th ACM International Conference on Multimedia '98, Bristol, England, September, 12--16.
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems https://openaccess.cms-conferences.org/#/publications/book/978-1-7923-8988-7
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2022): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-7923-8988-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional