Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

An Overview of Spatial Stimulus-Response (S-R) Compatibility Studies for Further Human Machine Interface Studies

1 City University of Hong Kong Shenzhen Research Institute, Shenzhen, P. R. China ↗
2 Institute of Human Factors and Ergonomics, College of Mechatronics and Control Engineering, Shenzhen University, Shenzhen, P. R. China ↗
3 Industrial Engineering Department, College of Engineering, King Saud University, Riyadh, Saudi Arabia ↗
Publication year
Digital Object Identifier10.54941/ahfe1002185

Abstract

This paper provides an overview of a series of spatial stimulus–response (S-R) compatibility studies which involved various audio/visual signals and hand/foot controls. The experiments examined the performance of right-handed/right-footed participants for various spatial stimulus–response (S-R) mapping conditions with signals and controls configured in different orientations. The effects of various signal modalities and intra/crossed modalities, crossed/uncrossed hands, single/dual tasks, discrete and continual tracking tasks on S-R tasks were examined. Performance measures including response times (RT), error percentages (EP), root mean square errors for hand and foot tracking (RMSHTE and RMSFTE) for various configurations were analyzed, and the major findings are summarized. The studies had provided useful ergonomics guidelines for designing hand/foot controls and audio/visual displays which are valuable for improving efficiency and performance in human–machine systems. The review also reveals the knowledge gaps left unfilled from past studies and further research directions are then suggested to fill in the gaps of the spatial S-R compatibility studies.

INTRODUCTION

Displays and controls provide the means of communication between peo- ple and machines in human-machine systems. Displays provide information about operational status, and control devices enable operators to take neces- sary actions and change the states of a human-machine system (Kang & Seong, 2001). When people operate a control, they have expectations about what it will do and what effect it will have on a display. The relationship between a control movement and the effect most expected by a popula- tion is known as a population stereotype or direction-of-motion stereotype.

© 2022. Published by AHFE Open Access. All rights reserved. 424

An Overview of Spatial Stimulus-Response (S-R) Compatibility Studies 425

In movement compatibility studies, it is a common finding that compatible settings lead to faster reaction times (RTs) and lower error rates than incom- patible settings (Chan & Chan, 2003). Other than movement compatibility, there is also the need to consider spatial compatibility in human–machine interface design if performance is to be enhanced (Sanders & McCormick, 1992). The concept of spatial stimulus–response (S–R) compatibility was introduced a long time ago by Fitts and Seeger (Fitts & Seeger, 1953), who showed that human performance relies not only on the type of signal or response arrays used, but also on the pairing of individual signals with responses. The importance of spatial S–R compatibility for practical interface design has been demonstrated for the layout of the function keys on a key- board and the corresponding labels for these keys on a screen (Bayerl et al., 1988). Reaction times (RTs) were shorter when the spatial positions of the control function keys and the display labels corresponded. Other than appli- cation specific studies, basic research aimed at fundamental understanding of spatial S–R compatibility with visual signals has been conducted with tasks involving the pressing of a right or left key in response to a light appearing to the right or left of a fixation point on a screen. Reactions associated with spa- tially compatible S–R pairings were faster than those with incompatible S–R pairings (Roswarski & Proctor, 1996). The reduction of visual RT in spatially compatible S–R pairing has been thought to arise from a ‘natural’ tendency to respond in the direction of stimulation. Several studies had been conducted to examine the spatial S–R compatibility effect with natural (hands uncrossed) and unnatural hand posture (hands crossed) (Nicoletti et al., 1984; Umiltá & Nicoletti, 1990) in visual choice reaction tasks. Irrespective of the S–R compatibility condition in the control–display configurations, responses were always found to be faster with the hands uncrossed than with the hands cros- sed. The overall slowing of RT with the hands crossed is believed to be due to the mismatch between the locational code and the anatomical code. In addition to visual signals, the use of auditory signals is becoming more com- mon in the design of information displays and warnings for reduction of high visual workload in many complex manufacturing systems (Nanthavanij & Yenradee, 1999). The advantages of auditory displays in complementing exi- sting visual displays were recently suggested by Chen and Carlander (2003). Many control systems consist of one or many auditory signals to be perceived and identified by operators (Bronkhorst et al., 1996; Chen, 2003). Industrial control systems and computer interfaces often contain auditory signals that demand attention and action from control operators. Visual and auditory displays are ubiquitous in human tasks (Wang & Kan, 2004). It has been reported that in some situations, auditory signals could be used to stimulate situational awareness and improve visual display effectiveness. Consequently, there is an increasing need to understand the interactions and relationships and particularly the compatibility between the displays and controls. It is beli- eved that, if appropriate spatial compatibility relationships are built between the control and display components in person–machine interfaces, advanta- ges like faster learning, faster reaction time, fewer errors, and higher user satisfaction will be achieved.

In practical situations, operators need to attend to signals of different modalities concurrently transmitted from a variety of machinery and equi- pment in manufacturing systems. They may be required to respond to visual signals and auditory messages by pressing related response keys or foot pedals. Recognizing the importance of spatial S–R compatibility with visual and audio signals, a series of spatial S–R compatibility research studies had been conducted in City University of Hong Kong over the last two decades. Table 1 listed the published spatial S–R compatibility papers that are included in this paper presentation.

METHODS

Participants Various groups of male and female Chinese students ranging from 20 to 38 with ages between 19 to 36 (Table 1) participated in the experiments with various control-display configurations. All participants were right- handers/right-footers. For those experiments involving visual signals, all par- ticipants had normal or corrected-to-normal vision (Optical Co., Inc. Model 2000P orthorator) and had normal color vision (Ishihara Pesudo Isochroma- tic Plates). Whereas for experiments involving audio signals, all participants had passed a standard audiometric test (Peters Audiometer AP27) in which pure tones of 500, 1000, 2000 and 4000 cps were presented to each ear sepa- rately. No participants had hearing loss greater than 20 dB at any one of the four frequencies tested.

Apparatus and Signals for Various Experiments In each experiment, a personal computer was used to display signals and capturing response data. During a presentation, either the visual/auditory signal or both were presented for testing with either spatially compatible or incompatible mappings. For hand-control operations, depending on different configuration requirements for each experiment, two to four keys for discrete tasks or a joystick for continual tracking tasks were employed in vertical/ho- rizontal/transverse/longitudinal planes. Using experiment 1 as an illustration, four keys labeled as ‘FR’ (front-right), ‘FL’ (front-left), ‘RR’ (rear-right) and ‘RL’ (rear-left) on the top plane of a control box were used for inputting responses by participants (Figure 1). Whereas for those experiments invo- lving foot controls, foot pedals with two keys each were used to response to various stimuli.

Summary for Various Control Display Configurations in this Paper’s S-R Compatibility Studies Various signal modalities (audio, visual, audio-visual and visual-visual), crossed/uncrossed hand controls, foot pedals, single/dual tasks, discrete con- trol/continual tracking tasks were conducted in this series of experiments. Performance measures including response times (RT), error percentages (EP), root mean square errors for hand tracking (RMSHTE) and root mean square errors for foot tracking (RMSFTE) for various control-display configurations

An Overview of Spatial Stimulus-Response (S-R) Compatibility Studies 427

Table 1. Spatial S–R compatibility research findings included in this overview paper.

Table available in the original PDF paper.

Participants Paper Reference Journal Paper Title Journal Information

Displays 26 (2005) 109–119

Total: 20 (M:20, F:0)

(Chan & Chan, 2005) Spatial S–R compatibility of visual and auditory signals: implications for HMI design App Ergon 37 (2006) 131–140

Total: 34 (M:26, F:8)

(Chan & Chan, 2006) Synchronous and asynchronous presentations of auditory and visual signals: Implications for control console design

Total: 36 (M:28, F:8)

Theo Issues in Ergon Sci 8(6), (2007) 557–581

(Chan et al., 2007) Auditory stimulus-response compatibility and control-display design

Total: 32 (M:16, F:16)

Int J of Ind Ergon 39 (2009) 396–402

(Chan & Chan, 2009b) Spatial stimulus–response (S-R) compatibility for foot controls with visual displays

Total: 32 (M:16, :6)

(Chan & Chan, 2009a) Spatial Stimulus-response (S-R) Compatibility Effect for Hand Controls with Visual Signals on Horizontal Plane

IMECS 2009, March 18 - 20, 2009, Hong Kong App Ergon 41(6), (2010) 840-848.

Total: 24 (M:19, F:5)

(Chan & Chan, 2010) Three-dimensional spatial stimulus response (SeR) compatibility for visual signals with hand and foot controls

Total: 32 (M:23, F9)

Ergonomics 54(3) (2011) 233–245

(Chan & Chan, 2011b) Spatial stimulus response compatibility for a horizontal visual display with hand and foot controls Displays 32 (2011) 237–243

Total: 38 (M:35, F:3)

(Chan & Chan, 2011a) Spatial stimulus–response compatibility for hand and foot controls with vertical plane visual signals

Total: 22 (M:12, F:10)

Euro J. Industrial Engineering, 6(5) (2012)

(Chan & Or, 2012) A comparison of semantic and spatial stimulus-response compatibility effects for human-machine interface design

Total: 36 (M:24, F:12)

(Tsang & Chan, 2015) Tracking and discrete dual task performance with different spatial stimulus–response mappings

Ergonomics, 2015 58(3) 368–382 Total: 32 (M:22, F:10) Int J of HCI, 33(1), (2017) 21–34 (Kang et al., 2017) Hand- and Foot-Controlled Dual-Tracking Task Performance Together with a Discrete Spatial S-R Compatibility Task

App Ergon 67 (2018) 39-49

Total: 36 (M:28, F:8)

(Tsang & Chan, 2018) Tracking and discrete dual task performance for different visual spatial S-R mappings with focal and ambient vision

Total: 36 (M:32, F:4)

Ergonomics, 64 (4), (2021) 485–501

(Tsang et al., 2021) Auditory versus visual spatial stimulus-response mappings in tracking and discrete dual task performance: implications for HMI design

Figure 1: Experimental setup for testing of visual and auditory signals in the longitu- dinal and traverse orientations. (Chan & Chan, 2005).
Original PDF Figure 1:

were examined and analysed. The following table (Table 2) summarised the signal modalities, task nature, dimension and input devices for the various control/display configurations for this paper.

FINDINGS AND DISCUSSIONS The results of these research provided useful ergonomic recommendations for the industrial design of control/display used in human-machine interfa- ces for improved human performance. Below is a list of design implications formulated from the findings of these studies.

(1) The relative positions of signals should be compatible with both the response key positions and the hand positions. If compatibility can only be built in one orientation, the transverse orientation should be selected. (2) Configurations requiring users to cross their hands to respond should not be used. (3) For faster responses, a visual signal is preferable to an auditory signal for requesting operator response on a control console. (4) For faster reactions, auditory signals should be positioned on the right- hand side of right-handed operators. (5) Synchronous signal presentation is better than asynchronous presentation. (6) For increased response accuracy, simultaneous visual and auditory signals should be presented to the right instead of the left side of operators. (7) Auditory signals placed in a transverse orientation (rather than longi- tudinal orientation) will produce quicker and more accurate responses. (8) The relative positions of visual signals and foot pedal response keys should be spatially compatible for the best human–machine system performance (9) Compatibility is stronger in the right–left dimension than in the front/rear dimension for foot controls.

An Overview of Spatial Stimulus-Response (S-R) Compatibility Studies 429

Table 2. The signal modalities, task nature, dimension and input devices of the S–R compatibility researches included in this overview paper.

Table available in the original PDF paper.

Input Devices

Paper Reference

Dimension (Transverse-T, Longitudinal- L)

Task Nature (Single-S, Dual-D)

Primary Task (P)/Secondary Task (S) (Visual-V, Audio-A)

Signal Modalities (Auditory-A, Visual-V, Intra-I/Cross-C modalities)

2 Front keys, 2 Rear keys

(Chan & Chan, 2005)

A, V S T, L P: Discrete A/V signal with Crossed/Uncrossed hand controls

2 Traverse keys

(Chan & Chan, 2006)

A, V S T P: Discrete A/V signal with Crossed/Uncrossed hand controls

2 Front keys, 2 Rear keys

(Chan et al., 2007)

A S T, L P: Discrete audio signal with Crossed/Uncrossed hand controls

2 Foot pedals

(Chan & Chan, 2009b)

V S T, L P: Discrete visual display with foot controls (Chan & Chan, 2009a) V S T, L P: Discrete visual display with hand control 2 Front keys, 2 Rear keys (Chan & Chan, 2010)

V S T. L (3D) P: Discrete Visual Fixation and hand/foot controls V (Horizontal Plane)

4 Hand buttons, 2 Foot pedals (Chan & Chan, 2011b)

S T, L P: Discrete visual signal with hand/foot controls V (Vertical Plane)

2 Hand buttons, 2 Foot pedals (Chan & Chan, 2011a)

S T, L P: Discrete visual signal with hand/foot controls

2 Hand buttons, 2 Foot pedals (Chan & Or, 2012)

A (Speech and non-speech)

2 Traverse keys

S T P: Discrete audio signal with hand controls

(Tsang & Chan, 2015)

Joystick (P), 4 Hand keys (S)

V D T, L P: Continuous visual tracking S: Discrete visual signal control (Kang et al., 2017) Joystick (P), Hand/Foot key (S) V-V (I), A-V (C) D T, L P: Continuous visual tracking S: Discrete auditory signal control

V (focal and ambient vision)

(Tsang & Chan, 2018)

Joystick (P), 4 Hand keys (S)

D T, L P: Continuous visual tracking S: Discrete visual signal control

(Tsang et al., 2021)

Joystick (P), 4 Hand keys (S)

V-V (I), A-V (C) D T, L P: Continuous visual tracking S: Discrete auditory signal control

(10) Responses with right foot are faster than that with left foot for right- footed participants. (11) For right-hand/right-footed operators, response devices for critical and immediate actions should be manipulated by hands of visual signals and response devices should be spatially and be positioned on the dominant right-hand side. (12) Regardless of foot control position, the compatibility effect was stronger in the front-hand/ rear-foot than in the front-foot/rear-hand mapping condition. (13) Visual signals and hand/foot response keys should be spatially com- patible (right keys for responding right signals and left keys for left signals). (14) Signals for delivering urgent messages should be positioned at the front-right position on a horizontal display. (15) Response devices for critical and immediate actions should be mani- pulated by hand rather than by foot. (16) Non-speech signals rather than speech signals are preferred. (17) If possible, both semantic S-R compatibility and spatial S-R com- patibility need to be considered and instituted in human machine interfaces. (18) If there is a conflict in building both semantic and spatial S-R com- patibility, the semantic S-R compatibility should be given with higher priority. (19) With a continuous task and a discrete response task, the display– control mapping for the discrete response task should be spatially highly compatible in order to minimize the time for encoding and hence the competition for the same resource. (20) If competition for focal vision is inevitable, the distance between the visual sources and their relative positions should be determined to minimize visual scanning. (21) When the hands and feet are required for continuous responses, there was a propensity to prioritize the hand response task at the expense of the foot response task. (22) Mixed modality of visual and auditory presentation within the same task should be avoided to eliminate or reduce response conflict and the modality shifting effect. (23) Dual task performance in a cross-modality (e.g. auditory visual) set- ting was significantly better than that in an intra-modality (e.g. visual visual) setting.

FURTHER RESEARCH AREA The spatial stimulus–response compatibility (SRC) effect refers to the robust finding that human performance is better for some spatial arrangements of controls and displays than for others. Usually, the spatial SRC effect is most marked when components of the response panel physically correspond in some obvious ways with those of the stimulus panel. Precue signals have been commonly used in air transportation systems (Liu et al., 2016) and vehicle

An Overview of Spatial Stimulus-Response (S-R) Compatibility Studies 431

systems (Yan et al., 2015) for pilots or drivers to prevent collisions. Although the precue effect on human performance in a single SRC task has been shown to be positive, no previous studies on spatial SRC effect did consider the pre- cue effect on human performance in a dual-/multi-task paradigm. There are several related research questions to be addressed in the future: (1) how does the modality of precue signals including visual, auditory and tactile influence human performance in the dual-/multi-task paradigm? (2) what is the inte- raction between the modality of precue signals and the modality of stimulus signals in influencing human performance in the dual-/multi-task paradigm? (3) do the precue signals with multiple modalities lead to better human per- formance than those with a signal modality? The answers to these questions are critical to the design of human-machine interface for improving human performance.

Apart from human performance, physiological responses, such as eye movement and neural oscillations, are useful to examine how people react to stimuli and the physiological mechanisms of the multimodal information processing. Nevertheless, little knowledge concurrent about these physiolo- gical responses of participants in the context of the spatial SRC effect has been observed in the literature. Hence, further research effort is essential to determine how to enhance signal presentation and processing to improve human performance with multi-modal interfaces in advanced and complex systems considering the spatial SRC effect. New methods of target signal pre- sentation, employing precue stimulus presentation to capture the attention of participants, are suggested to explore and determine its impact on and its interaction with the spatial SRC effect. The measurement of NASA Task Load Index will also give data for the understanding of the perceived workload of the operators in different compatibility conditions.

ACKNOWLEDGEMENT

This work was supported by the National Natural Science of Foundation of China (Grant No. 72071170). The authors extend their appreciation to the International Scientific Partnership Program ISPP at King Saud University for funding this research work through ISPP # 136.

REFERENCES

Bayerl, J.P., Millen, D.R. and Lewis, S.H. (1988), “Consistent layout of function keys and screen labels speeds user responses.” Human Factors Society Annual Meeting, Santa Monica, CA. Bronkhorst, A.W., Veltman, J. and Van Breda, L. (1996), “Application of a three-

dimensional auditory display in a flight task.” Human Factors, 38(1), 23–33. Chan, A.H.S., Chan, K. and Yu, R.F. (2007), “Auditory stimulus-response compatibility and control-display design.” Theoretical Issues in Ergonomics Science, 8(6), 557–581. Chan, A.H.S. and Chan, K.W. (2006), “Synchronous and asynchronous presenta-

tions of auditory and visual signals: Implications for control console design.” Applied Ergonomics, 37(2), 131–140.

Chan, A.H.S. and Chan, K.W. (2010), “Three-dimensional spatial stimulus–response (S–R) compatibility for visual signals with hand and foot controls.” Applied Ergonomics, 41(6), 840–848. Chan, A.H.S. and Or, C.K. (2012), “A comparison of semantic and spatial stimulus-

response compatibility effects for human-machine interface design.” European Journal of Industrial Engineering, 6(5), 629-643. Chan, K.W. and Chan, A.H.S. (2005), “Spatial S–R compatibility of visual and auditory signals: implications for human–machine interface design.” Displays, 26(3), 109–119. Chan, K.W. and Chan, A.H.S. (2009a), “Spatial stimulus-response (SR) compatibility

effect for hand controls with visual signals on horizontal plane.” Proceedings of the International MultiConference of Engineers and Computer Scientists, Chan, K.W. and Chan, A.H. (2009b), “Spatial stimulus–response (SR) compatibility for foot controls with visual displays.” International Journal of Industrial Ergonomics, 39(2), 396–402. Chan, K.W. and Chan, A.H.S. (2011a), “Spatial stimulus–response compatibility

for hand and foot controls with vertical plane visual signals.” Displays, 32(5), 237–243. Chan, K.W. and Chan, A.H.S. (2011b), “Spatial stimulus response compatibility for a horizontal visual display with hand and foot controls.” Ergonomics, 54(3), 233–245. Chan, K.W. and Chan, A.H.S. (2003), “Movement compatibility for rotary control

and circular display—Computer Simulated Test and real Hardware Test.” Applied Ergonomics, 34(1), 61–71. Chen, F. (2003), “Localization of two sounds simultaneously.” Proceedings of the 15th Triennial Congress of the International Ergonomics Association, Korea, Chen, F. and Carlander, O. (2003), “Localization of 3D sound in a noisy envi-

ronment.” Proceedings of the 15th Triennial Congress of the International Ergonomics Association, Korea, Fitts, P.M. and Seeger, C.M. (1953), “SR compatibility: spatial characteristics of stimulus and response codes.” Journal of Experimental Psychology, 46(3), 199–210. Kang, H.G. and Seong, P.H. (2001), “Information theoretic approach to man-

machine interface complexity evaluation.” IEEE transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 31(3), 163–171. Kang, S.X.Q., Tsang, S.N.H., Chan, A.H.S. and Liu, S.L. (2017), “Hand-and foot-controlled dual-tracking task performance together with a discrete spa- tial stimulus-response compatibility task.” International Journal of Human– Computer Interaction, 33(1), 21–34. Liu, J., Gardi, A., Ramasamy, S., Lim, Y. and Sabatini, R. (2016), “Cognitive pilot-

aircraft interface for single-pilot operations.” Knowledge-Based Systems, 112, 37–53. Nanthavanij, S. and Yenradee, P. (1999), “Predicting the optimum number, location, and signal sound level of auditory warning devices for manufacturing facilities.” International Journal of Industrial Ergonomics, 24(6), 569–578. Nicoletti, R., Umiltà, C. and Ladavas, E. (1984), “Compatibility due to the coding

of the relative position of the effectors.” Acta Psychologica, 57(2), 133–143. Roswarski, T.E. and Proctor, R.W. (1996), “Multiple spatial codes and temporal overlap in choice-reaction tasks.” Psychological Research, 59(3), 196–211. Sanders, M.S. and McCormick, E.J. (1992), “Human factors engineering design.”

McGraw-Hill.

An Overview of Spatial Stimulus-Response (S-R) Compatibility Studies 433

Tsang, S.N.H. and Chan, A.H.S. (2015), “Tracking and discrete dual task performance with different spatial stimulus–response mappings.” Ergonomics, 58(3), 368–382. Tsang, S.N.H. and Chan, A.H.S (2018), “Tracking and discrete dual task performa-

nce for different visual spatial stimulus-response mappings with focal and ambient vision.” Applied Ergonomics, 67, 39–49. Tsang, S.N.H., Chan, A.H.S., Pan, X. and Man, S.S. (2021), “Auditory versus visual spatial stimulus-response mappings in tracking and discrete dual task per- formance: implications for human-machine interface design.” Ergonomics, 64(4), 485–501. Umiltá, C., & Nicoletti, R. (1990). Spatial Stimulus-Response Compatibility. Adva-

nces in Psychology, 65, 89–116. Wang, A.H. and Kan, Y.F. (2004), “Effects of display type, speed, and text/background colour-combination of dynamic display on users’ comprehension for dual-task in reading static and dynamic display information.” The International Journal of Advanced Manufacturing Technology, 23(1-2), 133–138. Yan, X., Zhang, Y. and Ma, L. (2015), “The influence of in-vehicle speech warning

timing on drivers’ collision avoidance performance at signalized intersections.” Transportation Research Part C: Emerging Technologies, 51, 231–242.

Cover of the proceedings volume Human Factors and Systems Interaction

Explore the complete digital volume

Human Factors and Systems Interaction

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-28-5

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →