Wentao Li, Zhizhong Li (2024). Mental Workload Prediction Method Based on GOMS. In: Human Dynamics, Product Evaluation and Quality. AHFE Open Access, vol. 131, pp. 33–40. https://doi.org/10.54941/ahfe1004861
The conventional approaches to assessing mental workload with operators are time-consuming, and even more challenging when experienced operators are tougher to find. Prior to an experiment involving operators, mental workload prediction methods may be useful for having a preliminary evaluation of a system or interface. This study represented mental workload using the ratio of GOMS-based predicted task completion time to available time. The low-version and high-version maritime operation interfaces were compared. In the GOMS analysis, this study disassembled task goals based on a hierarchical structure and matched each subtask goal with a method. Given the presence of considerable repetitive GOMS operators throughout the task execution, the idea of operator sequence block was introduced for task analysis and reader comprehension. By nesting these blocks, the task was decomposed into keystroke-level GOMS operators. By accumulating the standard times of the GOMS operators, the time prediction results for operator sequence blocks, methods, hierarchical task goals, and overall task can be obtained following the bottom-up approach.The results indicated that the number of GOMS operators and the task completion time required for the operators significantly decreased when using the high-version interface. Consequently, it was anticipated that the high-version interface could notably reduce the operators’ mental workload. The mental workload prediction method based on GOMS proposed in this study can be used to guide early-stage interface design to enhance operator performance.
For complex systems such as nuclear power, shipping, and aerospace, predict- ing mental workload is a critical issue (Loft et al., 2007). Mental workload prediction model should be developed during an early design phase when a system is only conceptualized (Mitchell and Samms, 2009; Xie and Sal- vendy, 2000). Once a prediction model is proposed and validated, it will reduce the cost of evaluating systems and interfaces (Kim et al., 2013). The assessment methods for mental workload, such as subjective measurements, physiological measurements, and performance measurements, apply only to existing physical mock-ups of a system. However, they are not suitable for an early design stage since empirical testing is unavailable (Jo et al., 2012). Furthermore, the assessment methods with operator involvement can be time-consuming, and it becomes even more challenging when experienced
operators are scarce. Predicting mental workload before the development of physical mock-ups without operator involvement can facilitate rapid iteration of system designs and control operators’ mental workload.
High workload can be stressful, especially when it reaches a level that challenges the operators’ sense of competence and confidence in achieving the mission goals (Ferraro et al., 2022). A system design that places a high mental workload on its operators, even if capacity is not exceeded, reduces the available capacity for the task and becomes more difficult to use com- pared to a system design that imposes a low mental workload (Hertzum and Holmegaard, 2013). High mental workload can also lead to increased operator errors. For complex systems like ships and nuclear power, the conse- quences of operator errors can be severe. To minimize these errors, it is crucial to prevent operators from experiencing excessively high mental workload. In some cases, excessively low mental workload can also result in decreased per- formance because operators struggle to maintain high levels of attention and may lose motivation. Therefore, maintaining operators’ mental workload within an appropriate range is essential.
Workload may be defined as the relationship between the resources required to carry out a task and the resources available, if we consider resources in terms of time, when the demand exceeds the availability, there is an excessive workload (Wickens and Tsang, 2015). Hendy et al. (1997) and Wickens (2002) predicted overall mental workload with time to perform a task as a function of time available. In this study, we define mental workload similarly as follows:
Mental workload = Time prediction based on GOMS
Time available (1)
We predict time required by GOMS (Goals, Operations, Methods, and Selection-rule). It is one of the most widely used theoretical models in the HCI field (Card et al., 1983). With the efforts of many scholars, more variants of GOMS model have been developed, including KLM-GOMS, CMN-GOMS, NGOMSL, CPM-GOMS, and so on. GOMS model can predict task comple- tion time based on a series of GOMS operators, each with standard time or calculation formulas. An analyst can decompose a task through methods and selection rules to get GOMS operator sequences of the task. The predicted value of task completion time is the cumulative time of all GOMS operators. Gao et al. (2013) reported that GOMS was sufficient, as the R-square of the regression analysis between GOMS time estimation and actual operation time exceeded 0.9.
In this study, we focused on the analysis of simulated maritime tasks. Since the operators did not engage in extensive cognitive operations during task execution but rather followed commands to perform specific operations, the task was suitable for analysis using the GOMS model. This study predicted the number of GOMS operators and task completion time to compare two interfaces, guiding the design of task interfaces.
Mental Workload Prediction Method Based on GOMS 35
TASK AND INTERFACES In the simulated maritime operation task, the operators were required to lock onto targets following instructions and provide feedback upon completion. The experimental task involved three steps: preprocessing target informa- tion, calculating and selecting plans, and locking on targets. Mental workload when using two interface versions, IA and IB, was predicted and compared. IA represented the low-version interface, while IB was a redesigned version based on expert comments and interface design guidelines. The task process using IA is as follows:
Step 1 (Preprocess target information): Choose two targets, number them as 1 and 2, input both numbers into the designated box, and sequentially press “Prepare Response” and “Initiate Response”.
Step 2 (Calculate and select plans): Step 2.1 (Calculate plans): Click “Calculate Plans”, choose a target from the dropdown menu, press “Algorithm 1”, select an algorithm from the drop- down menu, click “Calculate”, and wait for stability of parameters displayed in the “Calculation Result”. Report important parameters.
Step 2.2 (Select plans): Click “Select Plans”, choose the target (as in Step 2.1) from the dropdown menu, select a plan from the dropdown menu, and press “Confirm”.
Repeat Step 2 for the other target. Step 3 (Lock on targets): Step 3.1 (Organize channels): Choose the channels corresponding to each of the two targets from the dropdown menu and press “Confirm”.
Step 3.2 (Preparation phase 1): Pick the channel corresponding to one of the two targets from the dropdown menu, press “Prepare Lock”, press “Start Preparation Phase 1”, and wait until a “Completed” message appears. Then adjust the basic locking mode parameters.
Step 3.3 (Preparation phase 2): Press “Start Preparation Phase 2” and wait for a “Completed” message to be displayed.
Step 3.4 (Lock phase): Click “Lock” and wait for the “Lock Completed” message.
Repeat Step 3.2, 3.3, and 3.4 for the other channel. On IB, the algorithms are automatically chosen. The operator’s task flow for IB is as follows: Step 1’ (Preprocess target information): Select two targets in the target information table one by one.
Step 2’ (Calculate and select plans): Choose a plan for each of the two targets.
Step 3’ (Lock on targets): Step 3.1’ (Preparation phase 1): Directly select the channel corresponding to one of the two targets in the channel operation interface, press “Prepare Lock,” press “Start Preparation Phase 1,” and wait for a message of “Com- pleted” (or the indication turns green). Then modify the basic locking mode parameters.
Step 3.2’ (Preparation phase 2): Press “Start Preparation Phase 2”and wait until a message of “Completed” is displayed (or the indication turns green).
Step 3.3’ (Lock phase): Press “Lock” and wait for the “Completed” message (or green indication).
Repeat Step 3.1’, 3.2’, and 3.3’ for the other channel.
36 Li and Li
OPERATOR SEQUENCE BLOCKS Since there were a large number of repetitive operator sequences in the whole task process, operator sequence blocks were introduced to facilitate task anal- ysis and description (see Figure 1). Four operator sequence blocks A,B,C,and D were specified in this study, namely “Press a button according to a com- mand”, “Select a parameter according to a command”, “Input parameters according to a command”, and “Lock a target according to a command”. In the figure, a hexagon represents an operator sequence block, a rectangle represents a GOMS operator and a straight line with an arrow represents sequence. Each operator sequence block is divided into two parts by dot- ted lines, the upper part is the task operations obtained by GOMS analysis, and the lower part is the corresponding GOMS operators. The vertical posi- tion represents the corresponding relationship between task operation and GOMS operators. Taking the operator sequence block C as an example, an operator receives instructions and, following an auditory response, translates the auditory signal into a series of actions. The operator moves the cursor to the input box, clicks twice to enter the pending state, shifts the hand from the trackball to the keyboard, inputs parameters by pressing keys, verbally confirms completion, and then moves the hand back to the trackball.
Figure 1: Examples of operator sequence blocks.
Mental Workload Prediction Method Based on GOMS 37
METHODS
The top-level goal of the task can be divided into three sub-goals, namely preprocessing target information, calculating and selecting plans, and locking on targets. Correspondingly, the completion of the three subgoals requires six methods, which are denoted as Method 1, Method 2, Method 3 for IA, and Method 1’, Method 2’, and Method 3’ for IB. An example is illustrated in Figure 2 to compare and analyse Method 1 and Method 1’.
In Method 1, an operator is required to input target numbers as instructed and click to select the corresponding targets. Firstly, the operator moves the hand to the trackball, navigating the cursor to the button “Preprocess”, and clicking the button. Following this, the operator executes the operator sequence block A by following instructions and clicking the button “Begin”. Commands are then given to input target numbers 1 and 2. The operator enters target number 1 through the sequence block C, followed by moving the cursor to another input box of target number. The operator clicks twice to select the input box and put it in the pending mode. Moving from the trackball to the keyboard, the operator presses the keyboard “2” to input target number 2. After completing the keystrokes, the operator needs to move the hand from the keyboard back to the trackball, navigate the cur- sor to the button “Confirm”, and click the button. After completing all these GOMS operators, the operator verbally reports that both targets have been selected. The operator initiates Method 1’ by moving the hand to the track- ball, subsequently executing two instances of the operator sequence block A. Each execution involves clicking to select one target, followed by verbal confirmation of the completion of the selections.
Figure 2: Comparison of Method 1 and Method 1’.
TIME PREDICTION Taking the operator sequence block A as an example, the process of time pre- diction is illustrated. This block consists of five operators: auditory reaction, simple cognition, pointing, keying, and oral reporting. Based on previous studies, the standard times for the first four operators are 0.2 seconds, 0.07 seconds, 1.1 seconds, and 0.28 seconds respectively. For oral report- ing, each syllable is estimated to take 0.25 seconds. Therefore, the completion
38 Li and Li
time of the operator sequence block A is 1.65 seconds plus 0.25 seconds mul- tiplied by the number of syllables. The number of GOMS operators and the predicted results of task completion time in the four operator sequence blocks are presented in Table 1, where the number of syllables was donated as xyj, the number of characters for parameter input was donated as xzf, and it was assumed that there is a need to input ys parameters and select yx parameters.
Table 1. Predicted results for operator sequence blocks.
Table available in the original PDF paper.
Note
Operator Sequence Blocks
Time Prediction (s)
Number of Operators
A Press a button according to a command 3.03+0.25xyj 7
1.65+0.25xyj 5 The number of syllables is xyj. B Select a parameter according to a command C Input parameters according to a command
2.73+0.25xyj +0.28xzf
8+xzf The number of characters for parameter input is xzf. D Lock on a target according to a command
29+(8+xzf) *ys+ 7yx Assuming the need to input ys parameters and select yx parameters.
Set xzf = xyj = 2, ys = yx = 1. The number of GOMS operators and the predicted results of task completion time in all methods and the whole task are presented in Table 2.
Table 2. Predicted results for all methods and the whole task.
Table available in the original PDF paper.
Methods Number of Operators
Predicted Task Completion Time (s) IA IB Diff. IA IB Diff.
Preprocess target information 26 12 14 13.84 7.70 6.14 Calculate and select plans 27 12 15 18.49 7.70 10.79 Lock on a target 107 82 25 69.16 53.20 15.96 The whole task 160 106 54 101.49 68.60 32.89
Based on the GOMS analysis results, compared to IA, IB was predicted to reduce the number of GOMS operators for the three subgoals by 53.8%, 55.6%, and 23.4%, respectively, and reduce task completion time by 44.4%, 58.4%, and 23.1%, respectively. The total number of GOMS operators and total completion time for the task were predicted to decrease by 33.8% and 32.4%, respectively. Therefore, under the same available time conditions, mental workload of IB was predicted to be remarkably lower than that of IA.
Mental Workload Prediction Method Based on GOMS 39
DISCUSSION
In this study, the resources supplied by operators to perform tasks were quan- tified in terms of time. In other words, predicted mental workload can be defined as the ratio of GOMS-based predicted task completion time to avail- able time. Wickens (2002) proposed that a ratio greater than 1 indicates that an operator is in a state of overload. This method, besides guiding the design of the entire task flow, can also facilitate comparisons between differ- ent designs for task steps. It can help identify effective designs and areas where operator sequences can still be improved. The proposed operator sequence blocks in this study can be applied for task analysis in similar task contexts and can be also defined independently based on other task contexts.
A task involves multiple resources. Operators allocate resources differ- ently across multiple channels in the same amount of time. Therefore, solely measuring time may not provide a sufficiently comprehensive assessment. Considering this limitation, this method is more suitable for tasks that consume unshared resources and do not have complex cognitive activities.
CONCLUSION
This study proposes a GOMS-based method to predict mental workload imposed on operators in a simulated maritime operation task. Mental work- load was assessed by the ratio of predicted task time based on GOMS to available time. The study decomposed task goals based on a hierarchical structure, and three subgoals were obtained, corresponding to six meth- ods. The operator sequence blocks were introduced for the sake of task decomposition explanation and comprehension. The study illustrated the decomposition of the six methods required to perform the task using the two interfaces. The predictions were made for the time required for opera- tor sequence blocks, methods, hierarchical task goals, and overall task. This method enabled the comparison of predicted mental workload under simi- lar or different time pressures and can be used to guide the iterative design of systems or interfaces. It could also be utilized in the future to quantify changes in mental workload during task execution, thereby allowing system designers to provide more support when operators are under high workload.
ACKNOWLEDGMENT
This study was partly supported by the National Natural Science Foundation of China (Grant No. T2192932).
REFERENCES
Card, S. K., Moran, T. P., Newell, A. (1983). The Psychology of Human-Computer Interaction. L. Erlbaum Associates, Hillsdale, NJ. Ferraro, J. C., Mouloua, M., Mangos, P. M., Matthews, G. (2022). Gaming experi-
ence predicts UAS operator performance and workload in simulated search and rescue missions. Ergonomics 65, 1659–1671.
40 Li and Li
Gao, Q., Wang, Y., Song, F., Li, Z., Dong, X. (2013). Mental workload measurement for emergency operating procedures in digital nuclear power plants. Ergonomics 56, 1070–1085. Hendy, K. C., Liao, J., Milgram, P. (1997). Combining Time and Intensity Effects in
Assessing Operator Information-Processing Load. Hum Factors 39, 30–47. Hertzum, M., Holmegaard, K. D. (2013). Perceived time as a measure of mental workload: effects of time constraints and task success. International Journal of Human-Computer Interaction 29, 26–39. Jo, S., Myung, R., Yoon, D. (2012). Quantitative prediction of mental workload with
the ACT-R cognitive architecture. International Journal of Industrial Ergonomics 42, 359–370. Kim, J. W., Ritter, F. E., Koubek, R. J. (2013). An integrated theory for improved skill acquisition and retention in the three stages of learning. Theoretical Issues in Ergonomics Science 14, 22–37. Loft, S., Sanderson, P., Neal, A., Mooij, M. (2007). Modeling and Predicting Men-
tal Workload in En Route Air Traffic Control: Critical Review and Broader Implications. Hum Factors 49, 376–399. Mitchell, D. K., Samms, C. (2009). Workload Warriors: Lessons Learned from a Decade of Mental Workload Prediction Using Human Performance Modeling. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 53, 819–823. Wickens, C. (2002). Multiple resources and performance prediction. Theoretical
Issues in Ergonomic Science 3, 159–177. Wickens, C., Tsang, P. S. (2015). Workload, in: Boehm-Davis, D. A., Durso, F. T., Lee, J. D. (Eds.), APA Handbook of Human Systems Integration. American Psychological Association, Washington, pp. 277–292. Xie, B., Salvendy, G. (2000). Review and reappraisal of modelling and predict-
ing mental workload in single- and multi-task environments. Work & Stress 14, 74–99.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Dynamics, Product Evaluation and Quality
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-07-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional