Vítor Fernando Plácido Da Conceição, Beatriz Sousa, Pedro Água, Joakim Dahlman (2023). Decision-making while interacting with unmanned vessels. In: Human Interaction & Emerging Technologies (IHIET 2023): Artificial Intelligence & Future Applications. AHFE Open Access, vol. 111, pp. 584–594. https://doi.org/10.54941/ahfe1004065
The presence of autonomous vehicles in the maritime domain is already a reality, even though being confined to very specific domains of operations (environmental monitoring, surveillance and defense, R&D) or segregated spaces (exclusive spaces for the operation of autonomous vehicles). Artificial Intelligence algorithms for navigation control applied in autonomous vessels are based on the adoption of rules that currently regulate navigation, namely the International Collision Regulation (ColReg), the maritime Buoyage System, and routing regulations. However, considering Jen Rasmussen's decision model, in many situations, the navigator makes decisions not only based on rules (Rule-Based) but based on perceptions that stem from his skills (Skill Based) or knowledge (Knowledge-Based). An example is the concept of safe speed or safe distance, defined in ColReg, but with a variable quantification depending on the circumstances. On the other hand, the navigator's perception of the concept of navigation safety varies significantly and usually goes beyond the ship domain. For instance, some may decide not complying a ColReg priority rule to facilitate another vessel's movement and prevent a decrease in the operation safety level. Safety perception is conceived holistically, that is, it is not restricted to the vessel, but to all those in the vicinity and the natural environment. Finally, it is important to understand the behaviour of navigators in the face of the existence or interaction with unmanned vessels, not only to understand how the decision process is affected but also to improve the AI algorithms applied in autonomous vehicles.To understand how the perceived status of the encountered vessels affects the navigator's decision, we conducted an experimental study to assess how the decisions made by the participant vary when interacting with unmanned vessels. Recognizing that trust in automation is a critical influential factor, we adopted existing framework models to evaluate the participants' perceptions of Maritime Autonomous Surface Ships (MASS), as classified by the International Maritime Organization.The adopted method comprises a combination of questionnaires and participation in six simulated scenarios. This mixed approach aimed to understand the familiarity with MASS; the need to change operational regulations; concerns, challenges, and opportunities from the implementation of MASS; trust in MASS; and the differences between the declared perception and decision-making when interacting with MASS.The study comprised three stages. firstly, a pilot study to appraise and validate the questionnaire, with 49 participants. Secondly, the online implementation of the questionnaire, with a desktop version of the six simulated scenarios, with 110 valid questionnaires, 73 students from the naval academy and 37 professional mariners. Each scenario presented an interaction situation with another vessel, referencing a clearly stated rule of the Collision Regulation. The target vessel could randomly assume one of three statuses: Manned vessel, Unmanned vessel and unidentified vessel. By varying the control mode of the target vessel in the same situation, we aimed to see if the perceived status of the vessel had any influence on the decision-making process. In the last stage of the study, the six desktop exercises of the scenarios were replaced by a simulator game of the same situation, with 33 participants. On the desktop exercise participants reported: Time for acting, change of heading, change of speed, and aimed final position. Reaction time, change of heading and speed were automatically logged on the simulator game. The questionnaire comprises four sections: Unmanned vessels and levels of automation, scenarios decisions, trust in automation and demographic data.The results suggest that despite having a reduced familiarity with autonomous ships, the participants have a very positive opinion. However, in the same situation, they react differently to conventional ships and autonomous ships. The way navigators react was analyzed through parameters such as reaction time, course and speed variation and the Closest Point of Approach between vessels. There is a greater discrepancy between those parameters in participants with less training, suggesting a need to address the issues of interaction with unmanned vessels during the course program. Results from the simulators provided more precise shreds of evidence, namely when interacting with unidentified vessels, pointing out the need to design solutions for clear identification of the target vessel.
The presence of autonomous vehicles in the maritime domain is already a reality, even though being confined to particular domains of operations (envi- ronmental monitoring, surveillance and defence, R&D) or segregated spaces (exclusive spaces for the operation of autonomous vehicles). Artificial Intel- ligence algorithms for navigation control applied in autonomous vessels are based on adopting rules that currently regulate navigation, namely the Inter- national Collision Regulation (ColReg), the maritime Buoyage System, and routing regulations. However, considering Jen Rasmussen’s decision model, in many situations, the navigator makes decisions not only based on rules (Rule-Based) but based on perceptions that stem from his skills (Skill Based) or knowledge (Knowledge-Based) (Rasmussen, 1983).
An example is the concept of safe speed or distance, defined in ColReg, but with a variable quantification depending on the circumstances. On the other hand, the navigator’s perception of navigation safety varies significantly and usually goes beyond the ship domain. For instance, some may decide not complying to a ColReg priority rule to facilitate another vessel’s move- ment and prevent a decrease in the operation safety level. Furthermore, safety
Decision-Making While Interacting With Unmanned Vessels 585
perception is conceived holistically. It is not restricted to the vessel but to all those in the vicinity and the natural environment (Conceição, Dahlman and Navarro, 2017). Finally, it is essential to understand the behaviour of navigators when facing unmanned vehicles, not only to understand how the decision process is performed (Rasmussen, 1983, p. 263) but also to improve the AI algorithms applied for autonomous vehicles operations by adopting Human-centred design approaches (Woods and Dekker, 2000; Wahlström et al., 2015; Costa, 2018).
The most deliberate reasoning does not always prevail, and sometimes intuitive responses tend to persist, not meaning that these processes should be avoided (Klein, 2003). However, research has been pointing clearly to the idea that neither analytical reasoning nor heuristic processing is sufficient to describe the totality of judgment under uncertainty (Evans and Stanovich, 2013). Indeed, the currently dominant conceptual view in this domain holds that human inductive judgment has a dual nature. An assumption common to different dualistic perspectives is that individuals have two distinct ways of processing information: an intuitive processing mode that tends to rely mostly on intuitions and forms of natural evaluation, such as heuristics, and another mode of analytical or deliberate processing based on rules.
Onboard, managing maritime safety requires some level of interaction with other vessels. This coordination activity aiming for safe operations can be understood as a joint activity (Klein et al., 2005). However, this coordi- nation requires predictability of actions, mutual directability and common ground. In addition, bringing unmanned autonomous systems into action raises new challenges, such as human performance, human-machine inter- faces, augmented cognition, training, and control, that must be addressed in time (Klein et al., 2004; Hancock et al., 2013).
Human judgments tend to occur in situations of social interaction, influ- encing the judgments of others and vice versa. Therefore, when interacting with an artificial intelligence (AI) source, we need to understand how the pro- cesses of social influence theories will be applied (Mugny et al., 1995), namely regarding the effect of low/high power experience in human-AI interactions (Fast and Schroeder, 2020).
To understand how the perceived status of the encountered vessels affects the navigator’s decision, we conducted an experimental study to assess how the decisions made by the participant vary when interacting with unmanned vessels. Recognising that trust in automation is an influential critical factor, we adopted existing framework models to evaluate the participants’ percep- tions of Maritime Autonomous Surface Ships (MASS) as classified by the International Maritime Organization.
Maritime Unmanned Surface Ship - MASS Mainly stemming from Sheridan and Verplank’s (1978) classification of Levels of Automation (LoA) in man-computer decision-making, some frame- works have been developed to classify marine platforms’ LoA (Lloyd’s Register, 2017; Utne, Sørensen and Schjølberg, 2017; Maritime UK, 2022). In 2021, the International Maritime Organization (IMO) proposed
a four-level framework to be used by shipowners, operators and academia (IMO, 2021).
Trust in Automation Trust plays a vital role in humans’ interaction with technology and is one of the main factors influencing interaction with automation (Parasuraman and Riley, 1997; Onnasch et al., 2014). However, trust in automation is a construct that is not directly observable. Consequently, questionnaires have been designed to provide indicators (Körber, 2019). Based on Lee and See (2004) work on modelling trust in automation, Körber (2019) proposed a six dimensions model with three dimensions underlying trust in automation: (1) reliability/competence (ability to perform the task at hand), (2) under- standability/predictability (combination of performance with expectations), and (3) Intention of developers. In addition, an individual’s subjective per- ception of the characteristics of a system determines ultimate confidence in automation (Lee and See, 2004). Thereby, Körber (2019) added to the study dimensions the propensity to trust dimension of the Mayer et al. model (1995). Körber (2019) also suggest that familiarity indirectly influences automation trust. With increased familiarity, operators form expectations, calibrate confidence, and eventually increase their confidence.
METHODS
The adopted method comprises a combination of questionnaires and partic- ipation in six simulated cases. This mixed approach aimed to understand the familiarity with MASS; the need to change operational regulations; concerns, challenges, and opportunities from implementing MASS; trust in MASS; and the differences between the declared perception and decision-making when interacting with MASS.
The study comprised three stages. Firstly, a pilot study was accomplished to appraise and validate the questionnaire with 49 participants. Secondly, we implemented an online questionnaire, with a desktop version of the six simulated scenarios (cases), with 110 valid questionnaires, 73 students from the naval academy and 37 professional mariners. Each case presented an interaction situation with another vessel, referencing a clearly stated rule of the Collision Regulation (ColReg). The target vessel could randomly assume one of three types: Manned vessel (Type 1), Unmanned vessel (Type 2) and unknown control mode vessel (Type 3). By varying the control mode of the target vessel in the same situation, we aimed to verify if the participants’ per- ceived status of the vessel influenced the decision-making process. In the last stage of the study, the six desktop exercises of the scenarios were replaced by a simulator game of similar cases with 33 participants.
The questionnaire comprises four sections: Unmanned vessels and levels of automation perceptions, case decisions, trust in automation and demographic data. The questionnaire was developed considering the domain of applica- tion and theoretical frameworks of trust in automation, adapting questions from the models proposed by (Kretschmann et al., 2015) for studying per- ceptions on MASS; (Schoettle and Sivak, 2014) for studying public opinions
Decision-Making While Interacting With Unmanned Vessels 587
on self-driving vehicles, and (Körber, 2019) about trust on automation, using a 5-point Likert scale.
On the desktop exercise, participants reported: Time for acting, change of heading, change of speed, and aimed final position. Reaction time, change of heading and speed were automatically logged on the simulator game. The rule and expected procedure were described immediately before each case to minimise any biases associated with the Collision Regulation knowledge level. In the desktop scenarios, cases 1 to 4 represent a “crossing situation” (ColReg - rule 15), Case 5 is related to an “overtaking” situation (ColReg - rule 13), and Case 6 to “Head on situation” (ColReg - rule 14). Figure 1 presents an example of the cases used.
Figure 1: Representation example of one case used for the desktop exercise.
RESULTS AND DISCUSSION
Most participants were male (N = 84; 76.4%), mainly cadets and naval officers from the Portuguese Navy. Table 1 presents the age, sex and group distributions.
Trust in MASS After selecting the most significant questions to set the six indexes of trust on autonomy presented in Figure 2, we found that despite the participants’ lower familiarity and propensity to trust, they revealed a positive perception towards automation, namely about the intention of developers and trust in automation. To support the subsequent analysis of the decisions made by the participants, we propose a MASS confidence index.
The MASS confidence index is computed by averaging the six indexes, as shown in Table 2. The alpha coefficient for the six items is .768, suggesting that the items have relatively good internal consistency. With a mean value of 3.2, the index suggests a positive trend in confidence in MASS. To split the participants into two groups distinguishing those with low confidence from the ones with high, we used the value for percentile 50 (3.1875).
Table 1. Participants statistics.
Table available in the original PDF paper.
Count N % Sex F 26 23.6% M 84 76.4% Total 110 100.0%
Age <20 33 30.0% 20 to 23 30 27.3% 23 to 26 21 19.1% > 26 26 23.6% Total 110 100.0% Group of participant Cadet 73 66.4% Leisure 1 0.9% Navy 30 27.3% Port Pilot 1 0.9% Shipping 5 4.5% Total 110 100.0%
Figure 2: Box plot of the six items for MASS confidence.
We found no evidence of the relation between age and MASS confidence (Kruskal-Wallis H = 2,159; df = 3; p>0,1). Furthermore, we also can not claim any relations between expertise (participants groups) and MASS confidence (U = 1228,500; p>0,1).
Decision-Making The subsequent analysis aimed to understand whether there were any rela- tions between confidence in MASS, from the MASS confidence index, and participants’ decision to avoid other vessels. Firstly we tested if the partici- pants of the two groups acted significantly differently when interacting with the vessels.
The results show no statistical evidence to claim the influence of MASS confidence over the decision when interacting with the three types of vessels.
Decision-Making While Interacting With Unmanned Vessels 589
N of Items 6 4 3 3 2 6 Mean 3.2818 3.1045 2.5515 2.8636 3.8182 3.60 3.2033 Std. Deviation .55747 .72186 .83065 .66406 .75031 .890 .50623
Variance .311 .521 .690 .441 .563 .793 .256
Skewness .330 .376 .494 .066 −.268 −.229 .134
Table 3. Index statistics for the six dimensions of MASS confidence.
Table available in the original PDF paper.
Reliability Statistics
Cronbach’s Alpha Cronbach’s Alpha Based on Standardised Items N of Items
.768 .785 6
Item-Total Statistics Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected Item-Total Correlation Squared Multiple Correlation
Cronbach’s Alpha if Item Deleted
15.9379 6.835 .714 .644 .700
Index of Reliabili- ty/Competence
16.1152 6.883 .481 .505 .742
Index of Understand- ing/Predictability
16.6682 7.389 .254 .080 .805
Index of Familiarity
16.3561 6.694 .608 .437 .713
Index Propensity to trust
15.4015 6.759 .488 .256 .740
Index Intention of Developers
15.6197 5.662 .654 .545 .692
Trust in automation
First, however, we aimed to see if the participants reacted differently regard- ing the different types of vessels. We found that independent of the MASS confidence, the target vessels’ status triggers a different response from partic- ipants, and in a few cases, this influence was statistically significant. To assess the participant’s decision, we considered that all the decisions were made to avoid the other vessel safely. Thus we measured the following factors: the closest point of approach (CPA) to the other vessel, the amplitude of heading change and Reaction Time (time taken to decide the following action). The graphics in Figure 3 show some examples of the variability in the different cases.
Although without statistical significance, as a more significant number of participants would be required for each case, the results support the initial hypotheses that the target vessel status can alter the participant decision- making process when considering the trends across the cases. In each case, the ColReg rule was to be equally applied since we were only varying the control mode of the vessel. These results also suggest that the par- ticipants might be adopting other decision-making processes than rational processes.
To better understand what could influence the decision-making process, we observed the track performed by the participants when performing similar cases on a simulator. To compare each situation, we extracted the distance variation to the target vessel, as it reflects the effects of altering speed and heading and consequently the participants’ concern with safety, i.e., passing close or far from the other vessel.
Decision-Making While Interacting With Unmanned Vessels 591
Figure 3: Mean plots of CPA (Nautical Miles) and heading change amplitude, in each case, depending on the encountered type of vessel.
By observing the results presented in Figure 4, the mean adopted track varies commonly across the cases when encountering the different types of vessels. Participants tend to pass closer to manned vessels (T1, black line) in opposition to the MASS (T2 – green line) and unknown vessels (T3 – red line). Moreover, they also tend to take more time before passing next to the MASS
Figure 4: Results from the simulators: mean plots of distances to the target vessel (Nautical Miles) and mean standard deviation in cases 1, 2, 3 and 4 (T1 – manned vessel; T2 – MASS; T3 – unknown).
and unknown vessels. Both behaviours might suggest that participants act to give more time and distance to appraise unfamiliar situations. Ultimately, these may suggest that participants are enlarging the safety barriers to cope with their perceived uncertainty.
Ultimately, the initial question persists since the outcome of the simula- tor challenges the overall positive confidence in MASS. If the users trust automation, why do they act differently in the same ColReg situation?
As AI is swiftly penetrating the expert work environment without giving time for the adaptation or training of experts, the emergence of technological surprises in maritime operations is expanding faster. Both trainees and train- ers need to embrace the issue of teaming and interacting with AI. However, we need to speed up the understanding further and implement the required transformation. Otherwise, we risk enlarging the gap between what is per- ceived and what is done, by both practitioners and organisations, losing the ability to manage the variability within safe boundaries, as pointed out by Erik Hollnagel (2014).
Limitations We might expect some bias associated with the demanding abstraction of the participant’s decisions made for each case, based on a static representation of the nautical chart and radar images and the expected effect of the new heading, speed and RT or final position after manoeuvring. To turn around these constraints, for the last trial, a simulator was developed, where the participants could immediately get feedback about the CPA and TCPA.
Decision-Making While Interacting With Unmanned Vessels 593
CONCLUSION
The results suggest that despite having a reduced familiarity with autonomous ships, the participants have a favourable opinion. However, they react differently to conventional and autonomous ships in the same situation. The way navigators react was analysed through parameters such as reaction time, course and speed variation and the Closest Point of Approach between vessels. There is a more significant discrepancy between those parameters in participants with less training, suggesting a need to address the issues of inter- action with unmanned vessels during the course program. Results from the simulators provided more precise evidence, namely when interacting with unidentified vessels, pointing out the need to design solutions for precise identification of the target vessel.
ACKNOWLEDGMENT
The authors would like to acknowledge the EU MEDEA Project under grant agreement no 787111, and the Portuguese Navy for their support.
REFERENCES
Conceição, V. P. da, Dahlman, J. and Navarro, A. (2017) “What is maritime navigation? Unfolding the complexity of a Sociotechnical System”, in Human Factors and Ergonomics Society Annual Meeting. Austin, Texas, USA, pp. 267–271. Costa, N. A. (2018) Human-centred design for maritime technology and organisa-
tional change. Chalmers University Of Technology. Evans, J. S. B. T. and Stanovich, K. E. (2013) “Dual-Process Theories of Higher Cognition: Advancing the Debate”, Perspectives on Psychological Science, 8(3), pp. 223–241. Fast, N. J. and Schroeder, J. (2020) “Power and decision making: new directions for
research in the age of artificial intelligence”, Current Opinion in Psychology, 33, pp. 172–176. Hancock, P. A. et al. (2013) “Human-Automation Interaction Research: Past, Present, and Future”, Ergonomics in Design, (April 2013), pp. 9–14. Hollnagel, E. (2014) Safety-I and Safety-II: The Past and Future of Safety Manage-
ment. Farnham, Surrey: Ashgate Publishing Limited. IMO (2021) MSC.1/Circ.1638 Outcome of the regulatory Scoping Exercise for the use of Maritime Autonomous Surface Ships (MASS), IMO - MSC. Klein, G. A. (2003) The Power of Intuition: How to Use Your Gut Feelings to Make
Better Decisions at Work. New York: Currency/Doubleday. Klein, G. A. et al. (2004) “Ten Challenges for Making Automation a “Team Player” in Joint Human-Agent Activity”, IEEE Intelligent Systems, 19(06), pp. 91–95. Klein, G. A. et al. (2005) “Common Ground and Coordination in Joint Activity”, in
Rouse, W. B. and Boff, K. R. (eds) Organisational Simulation. Hoboken, NJ, USA: John Wiley & Sons, Inc., pp. 139–184. Körber, M. (2019) “Theoretical considerations and development of a questionnaire to measure trust in automation”, Advances in Intelligent Systems and Computing, 823, pp. 13–30. Kretschmann, L. et al. (2015) MUNIN - Maritime Unmanned Navigation through
Intelligence in Networks, D9.2 Qualitative assessment. Lee, J. D. and See, K. A. (2004) “Trust in automation: designing for appropriate reliance”, Human factors, 46(1), pp. 50–80.
Lloyd”s Register (2017) LR Code for Unmanned Marine Systems. Lloyd”s Register Group Limited. Maritime UK (2022) Maritime Autonomous Ship Systems (MASS) UK Industry
Conduct Principles and Code of Practice. Available at: https://www.maritimeuk .org/priorities/innovation/maritime-uk-autonomous-systems-regulatory-working -group/mass-uk-industry-conduct-principles-and-code-practice-2022-v6/. Mayer. Roger C., Davis, J. H. and Schoorman, F. D. (1995) “An Integrative Model of Organizational Trust”, The Academy of Management Review, 20(3), pp. 709–734. Mugny, G. et al. (1995) “Judgments in conflict: The conflict elaboration theory of
social influence”, in Boothe, B. et al. (eds) Perception, Evaluation, interpretation. Seatle: Hogrefe & Huber Publishers, pp. 160–168. Onnasch, L. et al. (2014) “Human performance consequences of stages and levels of automation: An integrated meta-analysis”, Human Factors, 56(3), pp. 476–488. Parasuraman, R. and Riley, V. (1997) “Humans and Automation: Use, Misuse,
Disuse, Abuse”, Human factors, 39(2), pp. 230–253. Rasmussen, J. (1983) “Skills, Rules, and Knowledge; Signals, Signs, and Symbols, and Other Distinctions in Human Performance Models”, IEEE Trans- actions on Systems, Man and Cybernetics, SMC-13(3), pp. 257–266. doi: 10.1109/TSMC.1983.6313160. Schoettle, B. and Sivak, M. (2014) Public opinion about self-driving vehicles in China,
India, Japan, the US, the UK, and Australia, Report No. UMTRI-2014-30. Ann Arbor, Michigan, USA. Sheridan, T. B. and Verplank, W. L. (1978) Human and computer control of undersea teleoperators. Cambridge, MA. Utne, I. B., Sørensen, A. J. and Schjølberg, I. (2017) “Risk Management of
Autonomous Marine Systems and Operations”, in Proceedings of the ASME 2017 36th International Conference on Ocean, Offshore and Arctic Engineering. Volume 3B: Structures, Safety and Reliability. June 25–30. Trondheim, Norway, p. 10. Wahlström, M. et al. (2015) “Human Factors Challenges in Unmanned Ship Operations – Insights from Other Domains”, in 6th International Conference on Applied Human Factors and Ergonomics (AHFE 2015) and the Affiliated Conferences, AHFE 2015, pp. 1038–1045. Woods, D. D. and Dekker, S. W. A. (2000) “Anticipating the effects of technolog-
ical change: a new era of dynamics for human factors”, Theoretical Issues in Ergonomics Science, 1(3), pp. 272–282.
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-87-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional