Jukka Häkkinen, Iiro Törmä, Jedi Seppänen, Lukas Ob De Beke, Mirva Salokorpi (2025). Visual scanning strategies of maritime pilots during navigation. In: Advances in Human Factors of Transportation. AHFE Open Access, vol. 186, pp. 633–643. https://doi.org/10.54941/ahfe1006553
In maritime pilotage, the pilot operates the ship in cooperation with the bridge crew to ensure the ship’s safe passage through the route. In remote pilotage, the pilot does not enter the ship but works from a remote pilotage center (Grundmann et al., 2023). The latter has been studied in recent years as it offers many advantages compared to traditional pilotage, including being less dangerous for the pilots themselves and less costly. From a human factors perspective, remote pilotage presents significant challenges, as there is a risk that a remote pilot receives less information, potentially leading to reduced situational awareness. These challenges are related to changes in information sources, requiring the pilot to adapt from direct observation to relying on instruments and crew testimony. Such issues are critical, as problems with situational awareness are a common cause of maritime accidents in pilotage (Grech et al., 2002).To understand the potential problems in remote pilotage, the phases of ordinary pilotage should be well understood. Pilotage consists of several tasks such as route planning, master-pilot exchange, providing advice, ship navigation, communication and ship handling. In this study, we focus on navigation, where both the bridge crew and the pilot must maintain multiple types of situational awareness, including task, spatial, system, and social awareness. These can be further categorized into three levels: 1. Perception of the environment, 2. Comprehension of the situation, and 3. Projection of future status (Sharma et al., 2019). The types and levels of situational awareness are reflected in information requirements of pilot, which are indicated by the visual scanning of the environment.ObjectivesIn the current study, we aimed to understand the visual scanning strategies of maritime pilots operating in a simulator environment. Specifically, we sought to comprehend temporal changes in scanning strategies during different phases of navigation, enabling us to understand the information needs in various phases of navigation. This provides novel insights, as changes in visual scanning strategies have not been previously studied in maritime piloting.MethodsIn our study, each pilot and master navigated through a fairway in a simulator. Both participants were instructed to operate as they would in normal pilotage. The pilot wore a Pupil Labs Neon eye tracker, conversations were recorded with microphones, and actions were captured with two GoPro cameras. Additionally, actions were recorded in the simulator log files.DiscussionEye-tracking data revealed scan patterns that persisted throughout the route, as well as others that were specific to certain route phases, such as turning. The outside view was observed most frequently and was associated not only with understanding the ship’s direction and rate of turn but also with the pilot advising the master on current and upcoming navigational issues. A second significant finding was that the pilot confirmed commands by observing the master’s actions.Our study shows clearly why pilot-master cooperation is a sociotechnical system, where interaction with technology is intertwined with social interaction. An interesting challenge was how these social components could be maintained in remote pilotage, particularly in narrow archipelago routes where timely actions are critical.ReferencesGrech et al (2002). Human error in maritime operations: Analyses of accident reports using the leximancer tool. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 46(19), 1718–1721.Grundmann et al (2023). Use Case Remote Pilotage – Technology Overview. Journal of Physics: Conference Series, 2618(1), 012007.Haslbeck & Zhang (2017). I Spy with My Little Eye: Analysis of Airline Pilots’ Gaze Patterns in a Manual Instrument Flight Scenario. Applied Ergonomics 63: 62–71. Peysakhovich et al (2022). Classification of Flight Phases Based on Pilots’ Visual Scanning Strategies. In 2022 Symposium on Eye Tracking Research and Applications, 1–7. Seattle WA USA: ACM, 2022.Sharma et al (2019). Situation Awareness Information Requirements for Maritime Navigation: A Goal Directed Task Analysis. Safety Science 120, 745–52.
Maritime pilots are experienced seafarers who board vessels to assist the ship’s master and navigation team for the final miles before reaching the port (Hadley, 1999). These waters are often challenging for safe navigation and manoeuvring. The fairways located near the coastline and ports can be shallow, winding and narrow, and the volume of traffic on these channels is often high. In addition, navigation can become even more challenging due to various factors: tides, currents, and wind conditions, as well as situations where vessels must reduce speed for other reasons, all demand even greater skill from the ship’s officers and masters. The pilot is highly familiar with these local conditions (Hadley, 1999) and speaks the local language, enabling effective communication with other vessels in the fairway and with Vessel Traffic Service providers who monitor the traffic. Pilots are also generally experts in ship navigation and over the years they gain experience in handling different types of vessels (Berlin & Praetorius, 2023). Pilotage is fundamentally a matter of risk management (Wild, 2011).
The pilot’s main duty is to ensure that the vessel being piloted stays in the fairway and it can be navigated safely into or out of the port. The way this is carried out on board a ship depends on the preferences and level of expertise of the ship’s crew. Most often, the pilot is responsible for the
practical implementation of the vessel’s navigation and steering, giving the ship’s crew orders for each change in course and speed.
Pilotage is a very old profession and there is no indication that pilots will not be needed in the future. With digitalization, new options for piloting a ship have also been identified, namely remote pilotage. The ship would then be piloted from a shore-based center. The pilot would receive information about the vessel’s movements and its surroundings using various sensors and information systems and could be in contact with the vessel, for example by telephone. In addition to the technology enablers, important drivers for the development are also the need to improve pilots occupational safety and reduce environmental emissions in maritime traffic.
The implementation possibilities of remote pilotage have been tested and studied in several countries around the world, for example in Singapore, Denmark, Sweden, Finland, and Spain (Dimecc, 2023; HHI, 2021). In all tests, the technical and operational implementation methods have been slightly different, but what is common to all of them is that the costs of technology development, service implementation and new equipment needed for the ship have been kept as low as possible. This is understandable, but it is also very important to ensure that the new service format would produce added value and improve navigation safety in the future. A research project called Remote Pilotage MVP is underway in Finland, which has begun to study in more detail what kind of information pilots need and how it should be presented so that the pilot’s situational awareness remains good throughout the pilotage.
In remote pilotage, the pilot does not enter the ship but works from a remote pilotage center (Gsessiondmann et al., 2023). The latter has been studied in recent years as it offers many advantages compared to traditional pilotage, including being less dangerous for the pilots themselves and less costly.
Situational Awareness Operating a ship consists of multiple simultaneous tasks, as the master must navigate the ship, monitor its systems, and understand what the rest of the crew is doing at the same time. In addition to these routine tasks, additional demands may arise, such as responding to a radio message, answering a crew member’s question, or moving to a different location on the bridge, all of which must be managed alongside the core tasks.
In this complex environment, it is important that the master understands the factors that enable the ship to steer safely. This understanding is called situational awareness and is typically divided into three components: Spatial awareness refers to understanding the position and movement of one’s own ship, other ships, and other targets, as well as the factors affecting them. Task awareness involves understanding the task goals. System awareness means understanding the operational status of the ship’s systems.
Visual Scanning Strategies of Maritime Pilots During Navigation 635
When a pilot cooperates with a master, they share situational awareness, which, under optimal conditions, is similar. Differences in any aspect of situational awareness create a risk that may endanger the ship’s steering. Therefore, it is important to consider to what extent the pilot’s and master’s situational awareness are similar and what kind of information the pilot uses to maintain awareness.
An additional factor is the understanding of social interaction on the ship. The master is continuously present, allowing for a well-developed understanding of the crew’s status and capabilities. The pilot, on the other hand, has only a short time to form an initial impression of the crew’s competence, fatigue, and morale and to align pilotage accordingly. However, social observation continues beyond the initial impressions, and the pilot’s understanding can be updated through observations of the crew in different situations. Therefore, this could be referred to as social situational awareness to emphasize its continuity throughout the pilotage and align it with other aspects of situational awareness.
Each aspect of situational awareness operates on three interrelated levels. The first level, perception of the environment, involves selecting and detecting task-relevant information. The second level, comprehension of the situation, refers to forming an understanding and making decisions. The third level, projection of future status, involves predicting the future state of the ship and other vessels (Endsley, 1995; Sharma et al., 2019).
Maintaining situational awareness in remote pilotage is challenging, as the information available to the master and the remote pilot differs. For example, a remote pilot does not hear wind or engines, nor experience the ship’s vibrations, which provide quick and intuitive information that aids steering. Similarly, the ability to observe the captain’s actions, assess crew interactions, and check the environment through the ship’s windows is significantly reduced. As a result, remote pilots must adapt how they form their situational awareness. Addressing these challenges is crucial, as deficiencies in situational awareness are a common cause of maritime accidents (Grech et al., 2002).
Navigation and Pilot Information Needs To understand the potential problems in remote pilotage, it is essential to clearly understand the phases of ordinary pilotage, which include route planning, master-pilot exchange, providing advice, ship navigation, and communication.
In this study, we focus on navigation, which has been divided into four phases: preparation, turn, control, and transit (Hareide & Ostnes, 2017). In the preparation phase, the navigator prepares for the turn and identifies variables relevant to its execution. In the turn phase, the turn is executed, while in the control phase, its accuracy is assessed. In the transit phase, the ship proceeds to the next wheel-over point (WOP).
Our scenarios represent a specific case of navigation, as the pilot acts as an advisor to the master, who is responsible for the navigation. In this case, the pilot needs to maintain situational awareness as in ordinary navigation
but also needs to decide when to communicate important matters to the master while simultaneously observing the master conducting the navigation. The navigational task is thus indirect and has a social component. One way to characterize the information needs of a pilot in this situation is by analysing the scan patterns of the eyes. For example, Hareide et al. (2016) collected eye movement data from high-speed Royal Norwegian Navy Corvette navigators, showing that they spent 65% of their time looking outside and 27% at the ECDIS. Some attention was also directed toward the radar (4%) and conning display (3%). In another study, Lounis et al. (2024) differentiated the visual scanning strategies of airplane pilots based on their expertise, demonstrating that experts exhibited higher perceptual efficiency compared to novice pilots.
Study Objectives This study aimed to explore the information needs of a pilot advising a master by examining the pilot’s visual scanning strategies during a simulator-based navigation exercise. Specifically, we sought to identify key information areas and gain insight into the cognitive processes underlying the pilot’s decision- making in an advisory role.
METHODS
Participants Four male pilots, aged 48 to 50 years, participated in the study. Each participant was an experienced mariner with 20 to 26 years of seagoing experience as a master or pilot. All had prior experience using the simulator bridge. The pilots used a tablet containing a map of the fairway, which they regularly use in their work. This served as a replacement for the ECDIS as the primary source of map information.
Simulator The exercises were conducted at the Aboa Mare simulator center in Turku, Finland. The simulator center operator initiated the exercise, after which the participants on the simulator bridge had full navigation capability to operate the simulated vessel. The simulator bridge was equipped with a Furuno integrated navigation system, including radar, an ECDIS chart computer, an autopilot, video screens for the external vessel view, and all necessary indicators used by vessel navigators, such as a rate of turn indicator, rudder indicators, and engine parameters (Figure 1). The simulator bridge view could be rotated, allowing participants to observe a 360-degree view around the vessel using viewpoints from the center of the bridge and the bridge wings. This was controlled from the bridge’s central display.
Visual Scanning Strategies of Maritime Pilots During Navigation 637
Figure 1: Simulator bridge.
The simulator modeled the vessel’s hydrodynamic properties using a computer model. Environmental conditions, including wind, waves, and visibility, were set by the simulator operator at the beginning of the exercise. The pilot participating in the simulator exercises, who was familiar with the area, specified a typical wind that did not create challenging steering conditions, and visibility was set to be good.
Ship The selected ship, Viikki, is a bulk carrier operating under the Finnish flag. The vessel has a maximum draft of 15 meters, a length of 160 meters, and a deadweight tonnage (DWT) of 25,532 metric tons. Propulsion is provided by a single main engine with an output of 7,250 kW. The superstructure is positioned aft, optimizing space for cargo operations. Three loading cranes are installed on the port side, which may create visual obstructions during certain maneuvers. The simulated ship is a model of an existing ship, which will be used as a technical test platform in the Remote Pilotage MVP project.
Eye Tracking Eye tracking data were collected with Pupil Labs Neon eye-tracking glasses, which have a 200 Hz sampling rate. We measured the interocular distance of the participants using a pupillometer and set this value in the analysis system. The participants also fixated on a calibration target before the experiment, allowing us to assess potential offsets in the Pupil Cloud analysis system and apply necessary compensations.
Procedure Each participant selected a fairway for which they held a valid piloting license and chose a section featuring multiple turns, lasting approximately 60 minutes. The actual simulator exercise time varied from the pilot’s initial estimates, as shown in Table 1. In the simulator, one of the pilots acted as the master, steering the vessel along the selected route, while the other served as the pilot. The master received the planned route before the study. After completing the planned session, the participants took a 60-minute lunch break before switching roles, allowing participants to take turns as pilot and master.
Table 1: Routes.
Table available in the original PDF paper.
Route Name Route Length (km) Number of Turns Duration (Min) Hamina 26 7 69 Kokkola 15 5 50 Turku 19 7 57 Uusikaupunki 17 5 46
Upon arriving at the simulator, participants completed a background questionnaire and signed an informed consent form. They were instructed to behave as they would in a real piloting scenario, with the pilot advising the master, who was steering the ship. Additionally, they were informed to assume that the master had extensive seagoing experience but no familiarity with the specific fairway.
The pilot wore Pupil Labs Neon eye-tracking glasses to capture eye movement data. Additionally, two GoPro cameras mounted on the simulator’s ceiling recorded the participants’ actions. Conversations were recorded using Saramonic wireless microphones.
When analysing the eye-tracking data, we asked the pilots to explain why they looked at some locations for clarification.
RESULTS
Data Analysis We defined AOIs to all information displays, indicators and controls, resulting in total of 41 areas of interest. In the radar display we defined six AOIs. One was the central radar screen, while the other four were the corners of the display, each presenting a different type of information. The outside view was defined as a single AOI. We also defined master’s face and hand as areas of interest to capture social interaction during the simulator exercise. A manual fixation-to-AOI mapping was performed using the Pupil cloud analysis environment. A total of 25434 fixations from four participants were mapped to AOIs.
Figure 2 shows the dwell time for various AOIs for the four participants as a percentage of the total dwell time in the simulator exercise. There were 11 AOIs that received more than 1% of the total dwell time along with an additional 15 areas that accounted for a smaller amount of time. The dwell times show that pilots looked mostly outside but also at the pilot’s tablet, which displayed the map, radar and the top-right radar information area, containing details such as heading, speed, course over ground and speed over ground (Figure 3a). Pilots also looked at the master’s autopilot (Figure 3b).
All AOIs that start with “Top” refer to the indicators on the top panel (Figure 4). These include rudder angle, rate of turn (ROT), true heading, ROT visualization, and speed and depth indicators. Finally, fixations on captain’s face occurred 4.28% of the time, indicating a significant social component in pilotage.
Visual Scanning Strategies of Maritime Pilots During Navigation 639
Figure 2: Dwell time on AOIs. Dwell time represents the total duration a participant’s gaze remains within a specific area of interest (AOI).Figure 3: (a) Radar area R3 at the top right corner of the radar. (b) Master’s autopilot.Figure 4: Top panel of indicators.
To gain a deeper understanding of AOIs we further analyzed their temporal patterns and asked the pilots why they scanned certain areas. We selected one representative participant and analyzed the scan patterns in more detail. Figure 5 presents a sequence chart that shows fixations on six AOIs during the simulator session for a single participant. The x-axis shows fixations
ordered chronologically, with each vertical line representing one fixation. The numbered vertical lines indicate the moments when turning was initiated verbally either by the master or the pilot. When interpreting the visualization, it is worth noting that each row contains over 7000 fixations, so even thin lines may represent a large number of fixations.
Fixations on the outside view (Figure 5a) indicated that visual attention to the external environment remained consistent throughout the simulator session. Periods of prolonged disengagement from the outside view were typically associated with increased focus on the pilot’s tablet or the central area of the radar. Interviews with the pilots and analysis of the simulator session recordings suggested that the outside view was utilized differently during various phases of the session. First, it was used to verify that the ship was moving in the correct direction relative to visible landmarks. Second, it facilitated a shared understanding of upcoming steering needs, as the pilot pointed out navigational aids and leading marks, indicating suitable points to initiate a turn or the most favorable route to follow. Third, the outside view was used to confirm that a turn had started correctly, as it provided immediate feedback on the vessel’s movement.
Figure 5: The simulator session timeline shows areas of interest in timeline order for a one participant. The x-axis indicates the percentage of total time during. The AOIs are: a. Outside view, b. Pilot’s tablet, c. Radar central area, d. Top panel (all indicators), e. Master’s autopilot, f. Master’s face.
Spatial situational awareness was maintained through a scan pattern involving the outside view, pilot’s tablet, central radar area, and radar information area R3. Fixations on the pilot’s tablet (Figure 5b) and the central radar area (Figure 5b) occurred consistently throughout the
Visual Scanning Strategies of Maritime Pilots During Navigation 641
simulation session, making them, along with the outside view, the primary sources of spatial situational awareness information.
Fixations on the top panel (Figure 5d) typically occurred once the turn had started and was in progress. The primary information source when the turn was about to begin was the master’s autopilot (Figure 5e). When the pilot anticipated the turn, they initially checked the autopilot readings. As the master adjusted the autopilot settings, the pilot observed these actions. Finally, the autopilot’s operation was monitored throughout the turn.
The social nature of the piloting was indicated by the frequent fixations on the master’s face (Figure 5f), which were associated with both navigational discussions and small talk. The role of small talk increased during the less demanding parts of the route, especially toward the end of the simulator session when the route became simpler and cognitive load was lower.
CONCLUSION
To maneuver a ship effectively, the seafarer must maintain a situational awareness of their surroundings, which means understanding the location and movement of the ship as well as anticipating future positions relative to obstacles. In pilotage, situational awareness must be shared for navigation to be successful.
In our study, pilots’ visual scan patterns indicated that the most important enabler of shared situational awareness was a common view of the outside world, which was clearly the dominant source of information. Looking outside is effective, as the external environment provides immediate and precise information about the ship’s course. Furthermore, the pilot can effortlessly and naturally point out significant landmarks and buoys to advise the master on the current situation and necessary actions in the near future. Our findings align with those of Hareide et al. (2017), who demonstrated that the navigators of a high-speed corvette spent majority of their time looking outside.
The significance of the outside view may pose challenges in remote pilotage, where the pilot has a limited or no direct view of the ship’s surroundings. This not only affects situational awareness but also makes it more difficult to communicate critical information about the ship’s external environment to the master in a seamless and natural manner. There appears to be a need for innovation to facilitate the effective sharing of information about significant objects and navigational elements outside the ship.
The second significant issue in our results was the pilot’s observation of the master’s behavior. The pilot received immediate confirmation of the master’s actions through direct observation, particularly when the autopilot was in use. Although the pilot employed a closed loop communication method where the master repeated the pilot’s course suggestion, the pilot also visually verified that the master executed the command correctly by checking the autopilot when the course was being adjusted. In other words, the pilot ensured that the master’s actions aligned with verbal communication. This provided a convenient way to detect potential execution errors. Without this
verification, errors would only become apparent later when observing the ship’s movements.
In our scenarios the visual confirmation of actions occurred primarily in turning situations when the pilot monitored the autopilot, but there were also other occasions of checking where the master operated the engine telegraph lever or used manual steering.
Pilots also regularly checked whether the autopilot responded appropriately to course changes and mentioned deviations to the master or revised their orders. In these cases, there were discussions about the autopilot’s capabilities, such as whether the autopilot of this specific ship was “lazy” or whether specific wind conditions affected its behavior. This was an effective way to maintain a shared awareness of the ship’s capabilities.
Our study shows clearly why pilot-master cooperation is a sociotechnical system, where interaction with technology is intertwined with social interaction. An interesting challenge was how these social components could be maintained in remote pilotage, particularly in narrow archipelago routes where timely actions are critical.
ACKNOWLEDGMENT
The project was supported by Business Finland project 6454/31/2023.
REFERENCES
Berlin, C.& Praetorius, G.(2023) Applied Cognitive Task Analysis (ACTA) of marine piloting in a Swedish Context. 14th International Conference on Applied Human Factors and Ergonomics (AHFE 2023). Dimecc Oy (2023) Sea4Value – Future Fairway Navigation Final Report 2/2022.
Dimecc Publication series. Endsley, M. R. (1995) Toward a theory of situation awareness in dynamic systems.
Human Factors 37(1) 32–64. Grech, M., Horberry, T. & Koester, T. (2002) ‘Human error in maritime operations: Analyses of accident reports using the Leximancer tool’, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 46(19), pp. 1718–1721. Gsessiondmann, R., Ujkani, A., Weisheit, J., Seppänen, J., Salokorpi, M. &
Burmeister, H.-C. (2023) ‘Use case remote pilotage – technology overview’, Journal of Physics: Conference Series, 2618(1), p. 012007. Hadley, M. (1999) Issues in Remote Pilotage. The Journal of Navigation, 52(1). Hareide, O. S., Ostnes, R. & Mjelde, F. V. (2016) Understanding the Eye of the Navigator. In: European Navigation Conference, Helsinki. Confedent International. Hareide, O. S. & Ostnes, R. (2017) Scan Pattern for the Maritime Navigator.
TransNav, the International Journal on Marine Navigation and Safety of Sea Transportation 11, no. 1 (2017): 39–47. HHI (2021) Project “2018-086, VesCo Systems, Remote Pilot” Project Progress Report. Lounis, C., Peysakhovich, V., & Causse, M. (2021) Visual scanning strategies in the
cockpit are modulated by pilots’ expertise: A flight simulator study. PLoS one, 16(2), e0247061.
Visual Scanning Strategies of Maritime Pilots During Navigation 643
Sharma, A., Nazir, S. & Ernstsen, J. (2019) ‘Situation awareness information requirements for maritime navigation: A goal-directed task analysis’, Safety Science, 120, pp. 745–752. Wild, R. J. (2011) The Paradigm and the Paradox of Perfect Pilotage. The Journal of
Navigation Vol. 64, pp. 183–191.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Factors of Transportation
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-62-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional