Hanna Koskinen, Jari Laarni, Satu Pakarinen, Tomi Passi, Kristian Lukander, Tuisku-Tuuli Salonen, Marja Liinasuo (2024). Initial Operating Experiences in a Digital Control Room. In: Human Factors in Software and Systems Engineering. AHFE Open Access, vol. 140, pp. 144–152. https://doi.org/10.54941/ahfe1005047
In spring 2023, a new nuclear power plant (NPP) unit become operational in Finland and was connected to the national grid for electricity production. The turnkey construction of the unit was closely followed not only by the nuclear community but also the public. There are many features that make the new unit unique and of special interest. For example, the process control is carried out through a fully digital human-system interface (HSI). In nuclear industry, high safety and quality demands are set for the control room (CR) design and HSI systems to be used in the operation of the power plant. Before taken into use, a comprehensive quality and safety assurance through formal verification and validation (V&V) process of HSIs should be carried out to ensure that the new control system design continues to support safety and usability (YVL B.1, 2013). NUREG-0711 (O’Hara et al., 2012) review guide also set out several human factors activities that should be accomplished during the life cycle of the control system, for example, carrying out the early design phase analyses and allocation of functions to human and automation. In the case of the new NPP unit, in-service monitoring of the CR and HSI systems is a topical human factors activity after the commissioning. This is an important human factors activity even often less discussed since through it, for example training needs and important issues to be addressed in the product development process can be identified.We have conducted a walkthrough exercise of the new CR and HSI systems and a questionnaire study to collect initial operating experiences in the new digital CR and to support setting up the in-services monitoring activity regarding the operation of the new unit. Based on the results of the walkthrough, a systems usability questionnaire was developed. The questionnaire was distributed to the operators of the NPP.In the paper, we report the results of the HSI walkthrough and the survey. According to the simulator trainers, some new HFE issues have emerged after commissioning that were nor addressed in validation tests.Contrary to the handling of incident and accident situations that are central in V&V tests the main focus of in-service monitoring concerns normal operative activities. Topics that were addressed include, among others, preferences for different formats (paper vs. computer-based) of procedures, the use of two languages (English in HSIs and procedures and Finnish in normal conversation), and the needs for additional training.REFERENCESSäteilyturvakeskus: YVL B.1: Safety design of a nuclear power plant, 15 November 2013. Chapter 5.3: Control rooms.J. M. O’Hara et al., “Human Factors Engineering Program Review Model,” NUREG-0711, Rev. 3, U.S. Nuclear Regulatory Commission (2012).
In spring 2023, a new nuclear power plant (NPP) unit became operational in Finland and was connected to the national grid for electricity production. The turnkey construction of the unit was closely followed by the nuclear commu- nity, but it also raised interest among the public. There are many features that
Initial Operating Experiences in a Digital Control Room 145
make the new unit unique and of special interest. For example, and what is most interesting in terms of the present study, the process control is carried out through a fully digital human-system interface (HSI).
In the nuclear industry, high safety and quality demands are set for the control room (CR) design and HSI systems to be used in the operation of the power plant. Before a CR is taken into use, a comprehensive quality and safety assurance through a formal verification and validation (V&V) process of HSIs are to be carried out to ensure that the new control system design continues to support safety and usability (YVL B.1, 2013).
NUREG-0711 (O’Hara et al., 2012) review guide also sets out several human factors activities that should be accomplished during the life cycle of the control system, such as carrying out the early design phase analyses and allocation of functions to human and automation. In the case of a new NPP unit, in-service human performance monitoring of the CR and HSI sys- tems is a topical human factors activity after commissioning, since through it, for example training needs and important issues to be addressed in the prod- uct development process can be identified. Even though it is an important activity, often less attention is paid to it than the other HFE activities.
In this study, we first conducted a walkthrough exercise of the new CR and HSI systems, then developed a systems usability questionnaire based on the results of the walkthrough, and finally distributed the questionnaire to the CR operators of the new NPP. The aim was to collect initial operating experiences in the new digital CR and to support setting up the in-service human performance monitoring activity regarding the operation of the new unit.
Contrary to the handling of incident and accident situations that are central in V&V tests, the particular focus of in-service monitoring con- cerns normal operative activities. Topics that were addressed include, among others, preferences for different formats (paper vs. computer-based) of proce- dures, the use of two languages (English is used in HSIs and procedures and Finnish in normal conversation), and the needs for additional training.
Here, we report the results of the HSI walkthrough and the survey, repre- senting the collection of initial operating experiences from novel NPP, with a fully digital control room.
RELATED WORK
According to EPRI 3002004310 (2015), an in-service human performance monitoring program aims to verify the systems and HSI modification’s proper functioning, detect any issues with the new systems and HSIs after their instal- lation and regular use, and find any areas that require improvements. After identifying the issues requiring improvements and analyzing the root causes of the problems it is essential to initiate a process to design and execute appropriate solutions. This involves selecting the most effective and feasible interventions (e.g., additional training or product development), assigning responsibilities and resources, setting schedules, and monitoring progress. The goal is to eliminate or reduce the negative impacts of the problems and
prevent their recurrence. Thus, contrary to the handling of incident and acci- dent situations that are central in V&V tests, the particular focus of in-service monitoring concerns the human performance in normal operative activities. Consequently, it is important to follow and monitor how the system is oper- ated and experienced by the operators and if any specific concerns or issues arise.
As compared with conventional analogue and hybrid (i.e., containing both analogical and digital technologies) NPP control rooms, fully digital con- trol rooms offer several advantages, such as higher reliability, better plant performance, integration of information from several sources, flexibility in information presentation and additional diagnostic capabilities (e.g., decision aid). For instance, computer-based procedures have been shown to provide a wide range of benefits, making human errors of commission less frequent than when using paper-based procedures (Lin, Hsieh, Yang & Huang, 2016).
However, digital CRs also have been suggested to pose some new chal- lenges. For example, a fully digital CR based on visual monitoring displays may cause the loss of the ability to operators utilize well-learned, rapid eye scanning patterns and pattern recognition from spatially fixed parameter dis- plays that has been one of the advantages of the conventional analogue HSIs (O’Hara and Hall, 1992). Moreover, in a digital CR, it may be more difficult to maintain shared situational awareness among operators as they are not anymore required to move around the CR and execute control operations on the desks and wall-mounted operating panels. This conventional concept of operations allowed the crew members easily to be aware of others’ actions which supported the creation of shared situation awareness and task coordi- nation. In order to mitigate the effects of reduced shared situation awareness in digital CR, special communication practices must be developed and intro- duced (e.g., three-way communication). Another option is to provide an opportunity to monitor other operators’ actions on computer screens.
There may also be difficulties in navigating through and finding relevant information presented on the monitoring displays. With multiple screens and hundreds of graphics to navigate through, the operators may face challenges in quickly locating and interpreting relevant information. Consequently, the so-called secondary tasks (e.g., accessing information and navigation through displays) may require more operators’ cognitive resources which may thus increase operators’ cognitive load as well as reduce their situational awareness.
The keyhole effect is one problem operators may encounter in the con- nection of digitalized CRs in which the visual display monitors serve as a main HSI. A keyhole is a metaphor for the operator’s limited view of the process. That is to say, the operator only sees a small part of the controlled system through a display monitor, like looking through a keyhole only shows a small part of a room. Thus, the keyhole effect refers to the limited view that operators have of the NPP process when they rely on multiple screens and graphics to navigate through the data (Woods et al., 1990). For exam- ple, this can be particularly challenging in situations when several alarms are activated simultaneously, and the operators need to navigate through sev- eral monitoring displays to comprehend the process situation and to respond
Initial Operating Experiences in a Digital Control Room 147
appropriately. Thus, the keyhole effect can negatively affect the operator’s performance and decision making, and cause delays or potential operating mistakes, as the operator may not have a complete understanding of the sys- tem’s state. To avoid such errors and defects in process control activity it is important to develop strategies and techniques to mitigate the keyhole effect.
Other possible challenges of digital CRs have also been identified, such as reduced tactile feedback (i.e., analogue interfaces provide richer tactile feedback than digital interfaces), increased separation from the process (i.e., analogue panels surrounding operators are also mimic displays showing a geographical structure of the power process), and reduced collaboration among operators (i.e., operators’ attention is narrowed down to focus on the displays of their own workstation, less attention is allocated to other oper- ators and their actions). Finally, the adoption of advanced technologies in process control such as highly automatic functionalities or operator aids and the operator’s difficulties in understanding how these systems work can lead to either overreliance or lack of acceptance of these digital systems (O’Hara and Hall, 1992).
All the research above has made it clear that it is necessary to understand the phenomena related to digital CRs more deeply and to accumulate and document practical operating experiences on them. In creating new under- standing and knowledge base on human performance in digital CRs, the in-service human performance monitoring plays a crucial role.
METHOD AND MATERIALS We have conducted a walkthrough exercise of the new CR and HSI systems and a questionnaire study (Table 1) to collect initial operating experiences in the new digital CR and to support setting up the in-service human performance monitoring activity regarding the operation of the new unit.
Table 1. Data collection.
Table available in the original PDF paper.
Participants Method Task Description & Used Materials
CR and HSI walkthrough
Researchers (3) and trainers (2)
A systematic walkthrough of the CR and all main HSIs was conducted so that the related functions and roles were explained
Researchers (3)
Review of design documentation
The design documentation related to CR V&V process and results were reviewed
Systems Usability questionnaire
Operators 26(30), response rate was 87%
60 questions containing questionnaire was developed that addressed CR and HSI systems
Control-Room Walkthrough and Review of Documentation A walkthrough exercise of the new CR and HSI systems was conducted in autumn 2023. Three researchers and two experienced trainers/simulator instructors participated in the walkthrough exercise in the new unit’s full scale training simulator that was a high-fidelity replica of the actual main CR. Before the walkthrough exercise, the participating trainers gave their written informed consent. The study procedures adhered to the Declara- tion of Helsinki. The ethical statement was obtained from the board of VTT Technical Research Centre of Finland.
In the walkthrough, all the main HSIs were introduced and discussed from the perspectives of their function, control, and user experience. At the begin- ning of the walkthrough exercise the basic layout of the CR and the dedicated workstations of the different operator roles (i.e., reactor operator, turbine operator and shift supervisor) were introduced after which individual HSIs such as the main HSI and backup HSI, operating procedures, large screen displays, and alarm displays were viewed and discussed one after another.
The walkthrough exercise was video- and audio-recorded and the researchers made notes during the exercise.
Following the walkthrough, a review of relevant design documentation (e.g., V&V documentation and operating procedures) were conducted.
Systems Usability Questionnaire Based on the results of the walkthrough and a review of relevant design doc- umentation (e.g., V&V documentation and operating procedures), a systems usability questionnaire was developed. A questionnaire containing altogether 60 questions was introduced. Most of the questions were in the form of a spe- cific claim about the CR and HSI; the claims were asked to be rated in a scale from 1 to 5 where “1” is strongly disagreeing and “5” strongly agreeing. An example of such a claim is as follows: “Based on my operating experiences, I have noticed development needs in HSI”. In addition, the questionnaire included a few open questions in which the respondents could specify and give justification to their rating of the previous claim or provide additional information and raise up some issues of importance to them.
In the beginning of December 2023, the questionnaire was distributed to all operators of the NPP’s new unit via QuestbackTM (www.questback.com) online survey platform. 26 operators out of the 30 in the unit completed the questionnaire.
Data Analysis Audio recordings from the walkthrough exercise were listened, and video recordings were watched by the researchers, and main themes were identified from the audio and video data.
Central operative scenarios, critical operator tasks and human discrep- ancy categories as well as problem solutions suggested during the validation process were identified from the relevant design documents.
The inspection of the survey results is based on the distribution of the responses over the response options.
Initial Operating Experiences in a Digital Control Room 149
RESULTS
In-service human performance monitoring aims to identify possible prob- lems in the use of CR systems. As stated in EPRI 3002004310 (2015), it is typically based on opinions of operating personnel. A special activity for collecting this type of operating experiences is required, because end-users do not necessarily tell their opinions without motivating them. Contrary to the handling of incident and accident situations that are central in V&V tests the in-service human performance monitoring focusses on how the CR and the HSIs support normal operative activities.
Generally, based on the questionnaire responses, the operators are satisfied with the CR and the HSIs. Operators thought that the CR and the HSIs serve the process control well and support them to maintain an appropriate level of situation awareness and carry out their daily operative tasks. However, some areas for improvements regarding individual HSIs were also identified by the operators, such as the placement of phones and dysfunctional camera system. Some respondents also hoped that alarm prioritization and management in general would be improved and data retrieval would be simplified.
Next, four topics are addressed, having implications for the challenges of digital CRs which are described in Related work -section, such as devel- opment and maintenance of shared situation awareness in a digital CR, increased secondary task load, keyhole effect and reluctance and resistance to some digital tools (e.g., computer-based procedures).
Communication and Situation Awareness Regarding the claimed disadvantage of fully digital CRs according to which maintaining shared, team-level situation awareness is more difficult as it is no longer so easy to observe and monitor the other operators’ actions (compared to the operation of analogue panels where the physical location of an oper- ator directly indicates the system being operated) the questionnaire did not provide much support. Instead, based on the responses the general view was that other members of the operating crew were able to follow the progress of the work through the control room HSIs. The operators also generally thought that the CR supported the cooperation and coordination of activities between operators.
However, the questionnaire data did not shed light on how and from which sources of information the shared situational awareness was formed and maintained. As an implication for in-service human performance monitoring, we propose that the survey should be supported by interviews in which more detailed information about operators’ work practices and operator-system interaction can be collected.
Process Monitoring and Operation The so called “keyhole effect” is often discussed in the connection of pro- cess monitoring using a display-based CR and HSIs. The number of display screens in the new unit’s operator workstations was already increased from the original setup due to the operating experience. The questionnaire data
gave the impression that the current number of screens is considered suffi- cient, but the selection of the operating displays to be kept visible had been carefully considered so that the operators’ information needs would be satis- fied. Thus, it appears that, at least in this case, the keyhole effect is not seen as a major problem with digital HSIs.
Furthermore, large-screen displays are used to provide an overview of main process parameters for the operators through shared display screens, and they are common in modern control rooms. Also, in the studied CR they had three large-screen displays in use. The operators reported that they actively use the large-screen displays to build and maintain situational awareness in different operating situations of the facility. Consequently, it seems that the operators have adopted the large screen displays as an integral part of their process monitoring activities/routines. The displays provide still one essential measure against the limited view over the process in the display-based CR. In order to get more information about how situation awareness is devel- oped and maintained in a digital CR, operator performance should be studied through simulator experiments.
According to the survey results, the operators did not seem to have big problems with secondary task load. But more detailed information should be collected through simulator tests to more precisely evaluate the effect of dig- ital HSIs on process control. For example, little is known about what degree secondary task load could be mitigated by means of display design, overview displays, functional hierarchy of information presentation, and smart alarm management.
Operating Procedures Electronic procedures (i.e., computer-based systems that guide the opera- tors in performing tasks in various operative situations) are designed to improve the human performance, efficiency, safety, and reliability of the plant operation and they are suggested to provide several benefits such as the inte- gration of real-time process information and the delivery of feedback on the operation execution status.
The new unit has both paper and computer-based procedures available for the operators and they can select freely, based on their personal preference, which one to use. The questionnaire responses indicated that the operators preferred to use the paper-based procedures. They considered themselves as more familiar and comfortable with the paper-based procedures, and they have been trained to use them in the simulator. Moreover, the operators find the paper-based procedures easier to examine, annotate, and navigate, espe- cially in complex and lengthy operating tasks. These experiences may have been affected also by the technical issues with the electronic procedures, such as failures in opening or scrolling the procedure. One additional reason that was mentioned in favour of the paper-based procedures was that the oper- ators did not want to sacrifice one of their displays of important process information for the electronic procedures. There is some recent literature on the use of paper-based and computer-based procedures that shed light on this topic.
Initial Operating Experiences in a Digital Control Room 151
Operating Language From the safety perspective it is considered important that the operators can use their native language in their work tasks (e.g., in communication and different HSIs), thus without any exception this has been the convention in the CRs of Finnish NPPs.
In the present case, all displayed HSI information is in English, but operators speak Finnish. It could be thought that this causes a lot of mis- understandings and confusion, but it does not seem to be the case. Only few of the respondents thought that there were some miscommunications caused by language. Presumably, extensive training in which both Finnish and English are used made them proficient in changing language on the fly. Apparently, simulator studies can also provide useful information about the possible language-related misunderstandings and their effects on operator performance.
DISCUSSION
The collection of initial operating experiences on the NPP’s new unit through a walkthrough and a systems usability survey provided important and partly also unexpected results. As a whole, the results of the in-service monitoring activity proved to be highly informative as in it, such human factors engineer- ing issues were identified that were not addressed in the V&V tests. However, for some survey items, responses were dispersed between several response alternatives, indicating that respondents either disagreed about the topic or they understood the statement differently.
The feared keyhole effect (Woods, Roth, Stubler & Mumaw, 1990) was not as a big problem as was anticipated. The display screens of an opera- tor workstation with the supporting large-screen displays seem to provide enough information to the operators about the nuclear process. Somewhat astonishingly, in contrary with the results of Lin, Hsieh, Yang & Huang, 2016, the computer-based procedures were not much/strongly supported by the operators. This may be due to technical deficiencies the early versions of such procedures may exhibit (difficulties in scrolling the digital proce- dures etc.); these deficiencies can be amended by careful development during the early phases of the operation of the NPP. It seems that the form of the procedures is not as important as the content; the mere digital form of a pro- cedure does not make the procedure better. For instance, the outdated look of some computer-based procedures is not a feature that depends on the digital form of the procedure. Interestingly, many of the operators thought that the paper format made the procedure easier to use. The reason for this finding (e.g., whether it is caused by a fundamental weakness in their usability or the novelty of digital procedures) remains to be seen.
CONCLUSION
The collection of initial operating experiences through a walkthrough, an interview and a survey seem to be a viable mean to get an overview of the status of the usability of the CR and HSIs and, furthermore, a way to identify
development needs important to tackle. The in-service monitoring activity was conducted only about a year after commissioning of the NPP’s new unit, demonstrating the importance and feasibility of the activity soon after the commissioning. It also provides a benchmark for future in-service monitoring interventions.
ACKNOWLEDGMENT
The authors would like to acknowledge all operators, trainers, and human factors experts in the NPP sharing their experiences, knowledge, and insights on operation of the new unit. The research was funded by SAFER2028, the Finnish research program on nuclear power plant safety.
REFERENCES
EPRI 3002004310 (2015). Human Factors Guidance for Control Room and Digital Human-System Interface Design and Modification. Technical report. Electric Power Research Institute EPRI, Palo Alto, CA. Lin, C. J., Hsieh, T. L., Yang, C. W., & Huang, R. J. (2016). The impact of computer-
based procedures on team performance, communication, and situation awareness. International Journal of Industrial Ergonomics, 51, 21–29. O’Hara, J. M. and Hall. (1992). Advanced Control Rooms and Crew Performance Issues: Implications for Human Reliability. IEEE Transactions on Nuclear Science, Vol. 39, No. 4. O’Hara, J. M. et al. (2012). Human Factors Engineering Program Review Model.
NUREG-0711, Rev. 3, U. S. Nuclear Regulatory Commission. Säteilyturvakeskus. (2013). YVL B.1: Safety design of a nuclear power plant, 15 November 2013. Chapter 5.3: Control rooms. Woods, D. D., Roth, E. M., Stubler, W. F., & Mumaw, R. J. (1990, October).
Navigating through large display networks in dynamic control applications. In Proceedings of the Human Factors Society Annual Meeting (Vol. 34, No. 4, pp. 396–399). Sage CA: Los Angeles, CA: SAGE Publications.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Software and Systems Engineering
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-16-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional