Okan Yilmaz, Klaus Radermacher, Frank Beger, Jonas Roth, Armin Janß (2023). Usability evaluation of a process optimized integrated workstation based on the IEEE 11073 SDC standard. In: Healthcare and Medical Devices. AHFE Open Access, vol. 79, pp. 133–142. https://doi.org/10.54941/ahfe1003481
Several studies have identified that the Operating Room (OR) is costly to maintain, but also the most profit generating department. The 2019 approved ISO IEEE 11073 SDC family defines a manufacturer independent communication standard, which creates open interoperability in the OR and clinic for the first time. Using SDC, medical device data can be collected and used for display on a central cockpit, documentation purposes during an ongoing operation and also be streamed outside the OR. The goal of this research was to analyze and evaluate how medical device data and context information (current workflow step, location, patient, operator) could be used inside and outside the OR (e.g., maintenance and reparation) to improve clinical processes and benefit patient care. We also analyzed if the availability of such information might affect the efficiency and safety of patient care. A process analysis has been performed with clinical staff from the Uniklinik RWTH Aachen. The planning procedure of the OR management has been discussed and clinical use cases have been discussed with nurses, surgeons and anesthesiologists. Potential improvements using interoperable device data have been developed and corresponding interactive functional models have been implemented and discussed with the user groups. After several iterations, these concepts were implemented as part of surgical, anesthesia and OR management workstations. A formative usability evaluation (using the Thinking Aloud technique and questionnaires) has been carried out with a small user group (n=9) to check whether interoperable device data and context information improved the usability and safety of the clinical processes. Inside the demonstrator OR at the Chair of Medical Engineering (RWTH Aachen University) surgeons and anesthesiologists tested and evaluated different interfaces for OR procedures. During the formative evaluation, interfaces inside a remote SDC workstation were tested for: OR-Light, OR-Table, Video-Switch, High Frequency Cutting device and a navigation interface in combination with a standalone tracking system and a universal foot switch. The results were very promising and showed that most interfaces had a high degree of usability. Potential for improvement has been identified in the handling, comprehensibility and discernibility of the navigation system. In a second part, the developed processes for OR-optimization were discussed and evaluated. The clinical staff (n=9) reported that they need to gather information from various sources and systems during surgery (Ø 4.63 out of 5 on a Likert scale) and that repeated recordings of redundant information into multiple protocols is also required. They supported the access, availability and integration of device context information inside clinical processes. All clinical users agreed that utilizing automatic documentation through interoperable medical devices can save valuable time (Ø 4,85 of 5). A concept, in which workflow step specific device settings were suggested, was met with approval but also with criticism. The users saw benefits for time saving and standardization purposes, but also drawbacks for possible dependency and less independent thinking. The statement “The collected data helps to build up a database, on which OR-management can make more efficient decisions” was also met with approval (Ø 4,88 of 5). Interoperable medical devices (using ISO IEE 11073 SDC) can be used to create a useful data base and to support time and resource savings and helps to conduct efficient decisions inside and outside the OR. User interfaces were displayed and evaluated in combination with process supporting data. Workflow specific suggestions for device property changes were declared as helpful, although concerns have been expressed (see above). We showed that interoperable SDC medical device data and context information can be used to improve and support clinical processes.
Operating rooms are critical and complex environments. It is essential that they are optimized continuously to improve patient outcome, reduce human errors and increase the overall effectiveness and efficiency. Since surgeries are the biggest cost-factor in a hospital, workflow and resource optimizations could create great benefit.
With increasing complexity, optimizing clinical processes and interopera- bility have become major aims. The 2019 approved ISO IEEE 11073 SDC standard enables manufacturer-independent medical device interconnectivity inside the OR. Using the SDC standard, medical device data can be exch- anged through the OR network. This paper aims to explore, to analyse and to evaluates firstly, how process optimization can be achieved in a fully open-networked OR and secondly, how remote device interfaces can be used inside a cockpit to remotely control SDC compatible devices.
In the first chapter, we review current literature on operating room opti- mization and interoperability, as well as processes how to apply usability engineering to medical devices. In the second chapter, surgical medical device panels are presented and how process optimization is applied inside a work- station. The third chapter contains the formative evaluation of the developed system from nine clinicians from Uniklinik RWTH Aachen.
State of the Art Operating Room Optimization Enhancing efficiency in the OR can be accomplished through improving scheduling, better utilization, more effective information exchange, multi- disciplinary awareness training and personal accountability. Time saving,
Usability Evaluation of a Process Optimized Integrated Workstation 135
flexible OR staffing and information transfer were found to be the most cost- effective solutions to improve efficiency (Overdyk et al., 1998; Viapiano & Ward, 2000).
Interoperable Operating Room An early definition of interoperability is the “ability of two or more sof- tware components to cooperate despite differences in language, interface, and execution platform” (Wegner, 1996). That means, that an interoperable operating room is an environment, in which data can be exchanged betw- een different medical devices and IT-systems. Several manufacturers offer monolithic integrated solutions (e.g., iSuite - Stryker, OR1 - Karl Storz, Tegris - Maquet), where the interoperability to other external systems is very limited.
The German research project OR.NET (2012-2016) developed solutions to establish manufacturer-independent open device communication in the OR and clinic. 2019 the ISO IEEE 11073 SDC Standard family was approved and defines the communication semantic and syntax, responsibilities and speci- fic necessary device functionalities to allow open interoperability (Kasparick et al., 2018).
Other than SDC, projects like “Medical Device Plug and Play” MD PnP (USA) or the Smart Cyber Operating Theater SCOT (Japan) offer tools to implement an open manufacturer independent communication protocol (Arney et al., 2018; Okamoto et al., 2018). SDC still has advantages over these solutions: it is more flexible and the language is explicitly created to describe medical devices and their data. In addition, since 2019 it has several ISO and IEEE standards, which SCOT and MD PnP do not have. Therefore, SDC has a clear lead (Kasparick, 2020).
Application of Usability Engineering to Medical Devices The IEC 62366–1 is an international standard and describes how the usability engineering process on medical devices has to be applied. It offers guida- nce for medical device manufacturer on how to design and develop medical devices that are safe and useable regarding their intended use.
Requirement Analysis During the requirement analysis, the needs of the clinical users of Uniklinik RWTH Aachen were identified. Based on the requirements and specifications, innovative and user centered concepts were developed that meet the identified needs. The developed concepts will aim to prioritize the usability, safety and efficiency of all stakeholders.
The information used during the analysis was gathered from discussi- ons with clinical users, guidelines, standard operating procedures, trea- tment plans, clinical processes, clinical protocols, the hospital’s currently used system and tasks that need to be performed during surgery. The structural, organizational and medical requirements were also taken into account.
Workstations, remote device-, voice-, touch- and gesture-control methods were analysed regarding their risks and possible applications. To characterize
the system’s target user, user profiles were generated for the surgical team members (e.g., surgeon, physician, anesthesiologist). After conducting the requirement analysis, several pain points have been identified and considered when developing concepts: (Beger et al., 2022, 2021).
Increase interdisciplinary collaboration and coordination
Reduce number of phone-calls OR management has to do (up to 2000 phone calls a day)
Determining the status of an ongoing operation from outside the OR
Creating a database for OR management to make more efficient and information-based decisions (e.g., coordination of personnel)
Assistance for organisational and administrative work
Forecasts for OR duration
Intelligent context sensitive alarming
Workflow step specific support
Automatic transfer of data into protocols.
Integration of Devices Into an Open Interconnected SDC Workstation In a previous work, a SDC surgical workstation was developed to remotely display and control medical device properties in a safe and useable manner. (Yilmaz et al., 2020) In contrast to the previous work, the novelty of the presented work lies in the additional information provided to users. The deve- loped and integrated concepts have the ability to improve the efficiency even further.
The user interfaces were now also revised using User Interface Profiles. The User Interface Profile is a model based approach, which outlines design requirements and rules for HMI, taking into account usage risks, use pro- cess, and medical device functions, as well as input and output devices in an open networked system (Yilmaz et al., 2022). Medical device panels were integrated for an OR-Table, an OR-Light, a High Frequency Cutting device and a navigation (Localite GmbH) and tracking system (see Figure 1). Those developed user interfaces are now part of a surgical and OR management workstation.
Development of Process Support Systems The developed concepts contribute to increase efficiency by optimizing pro- cesses for surgery, anesthesia, OR planning and sterile supply management. In order to meet the previously specified requirements, process supporting systems were developed (see Figure 2) and integrated. The availability of addi- tional data (e.g., guidance and patient data), one system to look everything up (e.g., patient positioning, medication), one integrated system to remotely control every medical device, process step specific help and semi-automatic documentation were implemented into the following concepts and should increase the efficiency, safety and usability:
Usability Evaluation of a Process Optimized Integrated Workstation 137
Figure 1: Integrated user interfaces in a surgical workstation (top: or light, bottom: navigation system localite).
Documentation support while inducting general anesthesia and generation of protocols
Integration of WHO Surgical Safety Checklist (World Health Organiza- tion, 2023)
Guidance for instrument setup and OR preparation (integration of opera- tion specific datasets by IT4Process enterprise)
Guidance for patient positioning on OR Table
Guidance for instrument disposal after an operation
Documentation of sterile supply usage during an operation
Display of current workflow step
Smartphone assistance for nurses.
Usability Tests The evaluation of this setup contains on one hand an interaction centered usability test of the touch- and foot switch-based interfaces according to DIN
Figure 2: Process supporting interfaces (top: documentation support, bottom left: guidance for instrument setup, bottom right: WHO checklist).
EN 62366–1 (Application of usability engineering to medical devices) and on the other hand an evaluation of the implemented process optimizing methods.
All tests have been performed inside the usability lab at the Chair of Medical Engineering at RWTH Aachen University (see Figure 3). A represen- tative user group consisting of five neurosurgeons, three anaesthesiologists and one intensive care staff performed the usability tests during 26th-27th September 2022. The age of the participants was between 29 and 57 and have on average 12 years of experience in their current position. An anesthe- sia workstation was also part of this setup and has been evaluated, but is not part of this publication.
The evaluated aspects have been split into three parts: First the surgi- cal workstation, followed by the anesthesia workstation and finally the process-integrated workstation. For every task, participants got a standar- dized introduction, conducted the tasks and had to fill out a questionnaire afterwards.
Different usability evaluation methods have been applied here, which con- tained: Thinking aloud, user observation, discussion with users (after task was finished), questionnaires and measurement of effectiveness.
During the tests, the SDC Workstation was displayed and used on a 42-inch touchscreen monitor. The evaluation had been split into the following interfaces:
Usability Evaluation of a Process Optimized Integrated Workstation 139
Figure 3: Usability lab of the chair of medical engineering RWTH Aachen University.
Workstation in combination with a tracking camera, navigation system (TMS Navigator, Localite), universal foot switch (steute Technologies, mediTEC)
OR-Table interface on central workstation
Video Switch interface on central workstation
HF-Device interface on central workstation
OR-Light interface on central workstation
Smartphone assistance as a click-dummy
Documentation support and assistance while inducting general anesthesia and generation of protocols
Sign In Process
WHO Surgical Safety Checklist
OR site and patient preparation
Workflow Support Assistance
Patient positioning (e.g., Trendelenburg)
General questions regarding process support and automation.
RESULTS
The result of the questionnaire of the formative usability evaluation are displayed below in Figure 4, Figure 5 and Figure 6. The participants evalu- ated the concepts using a 5-Point Likert Scale. Figures 4, 5 and 6 show the results of the evaluation for the OR-Table, High Frequency Cutting Device and the navigation interface in combination with a tracking system and a universal foot switch.
Figure 4: Evaluation of operating room table.Figure 5: Evaluation of high frequency cutting device.Figure 6: Evaluation of navigation interface in combination with a tracking system and a universal foot switch.
Usability Evaluation of a Process Optimized Integrated Workstation 141
Figure 7: Evaluation of process optimization.Figure 8: Evaluation of process-step-specific device property adjustment.
The evaluation of process and workflow supporting features are displayed in Figure 7 and Figure 8. The effectiveness and user satisfaction were high during all tasks. Potential for improvement was identified in the area of the dialog control within the navigation system. The participants confirmed that the integration of cross-device data can support their work.
CONCLUSION
The presented work described how workflow optimization has been achie- ved through integrated medical devices (on the basis of SDC) in the OR and clinic. A requirement analysis was performed, followed by the development of concepts and an evaluation of the system through clinical users.
ACKNOWLEDGMENT
This research work has been funded within the project “PriMed - Workflow optimization by integrated medical devices in the OR and clinic” (State of North-Rhine Westphalia and the European Commission - European Regional Development Fund; grant no.: EFRE-0801369).
REFERENCES
Arney, D., Plourde, J., & Goldman, J. M. (2018). OpenICE medical device interoperability platform overview and requirement analysis. Biomedizinische Technik. Biomedical engineering, 63, 39–47. Beger, F., Janß, A., & Yilmaz, O. (2021). Der OP-Saal zwischen Effizient und Chaos.
meditronic-journal, 12–14. Beger, F., Janß, A., & Yilmaz, O. (2022). Prozessoptimierung im offen vernetzten Krankenhaus der Zukunft. meditronic-journal, 132–135.
Kasparick, M. (2020). Zuverlässige und herstellerübergreifende Medizingeräteinteroperabilität&dBeiträge zur IEEE 11073 SDC-Normenfamilie: Universität Rostock. Kasparick, M., Schmitz, M., Andersen, B., Rockstroh, M., Franke, S., Schlichting, S.,
Golatowski, F., & Timmermann, D. (2018). OR. NET: A service-oriented arch- itecture for safe and dynamic medical device interoperability. Biomedizinische Technik. Biomedical engineering, 63, 11–30. Okamoto, J., Masamune, K., Iseki, H., & Muragaki, Y. (2018). Development concepts of a Smart Cyber Operating Theater (SCOT) using ORiN technology. Biomedizinische Technik. Biomedical engineering, 63, 31–37. Overdyk, F. J., Harvey, S. C., Fishman, R. L., & Shippey, F. (1998). Successful
Strategies for Improving Operating Room Efficiency at Academic Institutions. Anesthesia & Analgesia, 86, 896–906. Viapiano, J., & Ward, D. S. (2000). Operating room utilization: the need for data.
International anesthesiology clinics, 38, 127–140. Wegner, P. (1996). Interoperability. ACM Computing Surveys, 28, 285–287. World Health Organization (2023). Surgical Safety Checklist: Patient Safety. Yilmaz, O., Janß, A., & Radermacher, K. (2022). Applying User Interface Profiles to Ensure Safe Remote Control within the Open Networked Operating Room in accordance with ISO IEEE 11073 SDC. In Healthcare and Medical Devices: AHFE International. Yilmaz, O., Wieschebrock, D., Heibeyn, J., Radermacher, K., & Janß, A. (2020).
Development and Evaluation of a Platform-Independent Surgical Workstation for an Open Networked Operating Theatre Using the IEEE 11073 SDC Communica- tion Standard. In V. G. Duffy (Ed.), Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. Posture, Motion and Health (pp. 79–92). Cham: Springer International Publishing.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Healthcare and Medical Devices
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-55-1
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional