Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

The User-Oriented Integration of a Decision Support System for Tunnel Control – Challenges in Methodic and Design

Institute of Media Technology, Ilmenau University of Technology, Ilmenau, 98693, Germany ↗
Publication year
Digital Object Identifier10.54941/ahfe1001237

Abstract

Developing a decision support system with the goal of a seamless integration into different existing complex system environments is a very challenging task. Especially if these environments are characterized by a high heterogeneity, a high dynamic of the information to be processed by the working operator and the uniqueness of occurring event situation that requires a non-linear task solving processes. Supporting operators in this context means supporting their decisions by providing an appropriate information basis for the quick and easy assessment of risk and event situations. The aim of this paper is to describe the methodical and the design challenges that occur during the individual phases of the development of such a system for the specific context of tunnel control. The results are based on analytical and empirical studies conducted within a research project on integration of a real-time-security-management-system for tunnel control centers. In order to determine the requirements, a methodical mix of interviews, observation and cognitive walkthroughs was performed in 12 tunnel control centers in Germany, Austria and Luxembourg. During this, the used methods could be refined and adapted to the development of a decision support system for the present context.

INTRODUCTION

To enhance the safety for road tunnel users, German highway tunnels are monitored around the clock by operators in tunnel control centers. The operators exert a typical monitoring activity in a special context. Their main task is to ensure a defined normal state of the objects to be monitored. In the case of incidents they have to decide about appropriate measures to be taken in order to return to the normal state. “Good decisions” require a thorough understanding of the specific situation in a particular tunnel. The possibilities offered by modern sensor, computer and network technology allow the more and more precise assessment of the situation on location. However the associated increase of the data volume and the constantly increasing number of monitoring objects leads to a growing complexity of the used control systems. Especially in event situations where quick decisions are required, the operators are not adequately supported by their control systems. The information needed is scattered, to get an accurate assessment of the situation including all relevant conditions is a time consuming issue. To support operators in the management of risks and incidents, a real-time-security-management-system is developed within the German research project ESIMAS1. Initial studies have shown, that supporting operators means supporting their decisions. This can be ensured by providing an appropriate basis of information for the assessment of the situation.

1 ESIMAS – “Real time security management system” is a research project funded by Federal Ministry of Economics and Technology. For more information visit www.esimas.de

(Spundflasch and Krömker, 2014) Here the term situation awareness (Endsley, 2012) plays an important role, mainly because (Stanners and French, 2005) could prove that there is a direct relationship between Situation Awareness and the quality of decisions.

The aspiration of supporting decisions has spawned countless systems in diverse fields of application in recent decades. A nice overview can be found in (Arnott and Pervan, 2008). Numerous efforts have been conducted mainly in the areas of air traffic control, (e.g. Erzberger, 2004), nuclear power plants (e.g. RODOS, 2000), business management (Little, 1979) or industrial process control (e.g. Löwe and Dalijono, 2012). In all these complex environments, users need to make critical decisions based on a variety of information. The theoretical considerations regarding the support of decisions show essentially two different approaches, expert systems and decision support systems. Both concepts support the user in making decisions, but in fundamentally different ways. Expert systems try to simulate an expert with expert knowledge in a special field. Here the computer calculates decisions or recommendations based on specific data which is captured by sensors, stored in a database or entered by the user. This is in contrast to decision support systems, where the system shows relevant Information which forms an appropriate base for the decision done by the user. In (Ford, 1985) a detailed analysis of the two concepts can be found.

Looking at the context tunnel control room the core idea of a decision support system seems best suited for the needs of the operators. Usually they have a great experience in management of critical situations, high expert knowledge in operational procedures and they are accustomed in making their own decisions. What they need is an information system adapted to their tasks and goals in management of risks and incidents. This system has to support the quick and accurate assessment of occurring event situations by showing all relevant conditions. Thereby the operator receives a holistic view of the situation which is the prerequisite for decision making.

Figure 1. Decision support for the context of tunnel control
Original PDF Figure 1.

The particular challenge in this case was the development of generic system, giving the possibility of operating in different control centers. Furthermore the conditions of the surroundings between different control centers vary, what requires an accurate analysis of the context of use. The ultimately goal was the integration of the support system into the existing workflow of the operators and into the existing system landscape of different control centers.

During the different phases of the project important experiences could be gathered. In the following, the lessons learned in terms of methodical- and design challenges that come up during the different phases of the development process were shown. Instead of describing detailed project-related results, the aim is rather a description of the approach and the challenges that occur.

Hereinafter an overview of the basic steps in the development is shown. In order to demarcate the scope of the findings a detailed description of the context of use is given secondly. An attempt is made to describe the context on an abstract level so that it is possible to project the later outcome to efforts in areas with similar context. Then, challenges in each development phases are described. These are methodical challenges within the requirements analysis and the phase of evaluation as well as design challenges in the phase of conception and prototyping.

THE DEVELOPMENT PROCESS Numerous scientific publications provide different approaches, models and frameworks for the design of decision support systems. However, in detail it is difficult to operationalize these models in a specific context. (Gachet and Sprague, 2005) argue that the solutions offered remain too distinct and project-specific and above all the context of the target systems is not adequately considered. Important for the development of a decision support system is the possibility of iterations in all phases of the development, an approach that allows rapid feedback from the end users and a quick and easy implementation of changes. (Sprague, 1980) In addition the context plays an important role and should be analyzed very carefully. (Gachet and Sprague, 2005) Taking this into account, a process according to the “human-centered design process for interactive systems” (9241-210, 2010) was chosen, because it´s focus is on the context of use and it allows iterations.

Figure 2. Development process of the decision support system. (Adapted from 9241-210, 2010)

Figure available in the original PDF paper.

As part of the requirement analysis 12 tunnel control centers in Germany, Austria and Luxembourg were investigated. The primary goal was the detailed understanding of the actual situation concerning the interplay of the operators, their task and goals and their use of the control room systems in order to accomplish the tasks and reach the goals. As part of the context analysis a goal directed task analysis (Endsley and Jones, 2012) was conducted, On the basis of analysis results, user requirements for the design of the system and requirements for the integration in different control centers were defined. These formed the core of the subsequent conception phase, where the functionality to provide by the system was defined. Using the concept as a fundament, different design variants were created in the next step. Afterwards these prototypes were tested and optimized iteratively with different operators in different control centers.

CONTEXT OF USE - TUNNEL CONTROL Understanding the context of use is a prerequisite for the system development, especially if the aim is to integrate an additional system to existing, already complex system environments for highly specialized purposes.

To define the actual situation we analyzed the specifics of users, tasks and the existing control systems. According to the results we determine how the decision support system must be integrated to support the process of problem solving and what functionality it must provide for. In addition the usability of the system plays an important role for user acceptance.

Figure 3. Context of used characterized through the interplay of operators, their tasks and the used control systems (Adapted from Frese and Brodbeck,
Original PDF Figure 3.

)

To describe the field of activity, the table below gives a survey of the characteristics typical for the current situation across all analyzed control centers.

Table 1: Characteristics of the context tunnel control center
Original PDF Table 1:

The specified context of use here serves as a kind of filter used to mark the area the methods and challenges described in the following are validity for. The description of the context on this abstract level shows a typical monitoring context. The following considerations should therefore be applicable to related areas.

CHALLENGES IN THE DEVELOPMENT PROCESS The careful use of goal- and context-adapted methods plays an important in the overall development process. This section describes the approach, the methods used and the challenges encountered during the various development phases.

Requirement Analysis The operators, although they are experts in their field, are usually not aware of what they actually need. In addition there is often a difference in what they think they need and what they really need to complete their tasks effectively, efficiently and adequate.

As part of the requirement analysis typically interviews and observation were used on the target group. However, it is precisely the mix of methods, which leads to a better understanding of the user. The goal is to understand the complex interplay of operators, their task and goals and the use of the control system for task completion. A holistic and comprehensive understanding of the current situation is important to determine how the system being developed should support this interplay.

Goal The investigations in the control centers pursued two main goals:

1. Analysis of the context of use in order to fit the system to the existing context

2. Goal-directed Task Analysis (Endsley and Jones, 2012) to structure the tasks which should be supported

  • Analysis of the tasks / action phases in the management of events and risks
  • Analysis of the goals of action at the various stages
  • Extracting the information needs to achieve the goals of action It should be noted that “Tasks are not the same as goals”. (Endsley and Jones, 2012, p. 68) The investigations should not only aim at analyzing the tasks and their execution using the existing control system. Important are the goals of the performed tasks because the later system should support the achievement of the goals and that may happen in a different way than previously done using the actual system.

Procedure and metodical Challenges

1. Interview with control center manager

In each visited control center an interview was conducted with the respective control center manager. The primary goal was to characterize the organizational, technical and physical environment (9241-210, 2010) that may influence the DSS integration. These are for example: number and characteristics of employees, number and size of tunnels to be monitored, technical equipment of the tunnels and the control room as well as typical everyday situations. Each control center also lives an individual philosophy, which is reflected also in the monitoring activity or in dealing with certain events. The challenge here is to recognize elements that are relevant for the DSS-design.

2. Interviews and Observation as iterative process

If it is possible, an appropriate way of analyzing the operator activity is by using a mix of observation and interview directly at the workplace. Here the interplay of operators, their tasks and the systems used can be particularly well observed and scrutinized. Because of the diverse activities of the operators with numerous ancillary tasks in a complex environment, it is advantageous if the observer has a clear understanding of the knowledge interest in mind. In our case, these were the relevant tasks and objectives in traffic management, especially in risk and incident situations. The subsequent determination of the functionality to be provided by the system can be done more purposeful this way. But this clear view should in no way mean that you are blind to anything else. The importance of some aspects is sometimes understood only in retrospect.

The operators have a high proportion of tacit knowledge which they are not aware of. Analyzing this knowledge by simple questions in interviews can be difficult and may lead to inaccurate results. The reflection of routine tasks is not easy for the operator and accordingly there is usually a difference between what they think they need and what they really use in certain situations. To extract this tacit knowledge, the combination of interview and observation has proved, however, but it should be deepened using the following methods in further steps.

3. Scenario-Based Walkthrough using thinking aloud method

Because events do not occur as frequently and tacit knowledge plays an important role especially in these situations, a scenario-based walkthrough using typical event situations was performed with the operators. The operators were asked about their goals in these situations and what they need to know in order to manage them. This procedure is ideally performed directly at their workstation, because this apparently increases their ability of reflection. In summary it can be said that it is precisely the use of stimulus material like the used scenarios which increases the ability of reflection rather than using just the interview and observation method.

4. Focus Groups

With focus on the management of risk and event situations it became clear, that the tasks and goals of the operators in different control rooms are the same on an abstract level, but they vary in detail. This can be seen in different strategies in the task- or problem solving process. This heavily depends on the knowledge and experience of the individual operator and the different philosophies of the various control centers. Accordingly there is no uniform information requirement by the operators.

In order to approximate these different perspectives focus groups as part of a workshop were conducted with operators and managers of different control centers. Split up into groups, the participants were asked to design a support system according to their needs. As a working material they got a paper screen and typical items of information that are relevant for the situation assessment. These items could be derived from the analysis results. Using predefined scenarios, the participants should arrange the needed information on the paper screen. In addition, they were asked to complete missing information elements. Thus, the participants were able to reflect their individual perspectives in the discussion. At the end, a common understanding was pictured on the paper screen.

Figure 4. The approximation of different perspectives by conducting focus groups
Original PDF Figure 4.
Table 2: Challenges and methods in the requirement analysis Phase Challenges Methods
Original PDF Table 2:

Conception Goal: Determination of the functionality of the system

The difficulty in this phase is to interpret the results of the investigations correctly in order to derive user requirements as a base for system development. Using the results of the context of use analysis and the goal-directed task analysis, requirements for the systems functionality and the integration of the support system into different control centers have been derived.

Challenges Defining the role of the system

In our case a system should arise which provides information to the user serving as a basis for decision making. Therefore it aims at supporting the operators in understanding detected events plus all relevant conditions of the situation. The system should not only support predefined task it should also work in unexpected situations. It should operate on a meta-level, displaying the most important information for situation assessment on the one hand, drawing the attention of the user to important aspects that can be deepen by using the existing system on the other hand.

Determine the systems functionality In a first step the relevant task, isolated using the goal directed task analysis, were compared and transferred into recurrent action patterns. These patterns were associated with the analyzed goals of the operators. Described on an abstract level in our case these goals were:

o Recognizing and understanding of notifications triggered by automatic detections o Understanding of the event situation and its impact on the immediate environment

o Recognizing and understanding of any other conditions of the tunnel affecting the current situation These high-level objectives were used to determine the basic functional areas of the system. Finally the analyzed heterogeneous information need used by the different operators to achieve these goals was assigned to the defined functional areas.

Table 3: Design challenges in the conception phase
Original PDF Table 3:

Prototyping Goal: Preparation of different prototype variants based on the concept

Challenges In the prototyping phase, based on the concept and in compliance with various established visualization principles, various prototype variants were created. Although it is a creative process to a large extent it is recommended to include key findings from the literature. As a result, some sources of errors are excluded already in this phase, which can shorten the needed time for the evaluation. In addition to the usual standards for control rooms the following findings were helpful:

o Errors and error sources, influencing the usability ((Prümper, 1994), (Endsley and Jones, 2012)) o Situation awareness-oriented design principles (Endsley and Jones, 2012)

Table 4: Design challenges in the prototyping phase
Original PDF Table 4:

Evaluation Goals

  • Evaluation of the adaptability for heterogeneous system environments
  • Evaluation of operators acceptance Procedure and methodical challenges

Sensitization for the benefits of the new system The biggest challenge for the evaluation process is to simulate the later conditions for the use of the system. This is really important, especially if the added value of the new system is to support the interplay of operators and their systems in order to perform their tasks in a more efficient way. However, the specifics of safety-critical system environments often lead to the problem that newly developed systems need to be tested in a “stand-alone” manner. This is not easy for the test subjects on the one hand because the test takes place on a fairly abstract level and so they had to image a lot of the later given functionality especially with regard to the foreseen interplay of the systems. On the other hand it is hard for the designers to project the results to the later conditions of use. The results are highly dependent of the open-mindedness and the capacity for abstraction of the individual operator. The key is a precise introduction regarding the role of the system and the given functionality. It is highly recommended to give it some time, because a clear understanding of what the system can do for the operators makes it much more easier for them to commit themselves to it and to evaluate the system in a more targeted way. According to our experience paper prototypes and predefined scenarios should be used for this. The immediate use of clickable GUI-prototypes drew the attention of the operators to the interaction with the system and one had the feeling that they don´t really listen anymore. They were caught in details sometimes not purposeful for the aim of the evaluation.

Discussion of the functionality Following the explanation the basic functional areas and the different design solutions were discussed with the operators. As stimulus material furthermore paper prototypes and scenarios were used. During the discussion, it became clear how much the routines for handling the existing control systems influence the behavior and the view of the operators. They always draw comparisons and linking points to the used control systems. If the new system goes a slightly different route to achieve the goals it seems hard for some operators to move away from the routines.

Another challenge is the fact that statements given during the test sometimes are contradictory to statements given in the requirement analysis. In addition the reviews of various operators are often very contrary. This is not surprising due to the fact that different strategies in problem-solving could be observed during the investigations. Though it makes the optimization of the prototypes sometimes complicated. From the second iteration it has proven to talk to two operators at the same time. Then they already know about the aim of the system and in some questions they relativize their statements each other in the discussion.

Discussion of the interaction While the previous steps focus on the evaluation of the functionality of the system, this step should evaluate the usability of the system. The operators were faced with clickable-prototypes implemented with HTML 5. Using the thinking aloud method, their task was to assess different risk and incident situations on the base of the information shown by the system. Finally they should decide about an appropriate intervention strategy for the individual situation. Furthermore they were asked to describe which information influenced their decision and which elements they missed. Thereby could be observed that the prototype were used in a different way as intended and explained before. Especially areas with colors similar to those used in the control systems draw the attention of the operators in a sometimes not intended manner.

Table 5: Challenges and methods used in the evaluation phase Phase Challenges Used methods
Original PDF Table 5:

CONCLUSION

The development of a decision support system for an already existing complex system environment requires a detailed understanding of the context of use. In particular if this system must be fit into the heterogeneity context of different control centers like in the present case, a broad and in-depth requirement analysis is important to get a comprehensive and holistic understanding of the different conditions. Here the understanding of the interplay between users, their task and goals and the used control system is very important. To reach the operator acceptance of the decision support system the seamless integration into the existing structures and routines is necessary. One of the biggest challenges in the development was the extraction of the tacit knowledge of the working operators. Therefore a method mix of interviews, observation, scenario-based walkthrough and focus groups has proven. Especially the use of stimulus material in form of everyday scenarios increases the operator ability of reflection. To handle the heterogeneity of the results in terms of different strategies for goal achievement and different information needs of different operators, the performing of focus group turn out to be an appropriate method to bring the various views to a common denominator. During the evaluation it became obvious that the operators are not necessarily open to additional systems. It is therefore particularly important to explain the aim of the system and the offered functionality very precisely in order to make them aware of the benefits they will have by using the decision support system. This detailed understanding is the prerequisite for the operator in order to evaluate the usability of the system. Therefore clickable prototype should be used only at a later time.

REFERENCES

Arnott, D., Pervan, G. (2008), Eight key issues for the decision support systems discipline, Decision Support Systems, 44 (3), pp.

657-672. Endsley, M. R., Jones, D. G. (2012), Designing for Situation Awareness: An Approach to User-Centered Design (2nd ed.).

London: Taylor & Francis. Erzberger, H. (2004), Transforming the NAS: The Next Generation Air Traffic Control System. NASA Technical Reports Server (NTRS) Website: http://www.aviationsystems.arc.nasa.gov/publications/tactical/NASA-TP-2004-212828.pdf. Ford, N. (1985), Decision Support Systems and Expert Systems: A Comparison. Information & Management Volume 8, Issue 1,

pp. 21 – 26. Frese, M., Brodbeck, F.C. (1989), Computer in Büro und Verwaltung. Psychologisches Wissen für die Praxis. Berlin, Heidelberg, NY: Springer-Verlag. Gachet, A., Sprague, R. (2005), A Context-Based Approach to the Development of Decision Support Systems. Proceedings of

the Workshop on Context Modeling and Decision Support, Paris, 2005. Little, J.D.C. (1979), Decision Support Systems for Marketing Managers, Journal of Marketing, 43 (3), pp. 9-26. Löwe, K., Dalijono, T. (2012), Entwicklung eines Operatorunterstützungssystems zur Steigerung der Sicherheit hochautomatisierter verfahrenstechnischer Anlagen, Chemie Ingenieur Technik, 84 (11), pp. 2027-2034. doi: 10.1002/cite.201200085 Prümper, J. (1994), Fehlerbeurteilung in der Mensch-Computer Interaktion. Reliabilitätsanalysen und Training einer

handlungstheoretischen Fehlertaxonomie. Internationale Hochschulschriften. Münster: Waxmann Verlag GmbH. RODOS – Forschungszentrum Karlsruhe GmbH. (2000), Decision support for nuclear emergency management [Brochure].

Karlsruhe: Raskob. W. Sprague, R.H. (1980), A Framework for the Development of Decision Support Systems, MIS Quarterly, 4 (4), pp. 1-26. Spundflasch, S., Krömker, H. (2014): Operators in tunnel control – analyzing decisions, designing decision support. International Conference on Ergonomics & Human Factors, Southampton, 2014. (accepted paper) Stanners, M., French, H. (2005), An Empirical Study of the Relationship Between Situation Awareness and Decision Making.

Defence Science and Technology Organisation Website: http://www.dsto.defence.gov.au/publications/scientific_record.php?prefix=1947&record=4318. 9241 – 210, DIN EN I.: Ergonomics of human-system interaction, part 210: Human-centred design for interactive systems. (ISO 9241-210:2010); German version EN ISO 9241-210:2010.

Cover of the proceedings volume Advances in Ergonomics In Design, Usability & Special Populations: Part II

Explore the complete digital volume

Advances in Ergonomics In Design, Usability & Special Populations: Part II

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-4951-2107-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →