Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Metrics to Evaluate Multi-Stakeholder Decision-Making Processes – A Critical Discussion

Publication year
Digital Object Identifier10.54941/ahfe1003845

Abstract

Since the beginning of the century, airport management decision-making processes have been under scientific discussion. The introduction of Airport Collaborative Decision Making (A CDM) has set an operational standard which is a about to be succeeded by Total Airport Management and Performance Based Airport Management. Within the design and validation of these concepts and the necessary tools multiple assessments of the decision-making processes have been made. Although being under research for almost 20 years, the right selection of metrics to evaluate the decision-making processes remains still a challenge. Reflecting the different stakeholder objectives and the intricate dependent working processes into metrics and performance indicators is complex and was in some cases not sensitive to the introduced operational improvement.Summarizing the experiences of the past, this paper suggests a novel approach towards the evaluation of airport control centre decision-making processes. This approach assesses performance on bases of comprehensive indicators such as costs and decision time which are valid for all stakeholders. It allows the application of computational power to calculate reliable reference values as well as to identify optimization potentials. Moreover, this paper suggests to encounter objective metrics for human factors aspects as well as to consider additional communication and personality indicators. Last but not least recurrence analysis and cross-lagged panel designs are introduced to analyse effects over time and causal relationships between human factors and performance indicators.This novel approach to decision-making evaluation leads away from single performance indicator selection and assessment to a more comprehensive evaluation of the airport in connection with highly sophisticated communication pattern analysis.

BACKGROUND

Airport management started with relatively loosely coordination across air- port stakeholders (de Neufville and Odoni, 2002) and is still employed in many small and medium sized airports, suiting their own requirements. A new airport management paradigm was proposed (Hartmann, 2001),

© 2023. Published by AHFE Open Access. All rights reserved. 598

Metrics to Evaluate Multi-Stakeholder Decision-Making Processes 599

foreseeing a more hierarchical approach with rigid monitoring and same- level control management. Airport Collaborative Decision Making (A-CDM; EUROCONTROL, 2003, updated 2006 Ed. 3.0, EUROCONTROL, 2013) introduced a more integrated management cooperation, providing first steps in coordination across stakeholder boundaries. DLR in cooperation with EUROCONTROL pursued the first instalment of an operational concept for Total Airport Management (TAM; Günther et al., 2006) and foresaw a true collaborative decision-making approach between the relevant airport stakeh- olders. Corresponding support systems for the human stakeholder operators of the Airport Operations Center (APOC; Spies et al., 2008), which is the central concept element, needed to be designed, along with a suited proof of concept and evaluation environment based on human-in-the-loop real-time simulations (Suikat and Deutschmann, 2008). A high-level view on traffic flows in and out of the airport was considered to be sufficient to execute demand and capacity balancing to mitigate bottleneck situations. The total airport operations planner (TOP; Piekert and van Dongen, 2008) allows APOC stakeholders to negotiate between different levels of Key Performa- nce Indicator adherence. These are based on modelled operational decisions as an outcome of the supported collaborative decision-making process.

Since weather is the most influential factor for issues at airports the inclu- sion of meteorological information into TOP is at hand (Piekert and Feldhaus, 2009). The collaborative process between the stakeholder operators resul- ted in decisions on found balances between the performance parameters and available weather information, this was given to an optimization component (part of the TOP; Pick and Rawlik, 2011, Jipp et al., 2011) and subsequently put into effect through operator controlled planning support tools. The result was provided to operators again and the operational feasibility of the taken decision assessed.

Projects explored the different TAM concept elements, e.g. combined plan- ning instances covering pre-tactical and tactical, arrival, turnaround, landside and departure traffic management (Depenbrock et al., 2012) or took the rese- arch to European levels and introduced the Airport Performance Framework (Inard et al., 2011) and the Key Performance Indicator driven approach (KPI; Helm et al., 2015, Loth and Helm, 2015, Kosanke and Schultz, 2015). The importance of weather information as parameters was explained by Günther et al. (2015). But, harmonised and well-structured communication processes within an APOC were firstly considered by Papenfuß et al. (2015), Papenfuß and Biasotto (2017). A general approach to assess collaboration aspects in control rooms was taken by the project Collaborative Operations in Control Rooms (COCO; Schulze Kissing and Eißfeldt, 2015) but lacked the airport operations domain specifics.

Piekert et al. (2017) explained how these specifics can be addressed in the project PJ.04 TAM (SESAR Joint Undertaking, 2015) and consequently, how the European Operational Concept Validation Methodology (E-OCVM; EUROCONTROL, 2010) can be applied. Previous research identified that APOC decision making, multi-objective optimization and following a stru- ctured communication process as part of the APOC collaborative decision- making process provides positive gains (Jipp et al., 2011, Carstengerdes et al.,

2012, Papenfuß et al., 2017, Rousseau et al., 2019, Baleras et al., 2019). Dausch et al. (2019) point out challenges regarding the decision-making assessment, since used indicators are subjective by nature and their opera- tionalisation by direct and objective measurements is complex and further Freese (2019) showed the influence of human emotions on decision-making capabilities.

DEFICIENCIES AND RESEARCH CHALLENGES There are at least two dimensions to the problem of benefit assessment in APOC decision-making. Firstly, having a suitable set of metrics for opera- tional performance assessment with sufficient data to compute and soundly express these objectively. Further, assessment of the decision-making process and its impact on the outcome. While the Airport Operations Plan (AOP) con- tains data for the operational performance assessment, recording of data for the decision-making process only recognizes formal aspects like the overall impact and solution messages (Busink et al. (2019). The human-to-human interaction of the decision-making between the operators is not yet recorded, making any objective assessment approach difficult.

In addition, Jipp et al. (2012) pointed out that the isolated look at only single objective performance indicators (e.g. traffic flows, slot compliance, various punctualities and delay figures, passenger dwell times or ‘passengers missing their flight’ rates) without considering corresponding others offers potential for misinterpretation of results.

Taking these weaknesses into account, Carstengerdes et al. (2015) desi- gned a Human-in-the-Loop multi-scenario exercise with several validation objectives and assigned success criteria, but it was shown that increased situ- ation awareness, an enhanced decision-making process with higher efficiency and optimized overall airport performance not necessarily are achieved upon implementation of corresponding support tools under the research scenarios (Carstengerdes et al., 2016), due to the cumbersome and time-consuming procedures themselves. More importantly in this context are the questions ‘were adequate metrics for measurement available and have they been used?’ and analysis showed that suitable metrics in that context were not available.

The project PJ.04 TAM (Piekert et al., 2017) set out to revise the poin- ted out weaknesses and executed a set of independent validation exercises (Rahatoka et al., 2019a, Rahatoka et al., 2019b). The project addressed stakeholders’ needs within the Key Performance Area targets Efficiency, Pre- dictability, Punctuality, and Resilience and set out to provide evidence of transversal performance assessments, among those around human performa- nce. During the validation exercises, it was found that the results (Rousseau et al., 2019) could not support any conclusion about efficiency. However, there was clear indication that the solution improves resilience, and provides benefits for predictability and punctuality of airport operations. Regarding human performance, Piekert et al. (2019) presented first assessment results, providing evidence of an operator perceived better collaborative decision- making process when using the project tool suite and by following the updated SESAR collaborative decision-making procedures.

Metrics to Evaluate Multi-Stakeholder Decision-Making Processes 601

Research Challenges In this work the focus will remain on the challenge how to assess decision- making. The area around operational performance measurement is further explored by Suikat et al. (2023), where the what-if analysis function of APOC support systems is used to assess performance effects and considerations of stakeholder operational options prior plan implementation. It is obvious, though, that these two research directions have a relationship. The referred-to work helps to understand the individual decision bandwidth of stakeholders, which complements the overall group decision-making approach.

The way operators interact with available decision support systems influ- ences the decision-making process itself. Factors decide if the process can be efficient, e.g. how they evaluate the systems’ utilities, how difficult to use it is or if the systems are unreliable. These factors then introduce non-negligible bias in any subjective assessments. Subjective metrics had to be used to assess satisfaction, quality of support tools and usability of collaborative procedu- res since meaningful objectivized human performance metrics currently do not exist in this context.

For example, communication behaviour or frequency between operators can be analysed and counted. But what would the scenario-independent eva- luation or evaluation without operational performance correlation reveal? Is more communication good or is it bad? Does a higher communication frequ- ency (compared to some baseline) indicate a worse decision-making course or is it better since it possibly enhances the overall quality of the outcome – if it does – or is it just another indicator of a more complex operational pro- blem at hand? Additionally, how is efficiency of a structured communication and decision-making process defined? Taken that APOC decision-making will be under the umbrella of an Airport Performance Framework, recording of the decision-making process steps with all options and opinions might be required simply due to liability aspects between the contractual framework partners.

As hinted above, one assumption regarding the communication structures and intensity between APOC representatives is, that the patterns might vary from situation to situation that the airport is confronted with, based on the complexity of the operational problem. In order to assess the decision-making requires to normalise communication analysis results and to cluster different courses of decision-making. This again shows the necessity of an objective identification and classification of the underlying current and predicted traf- fic situation and which again shows the interdependency to operational performance aspects. Figure 1 shows a simplified control cycle approach.

Given the nature of diversity in human beings’ communication, it is close at hand that two stakeholder groups might communicate differently, but pos- sibly achieving similar outcomes in the end (decision-making success and agreement adherence assessment). This implies to identify the outcomes of the plan implementation by similar approaches than the traffic situation. Deci- sions in the APOC will be taken with an unspecified amount of lead time (possibly up to 24 hours in advance), the process of assessment and situation development tracking needs to be conducted repeatedly, until the predicted

Figure 1: Control and assessment cycle.
Original PDF Figure 1:

bottleneck situation occurs – only then the original resolution decision can be evaluated regarding its anticipated effectiveness. This leads to another resea- rch challenge, to distinguish the impact of the implemented APOC decisions and those occurring ‘naturally’, e.g. by time passing or network effects. The feasibility is yet unknown and not further explored in this work.

High level operational decision makers of airport processes are rarely involved in simulations, since they are deeply involved in the day-to-day ope- rations and availability of temporary personal is mostly not given. Taking an entire group from the same airport is even more difficult, but Suikat et al. (2023) managed to conduct a study with a group of experts from the same airport. Mixing experts and novice personal or students rarely delivers rea- listic results either and biases the course of decision-making procedures and outcomes. Higher numbers of groups need to be taken in order to balance and harmonise end results and minimise group effects. Since these experts are rare, low numbers of available participants pose another great challenge to have more groups.

DECISION MAKING ASSESSMENT This section discusses the need for a broader approach to human perfor- mance assessments. Questions will be addressed, such as what defines a good decision-making outcome or a good process or what is the subjective interpretation of objective measurable terms.

Need for Specific Hypotheses First of all, we need to have specific hypotheses how the APOC influences the decision-making process. Are we expecting a shorter process duration, less or more communication, better situation awareness? Is our aim to reduce confli- cts between involved stakeholders and to strive for higher satisfaction ratings regarding the decision-making process within an APOC? Are we expecting more pro-active instead of reactive behaviour? How are the operationali- sed decisions represented in metrics? Or does it all come down to airport performance metrics only?

Metrics to Evaluate Multi-Stakeholder Decision-Making Processes 603

Need for Objective Measurements for Human Performance Benefits If we agree that we also aim to improve human performance issues besides operational airport performance aspects, then we need to think about suita- ble ways to measure these benefits. One approach is subjective metrics via questionnaires, answers from workshops and debriefing sessions or behavi- oural observations. A plethora of well-established questionnaires is available to gather most information about human factors related issues like situa- tion awareness (individual and team), workload, trust, usability or teamwork (SHAPE questionnaires, Dehn, 2008, e.g SUS, Brooke, 1995). However, using only subjective measures poses a risk, ratings are biased by nature. The evaluation of situation assessment as defined by Endsley (1995) requires the perception and comprehension of the current situation and the proje- ction of its status into the future. But it is impossible to state information not knowing that it is missing. Just asking simple questions is insufficient and more elaborate measures are required. Kraemer and Süß (2015) intro- duce with SARA-T an online, objective, reliable and non-intrusive method for measuring situation awareness, using computer-generated situation- and person-specific questions based on the current situation. Recording reaction times provide an indication of mental workload. The questions can be embed- ded into discussions with virtual agents, which simulate relevant airport personnel, increasing the degree of realism and decreasing the intrusiveness (Dausch et al., 2019). This approach is novel as it provides an objective, reliable and fast real-time assessment of the most important human factors (situation awareness and mental workload). The analyses can already be used during debriefing sessions with participants to guide discussions and additionally rendering time-consuming and error-prone post-run analyses of questionnaires unnecessary.

Need for Communication and Interaction Metrics Communication and interaction metrics are proposed as objective additions, to assess decision-making processes in an APOC setting. Communication is a means of coordination between team members and builds a mutual under- standing of the situation. It can be considered as a relevant indicator for team processes, to explain differences in team performance. By assessing commu- nication data, it can be derived if team members followed the structure and whether performance changes are a result of the proposed guidelines. How- ever, standards for performance-relevant communication metrics are still missing (Papenfuß, 2019). Although quite obvious, communication metrics could encompass content, duration and frequency of communication units, sender and recipient and distribution between team members. Based on the derived communication data, recurrence patterns indicating adaptability and resilience can be identified (Schulze Kissing et al., 2018). These results can then be used by the airport decision-makers to foster compliance to defined communication procedures.

Need for a Sound Methodology In team settings there is a large variety of personal variables, e.g. team structure and personality traits (Biella et al., 2019, Papenfuß et al., 2017,

Papenfuß and Biasotto, 2017) as well as emotional states (Freese, 2019), that need to be taken into account. It will be impossible to control them by experimental design or randomization. Nevertheless, these effects have to be measured and statistically controlled for in order to draw valid conclusions about APOC benefits. Cross-lagged panel designs are suggested to analyse effects over time and causal relationships, like interdependencies between multiple influencing variables, human factors and performance metrics. By means of structural equation modelling and multiple regressions, analy- ses of relationships between these variables and inspections of directional causations are possible.

As an illustration, consider the two variables team communication and number of cancellations, which are measured at times t1 and t2 to obtain team communication 1 (at t1), team communication 2 (at t2), number of cancellations 1 (at t1) and number of cancellations 2 (at t2) (see Figure 2). Besides the correlation between team communication 1 and number of cancellations 1 (at t1) and team communication 2 and number of cancellati- ons 2 (at t2), it is possible to analyse how team communication 1 correlates with team communication 2 (the variable team communication over time) and number of cancellations 1 with number of cancellations 2 (the variable number of cancellations over time). But most interesting are the so-called cross-lagged correlations from team communication 1 to number of cancel- lations 2 and number of cancellations 1 to team communication 2. If these correlations are significantly different from zero, then you can conclude that causal effects exist between different variables over time. We then could analyse if team communication at t1 influences the number of cancellati- ons at t1 and more importantly, if there is an impact on cancellations at t2. This would provide a direct indication of the effects of earlier communication behaviour on later performance metrics and could prove benefits of APOC decision-making beyond subjective ratings.

Figure 2: Example for a cross-lagged panel design.
Original PDF Figure 2:

CONCLUSION

The evaluation and assessment of different airport management decision- making processes is a great challenge due to the fact that airport management consists of different stakeholders, performing a broad variety of tasks. More- over, existing indicators have a strong focus on performance metrics, while decision-making processes also aim on improving communication and col- laboration among the stakeholders. It is obvious that the transition from

Metrics to Evaluate Multi-Stakeholder Decision-Making Processes 605

a selection of single indicator assessments to a comprehensive approach taking the whole airport as well as human factors and performance indi- cators into account is necessary. Therefore, human factors and additional airport performance metrics are suggested.

  • Objective metrics: in addition to the well-known subjective metrics, objective metrics should be used to provide reliable indicators. Using SARA-T for objective workload and situation awareness measurement in combination with subjective ratings is one possibility to apply this approach.
  • Metrics to assess different communication styles, e.g. for personality and communication, should be used, refined and evaluated to consider the stakeholders’ possibilities and their success in implementing the deci- sion process. For instance, communication duration for each stakeholder can give an insight who participated in the discussion and who was not considered.
  • Methodological considerations: using recurrence analysis is a most pow- erful way to find communication patterns. The use of cross lagged panel designs is giving the opportunity to analyse causal relationships between communication metrics and performance indicators. Thereby, dependencies of indicators like communication duration and number of cancellations can be assessed.
  • Comprehensive metrics: instead of evaluating the outcome of the decision- making by multiple indicators for different processes (e.g. throughput for runway usage), a comprehensive metric which is relevant to all airport processes should be used, e.g. costs and decision time are suggested.
  • Reference: all metrics should be evaluated in comparison to a reference, only by which a reliable assertion on the success of the examined decision- making can be given. This reference can most likely be provided by a fast time simulation, calculating the airport processes without any influencing action.
  • Optimization: optimizing decisions regarding dependent indicators should be performed in a way that the priorities of the indicators are clarified and the power of computational algorithms is used to determine the optimum. For instance, the ratio of cancellations and delay in a certain operational situation can be derived by using advanced computer algorithms.

REFERENCES

Baleras, A., Busoni, S., Deborde, P., Hølvold, F., Grimm, A., Kalita, H., Robinson, T.

& Rousseau, F. 2019. SESAR Solution PJ.04–02: COST BENEFIT ANALYSIS (CBA) FOR V2. [Accessed 20200409]. Biella, M., Carstengerdes, N. & Papenfuß, A. Relevance of Human Factors for Disruption Management. Airport Operational Excellence Conference; IQPC, 2019 2019. Brooke, J. 1995. SUS: A quick and dirty usability scale. Usability Eval. Ind., 189. Busink, J., Derogee, E., Desart, B., Grimm, A., Grosmann, R., Günther, Y.,

Hølvold, F., Inard, A., Lapteva, L., Larsson, I., Marrazzo, G., Marsden, A., Miklos, E., Nechaj, P., Robinson, T., Sinen, M., Skoglund, D. & Verhoogt, T.

2019. Sesar Solution PJ. 04–02 SPR-INTEROP/OSED for V2 - Part I. [Accessed 20190826]. Carstengerdes, N., Jipp, M., Piekert, F., Reinholz, A. & Suikat, R. 2012. Validation of the Total Airport Management Suite - Research Report. Braunschweig, Germany: Deutsches Zentrum für Luft- und Raumfahrt E. V. (DLR), Institut für Flugführung. Carstengerdes, N., Papenfuss, A., Piekert, F., Rafidison, J.-P., Suikat, R., Welman,

C., Marsden, A., Bogers, H., Delain, O., Linde, M., Rodriguez Domenech, G. & Jimeno Cuenca, D. 2015. EXE-06.03.01-VP-757 Validation Plan. 0.1.1 ed. Brussels, Belgium: SESAR Consortium. Carstengerdes, N., Papenfuss, A., Suikat, R., Schier, S., Günther, Y., Piekert, F. & Marsden, A. 2016. EXE-06.03.01-VP-757 Validation Report. 0.1.1 ed. Brussels, Belgium: SESAR Consortium. Dausch, c., Schier, s., Carstengerdes, N. & Papenfuss, A. 2019. Talking with CLAUDI

- Using virtual agents for an immersive approach to measure workload and situa- tion awareness in human-in-the-loop simulations. Deutscher Luft- und Raumfahrt Kongress 2019. Darmstadt, Germany: Deutscher Luft- und Raumfahrt Kongress. De Neufville, R. & Odoni, A. 2002. Airport Systems: Planning, Design, and Management, McGraw-Hill Education. Dehn, D. M. 2008. Assessing the Impact of Automation on the Air Traffic Controller:

The SHAPE Questionnaires. Air Traffic Control Quarterly, 16, 127–146. Depenbrock, F., Urban, B., Temme, A., Günther, Y., Schaper, M. & Helm, S. 2012.

TAMS Operational Concept Document. Braunschweig, Germany: DLR. Endsley, M. 1995. Endsley, M. R.: Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors Journal 37(1), 32–64. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37, 32–64. Eurocontrol 2003. Airport CDM Operational Concept Document. 1.0 ed. Brussels,

Belgium: EUROCONTROL. Eurocontrol 2010. E-OCVM Version 3.0. In: Eurocontrol (ed.) European Operational Concept Validation Methodology. Brussels, Belgium: EUROCONTROL. EUROCONTROL 2013. Airport CDM Implementation. The Manual. Brussels,

Belgium: EUROCONTROL. Freese, M. 2019. Mensch frustiere dich nicht oder doch? Der Einfluss von Emotionen auf Ergebnisfaktoren von Team-Entscheidungen am Beispiel eines Planspieles für das Flughafenmanagement. Günther, Y., Inard, A., Werther, B., Bonnier, M., Spies, G., Marsden, A., Temme,

M., Böhme, D., Lane, R. & Niederstraßer, H. 2006. Total Airport Management - Operational Concept & Logical Architecture. Braunschweig, Germany: DLR. Günther, Y., Pick, A., Kern, S., Lorenz, S., Gerz, T., Keis, F. & Köhler, M. 2015.

Improved airport operations planning by using tailored forecasts of severe wea- ther. 5th International Air Transport and Operations Symposium (ATOS 2015). Delft, The Netherlands: TU Delft. Hartmann, H. 2001. Final Report of Preliminary Study Total Airport Management.

care Action “innovation”. 1.1 Ed. Braunschweig, Germany: DLR. Helm, S., Loth, S. & Schultz, M. 2015. ADVANCING TOTAL AIRPORT MANA- GEMENT – AN INTRODUCTION OF PERFORMANCE BASED MANAGE- MENT IN THE AIRPORT CONTEXT. The 19th ATRS World Conference. Singapore. Inard, A., Groskreutz, A., Depape, P., Burgess, m., Ianucci, S. A., Eveleigh, R.,

Mathewson, S., Alvarez, I. & Garcia Alonso, A. 2011. D11 - Airport Detailed Operational Description - STEP 3 (initial). SESAR WP 6.2. 00.01.00 ed. Brussels, Belgium: SJU SESAR Joint Undertaking. Jipp, M., Carstengerdes, N. & Papenfuss, A. 2012. TAMS Validation Concept Document. Braunschweig, Germany: DLR.

Metrics to Evaluate Multi-Stakeholder Decision-Making Processes 607

Jipp, M., Depenbrock, F., Suikat, R., Schaper, M., Papenfuß, A. & Kaltenhäuser, S.

2011. Validation of Multi-Objective Optimization for Total Airport Management. 8th Asian Control Conference. Kaohsiung, Taiwan: ASCC. Kosanke, L. & Schultz, M. 2015. Key Performance Indicators for Performance-Based Airport Management from the perspective of airport operations. 5th International Air Transport and Operations Symposium (ATOS 2015). Delft, The Netherlands: TU Delft. Kraemer, J. & Süß, H.-M. 2015. SARA-T: Validierung eines Echtzeit-Mess- und

Validierungstools für das Situationbewusstsein. Loth, S. & Helm, S. 2015. Punctuality as KPI for Performance Based Airport Management. AIAA Aviation 2015 Conference. Dallas, Texas, USA. Papenfuß, A. 2019. Identifizierung leistungsrelevanter Merkmale der Teamkommu-

nikation in der Domäne Flugsicherung. Doctoral Thesis, Technische Universität Berlin. Papenfuß, A. & Biasotto, A. 2017. Cooperation under Guidelines: How guidelines for cooperation affect interaction behavior of airport experts. 7th SESAR Innovation Days 2017. Belgrad, Serbien. Papenfuß, A., Carstengerdes, N. & Günther, Y. 2015. Konzept zur Kooperation in

Flughafen-Leitständen. 57. FACHAUSSCHUSSSITZUNG ANTHROPOTECH- NIK. Rostock, Germany: DGLR. Papenfuß, A., Carstengerdes, N., Schier, S. & Günther, Y. 2017. What to say when: Guidelines for Decision Making. An evaluation of a concept for cooperation in an APOC. Twelfth USA/Europe Air Traffic Management Research and Development Seminar (ATM2017). Seattle, USA. Pick, A. & Rawlik, T. 2011. Optimization of Airport Processes - Support system

for human decision making in total airport management. The 8th Asian Control Conference - ASCC 2011. Kaohsiung, Taiwan. Piekert, F., Carstengerdes, N. & Suikat, R. 2019. Dealing with Adverse Weather Conditions by Enhanced Collaborative Decision Making in a TAM APOC EIWAC 2019. Tokyo, Japan: ENRI. Piekert, F., Delain, O., Martin Dominguez, E. & Marsden, A. 2017. EUROPE’S

NEXT STEP IN TOTAL AIRPORT MANAGEMENT RESEARCH. ATRS 2017. Antwerp, Belgium: ATRS. Piekert, F. & Feldhaus, H. 2009. Environmental Issues and application of corresponding Models in the context of Total Airport Management. CEAS 2009 European Air and Space Conference, 2009–10-29 2009 Manchester, United Kingdom. Manchester, United Kingdom: Council of European Aerospace Societies, 30. Piekert, F. & Van Dongen, A. Total Airport Management Negotiation of Airport

Performance Parameters. 7th EUROCONTROL Experimental Center INO Work- shop, 2008–12-04 2008 Brétigny sur Orge, France. Brétigny sur Orge, France: EUROCONTROL, 7. Rahatoka, C., Busoni, S., Desart, B., Suikat, R., Marsden, A., Defourny, D., Miklos, E., Nechaj, P. & Piekert, F. 2019a. SESAR Solution PJ04-02: V2 Validation Plan (VALP) - Part I. Braunschweig, Germany: DLR. Rahatoka, C., Suikat, R. & Nechaj, P. 2019b. SESAR Solution 04-02: V2 Valida-

tion Plan (VALP) - Part IV - Human Performance Assessment Plan. Braunschweig, Germany: DLR. Rousseau, F., Desart, B., Marsden, A., Miklos, E., Lopez, C., Rahatoka, C., Salomon, R. & Suikat, R. 2019. SESAR Solution PJ04-02: Validation Report (VALR). [Accessed 20191010]. Schulze Kissing, D., Bruder, C., Carstengerdes, N. & Papenfuss, A. Making Multi-

team Systems More Adaptable by Enhancing Transactive Memory System Structu- res – The Case of CDM in APOC. In: AHRAM, T., KARWOWSKI, W. & TAIAR,

R., eds. 1st International Conference on Human Systems Engineering and Design (IHSED2018), 25.10.2018 2018 Reims Champagne-Ardenne, France. Springer International Publishing, 215–220. Schulze Kissing, D. & Eißfeldt, H. 2015. ConCent: Eine Simulationsplattform zur Untersuchung kollobarativer Entscheidungsprozesse in Leitzentralen. In: GRANDT, M. & SCHMERWITZ, S. (eds.) 57. Fachausschusssitzung Anthropo- technik: Kooperation und kooperative Systeme in der Fahrzeug- und Prozessfüh- rung. Rostock: Deutsche Gesellschaft für Luft- und Raumfahrt Lilienthal - Oberth e. V. SESAR JOINT UNDERTAKING. 2015. Total Airport Management [Online]. Brus-

sels, Belgium: European Commission. [Accessed 2017-05-02 2017]. Spies, G., Piekert, F., Marsden, A., Suikat, R., Meier, C. & Eriksen, P. 2008.

Operational Concept for an Airport Operations Center to enable Total Airport Management. In: GRANT, I., ed. 26th Congress of International Council of the Aeronautical Sciences (ICAS) 2008, 2008-09-14 2008 Alaska, USA. Alaska, USA. Suikat, R. & Deutschmann, A. 2008. Total Airport Simulation Supporting Research for Future Airport Management Concepts. AIAA Modeling and Simulation Tech- nologies Conference and Exhibit. Honolulu, Hawaii, USA: American Institute of Aeronautics and Astronautics. Suikat, R., Kjenstad, D., Sinen, M., Carstengerdes, N. & Günther, Y. 2023. Handling

disruptions in Total Airport Management using what-if enabled systems. AHFE. San Francisco, U. S. A.

Cover of the proceedings volume Human Factors in Transportation

Explore the complete digital volume

Human Factors in Transportation

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-71-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →