Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Designing by Degree: Levels of Automation and Team Design Patterns as a Shared Language

TNO (Dutch Organization for Applied Scientific Research), The Netherlands ↗
Publication year
Digital Object Identifier10.54941/ahfe1008221

Abstract

Rapid advances in Information Technology (IT) and Artificial Intelligence (AI) have resulted in increasingly complex socio‑technical systems, placing new demands on human–automation collaboration. These demands are particularly acute in VUCA (Volatile, Uncertain, Complex, Ambiguous) environments, where failures can have immediate and severe consequences. Effective human‑automation teams must therefore adapt task and control allocation dynamically, while maintaining safety, accountability, and operator understanding. Adaptive automation (AA) has been widely studied as a means to support such flexibility, often described using Levels of Automation (LoA) frameworks. However, many existing LoA frameworks do not align well with decision‑cycle models commonly used to reason about responsibility distribution in VUCA contexts. This misalignment complicates the design and reuse of AA-solutions for real‑world applications. Team Design Patterns (TDPs) offer a promising approach by capturing reusable solutions to recurring challenges in human–automation teamwork. Yet, the systematic construction of coherent TDP sets remains difficult due to a lack of structured design support. To address this, we present an integrated human‑automation teaming framework that facilitates TDP development and supports cross‑disciplinary dialogue between designers, engineers, command staff, and policy‑makers. The framework extends NASA’s eight‑level LoA framework by assigning descriptive level names, explicit human‑loop status (in/on/out‑of‑the‑loop), and visual responsibility mapping inspired by the European Defence Agency’s methods‑of‑control framework. Based on a naval Uncrewed Surface Vessel use case, we developed and evaluated three TDPs with domain experts: uniform task delegation, uniform goal delegation, and a mixed‑level pattern combining different LoA across decision-cycle stages. Expert evaluation confirmed that mixed‑level TDPs best reflect operational, regulatory, and technological realities. Overall, the framework provides a structured basis for designing flexible and context‑appropriate adaptive automation in VUCA environments.

INTRODUCTION

In socio-technical systems, humans and technology interact closely and are mutually dependent in order to achieve a shared goal. The performance, safety, and effectiveness of these systems do not emerge from technology

Received May 4, 2026 Revised June 19, 2026 Accepted July 6, 2026 Available online September 25, 2026

Designing by Degree: Levels of Automation and Team Design Patterns 833 alone, but rather from the joint optimization of the social (human) and technological (automation) elements. Thus, designing effective technology also means defining the collaboration and responsibilities for the socio- technical system as a whole. Rapid advances in Information Technology (IT) and Artificial Intelligence (AI) have led to increasingly complex technology, leading to increasingly complex socio-technical systems. This places new demands on how humans and technology collaborate. Properly defining these collaborations is especially relevant for applications in VUCA (Volatile, Uncertain, Complex, Ambiguous) environments such as healthcare, emergency response, and military operations, as failures in these environments can have major, immediate ramifications.

Effective human-automation teams must adapt task and control allocation to changing conditions without compromising safety, accountability, or operator understanding – a dynamic process known as adaptive automation (AA; de Greef et al., 2010; see Richardson et al., 2025; Scerbo, 1996 for overviews). Levels of automation frameworks are commonly used to describe how authority is distributed between humans and automation in AA systems. A wide array of Levels of Automation (LoA) frameworks have been developed, reflecting different perspectives on how responsibility can be distributed between humans and automation. Frameworks differ in their focus: some define LoA levels by system decision authority, while others by task allocation or supervisory control (Ruscio et al., 2024). Although these frameworks are often used to assign responsibility in VUCA environments, they typically provide limited guidance on how effective decisions are actually formed and adapted over time. As a result, they struggle to support dynamic, context-sensitive decision-making, which is essential for the design of adaptive automation in real-world operational settings. Incorporating the Observe–Orient–Decide–Act (OODA) loop can help address this limitation. Rather than specifying static allocations of authority, the OODA loop models decision-making as a continuous, iterative cognitive process that integrates perception (observe), interpretation (orient), choice (decide), and action (act; Richards, 2020). As such, the OODA loop provides a conceptual structure for analyzing and supporting effective decision-making under VUCA conditions, complementing LoA frameworks by grounding automation design in the dynamics of decision processes rather than fixed control levels.

Capturing task and control allocation in human–automation teams is essential for ensuring clarity of roles, responsibilities, and dynamic adaptation to changing conditions. To address this, van Diggelen and colleagues (van Diggelen et al., 2019; Van Diggelen and Johnson, 2019) developed Team Design Patterns, expanding traditional models of task and control allocation with intuitive diagrams that map out important aspects of human- automation teaming. These structured templates capture best practices for joint operations and form a part of the Socio-Cognitive Engineering (SCE) method (Neerincx et al., 2019). A TDP describes how roles, responsibilities, and coordination are structured at the team level to achieve effective and

resilient human–agent teamwork. An Interaction Design Pattern (IDP), on the other hand, specifies how those roles interact in concrete situations over time – including task allocation, information exchange, and shifts in authority between humans and automation (Borchers, 2000; Neerincx et al., 2016). Together, TDPs and IDPs provide systematic templates for addressing recurring challenges in human–automation teamwork, supporting designers in developing solutions that are both adaptable and context-sensitive, and involving stakeholders in the design process (Beuker et al., 2021). However, constructing a coherent and consistent set of TDPs remains challenging, particularly due to the complexity of adaptive automation and the lack of structured development methods for TDPs and IDPs.

Therefore, we present an integrated human-automation teaming framework tailored -but not limited- to applications in VUCA environment, that facilitates the formulation of TDPs and cross-disciplinary dialogue between designers, engineers, command staff, and policy-makers.

AN INTEGRATED LEVELS OF AUTOMATION FRAMEWORK FOR DESIGNING HUMAN-AUTOMATION TEAMING Our integrated framework builds on the NASA Levels of Autonomy (LoA) (Proud et al., 2003), which combines an eight-level LoA model with the OODA-loop to define human-automation teamwork in detail. We further refined the NASA LoA to improve the information on the allocation of responsibilities by (see Figure 1):

1. Assigning human-loop status (human in, on, or out of the loop; Fischer et al., 2021) for each level. 2. Assigning descriptive names to each level based on delegation to autonomy literature, rather than merely numbering the levels. 3. Visually clarifying the framework by incorporating the European Defence Agency (EDA) methods-of-control framework (European Defence Agency, 2022, pp. 17–22), which uses visual representations to convey the balance of control between human operators and autonomous systems;

Human-Loop Status While LoA levels describe how much automation is present, they focus less on how engaged an operator needs to be. Incorporating loop-status helps users identify when the operator must remain actively engaged in all OODA- loop steps (in-the-loop), when monitoring and/or supervisory involvement with the possibility to intervene is sufficient (on-the-loop), and when the system operates autonomously (out-of-the-loop). Note that Including loop- status could also support identification of out-of-the-loop performance risks, clarifies accountability and decision authority, and strengthens trust calibration and transparency in human–automation teams.

Figure 1: Integrated levels of automation framework. based on the NASA LoA framework (Proud et al., 2003), with human-loop status, descriptive

Figure available in the original PDF paper.

names, and visual representations.

Descriptive Names for Each Level of Automation Giving each of the eight levels a descriptive name helps users understand their purpose and differences, making it easier to discuss and compare them. Without these labels, referring to levels by number alone can hide their key meaning. Here we give the descriptions of each LoA, and why a certain name was assigned.

  • Level 8 “Mission delegation” – The human assigns only a mission goal, after which the automation executes the mission autonomously. The human cannot obtain mission status updates from or further interact with the automation. As delegating the overall mission objective to the automation is the only role of the human, this level received this label. The human is ‘out of the loop’.
  • Level 7 “Mission delegation with veto” – The human is only informed if the mission context requires it and is allowed to override a decision only after execution has begun, hence the label. This level corresponds to the human assigning a mission goal, with the automation fulfilling the mission and showing the human what choices have been made. The human is ‘on the loop’.
  • Level 6 “Goal delegation” – The human provides a set of goals, hence the label. The automation generates ranked tasks and informs the human of the automatic execution of the best option, while the human can intervene only after execution has begun. The automation can generate new tasks, in addition to selecting pre-defined actions. The human is ‘on the loop’.
  • Level 5 “Play delegation” – The human provides a set of constraints and goals, shadows the interpretation of the automation, and is allowed a context-dependent time to veto before an option is executed. At this level, the automation selects the most appropriate play for the available assets from a set of pre-defined plays, based on constraints set by the human for the current situation, hence the label for this level. The human is ‘on the loop’.
  • Level 4 “Task delegation” – Both human and automation interpret and rank information, but the automation’s ranking considered more important than the human’s. The human selects a proposed task, which the automation then executes. If the human does not select a task within a pre-programmed timeframe, the automation will proceed with their proposed task. This level features automation proposing tasks for assets, while humans decide their suitability, keeping the human ‘in the loop’.
  • Level 3 “Informed” -The human generating and manually assigning tasks, which the automation then executes independently but under strict human supervision. The label ‘informed’ is assigned as the human remains fully apprised of all data and decisions, ensuring they are always aware and able to approve or intervene at any stage of the decision- making process, keeping the human ‘in the loop’.
  • Level 2 “Operated” - The human is responsible for performing all OODA-loop steps but can use automation as a tool for decision making. The automation shadows for contingencies. This is aligns with tele- operation of a remote system, however with some automation support.

Designing by Degree: Levels of Automation and Team Design Patterns 837 As the human actively operates the system, the corresponding level name was assigned. The human is ‘in the loop’.

  • Level 1 “Human” - The human is solely responsible for all OODA-loop steps, with no assistance of any automation, hence the label. The human ‘is the loop’.

We selected the term ‘delegation’ for multiple framework levels because it reflects the increasing autonomy of automated systems and the evolving way humans assign control (Miller et al., 2005; Parasuraman et al., 2000). Delegation spans from simple task assignment to specifying high-level goals, allowing users to maintain oversight as automation performs increasingly complex functions (Parasuraman et al., 2000). Using ‘delegation’ in level names helps clarify how authority and control shift between human and automation (Kaber and Endsley, 2004).

Visually Clarifying Roles and Responsibilities In TDPs, visual diagrams are used to clarify the division of responsibility between human and system; applying this to our framework by adding descriptive visual representations of OODA responsibilities to each of the eight levels can help users understand both responsibility divisions and differences between levels. The automation is represented by a combination of icons of uncrewed systems for land, air, water, and underwater applications. The relative size of the human and automation icons roughly represent the weight of the authority distribution at that level of automation. The orange lines show the capabilities of each team member, and how they relate to each other, in line with the EDA ‘Best practice guide Unmanned maritime systems handling, operations, design and regulations’ (European Defence Agency, 2022, pp. 17–22). Solid lines represent a leading role in taking initiative and having authorization, where dashed lines represent a supporting role by giving feedback or represent data and/or information flows that are less important for the team interaction. Lastly, curved arrows represent internal reasoning and cognitive capabilities. These curved arrows contain a small icon representing the different OODA-loop components relevant for that team member at that level. An icon being present denotes a role for that team member at that level of automation, for that OODA-loop component, while the intensity of an icon represents the relative contribution of that team member to the common process, as can be seen in Table 1.

TEAM AND INTERACTION DESIGN PATTERNS To investigate the practical value of the framework, we constructed two distinct TDPs corresponding to Levels 4 and 6. Examining these two levels is especially relevant, as they highlight key transitions in human– automation collaboration. Level 4 features human-driven task selection with the automation suggesting options, while Level 6 allows automation to autonomously generate and execute tasks based on goals, followed by human review. These levels illustrate a transition from extensive human involvement with automation assistance (near-future) to more independently operating automation (future) and clarify shifts in authority and responsibility.

Table 1: Team design pattern for adaptive automation, allowing the human and automation to dynamically adjust responsibilities during the scenario.

Table available in the original PDF paper.

Title Mix of Automation Levels During Naval Surveillance With Autonomous USVs Description This pattern aims to balance human-automation teaming to maintain situation awareness, rapid sense-making, context-compliant decisions, and safe execution in a maritime surveillance mission with multiple USVs assets, without over-automation or over-burdening the human. By not following a single level for the entire OODA-loop, mismatches can be avoided.

Structure

1. Observe: System autonomously gathers, filters, and prioritizes information (e.g. about detections) to reduce operator cognitive load, while also interpreting and overlaying predictions for later review (Level 6 automation). Human is able to see detailed data on demand. 2. Orient: System performs initial interpretation, proposes situation hypotheses by labelling, and provides “why” rationales (Level 6 behavior). Human still has the opportunity to see unfiltered but highlighted views (Level 4 transparency) to validate or refine interpretation. 3. Decide: System proposes ranked actions (Level 4 rule), but also provides reasoning (Level 6 explanation). Human can create actions to overrule systems raking (e.g. with context-sensitive decisions). 4. Act: For safety-critical or escalatory actions, the system executes only after human approval (Level 4 veto). For non-critical background maneuvers (e.g. deconfliction), system can execute automatically (Level 6 autonomous).

Human requirements

  • Ability to interpret raw sensor information when needed.
  • Competence in evaluating ranked action options under time pressure.
  • Understanding of what is automated and what is not.
  • Training in interpreting system rationales (“why” explanations) to calibrate trust and avoid over-reliance.

System requirements

  • Capable of filtering, prioritization, and interpretation of multi-sensor data with confidence values.
  • Can show unfiltered information, highlight actions, rank proposals, and provide time-to-veto controls.
  • Explanation generator (“why this interpretation / why this action”).

(Continued)

Table 1: Continued.

Table available in the original PDF paper.

Title Mix of Automation Levels During Naval Surveillance With Autonomous USVs Advantages (claims) + Controlled execution: Human oversight is maintained in decision- making and execution, supporting safety, and adherence to the mission context. + High responsiveness: The system manages detections efficiently, minimizing human workload so operators can concentrate on key decisions. + Shared understanding: Operators retain situation awareness by accessing detailed data while benefiting from system-supported interpretation. + Scalable supervision: The system allows operators to manage multiple USVs with reduced workload, balancing automation and human ownership for efficient supervision.

Disadvantages - Mode confusion: Configurable execution modes may increase the risk of operator misunderstanding who is in control. - Trust calibration challenges: High autonomy in early phases (observe/ orient) can lead to over-reliance or delayed intervention later. - System complexity: Multiple control modes and detailed levels may increase development efforts.

As an example use case of (semi)autonomous agents that are expected to operate in VUCA environments, this paper focuses on the development and deployment of uncrewed boats sailing in a fleet. A Human-Machine Interface was developed for each TDP to demonstrate the operational workflow and information exchange at its specific level. These interfaces were evaluated by two domain experts in naval operations through a three-step scenario with increasing time pressure – from low urgency to an acute threat.

Results showed that there was no clear preference for one of the levels in each situation, but highlighted strengths of both levels in different situations. Therefore, we created a mixed-level TDP integrating elements of both levels to better match real-world conditions where responsibility may shift during different OODA-loop phases. A new interface was designed and evaluated by two additional experts using a similar method.

The evaluations confirmed a context-dependents preference for different LoA for each OODA-loop step rather than a uniform level throughout the decision making process.

An IDP has been prepared from this TDP to serve as an example of how an operator and the automation interact.

Table 2: Interaction design pattern presenting an example how to design for context dependent autonomous execution.

Table available in the original PDF paper.

Title Conditional Execution With Time-to-Veto or Autonomous Execution Design problem Supporting both human-approved execution for safety-critical actions and autonomous execution for routine actions requires careful interaction design to avoid operator confusion, automation surprise, and loss of accountability.

(Continued)

Table 2: Continued.

Table available in the original PDF paper.

Title Conditional Execution With Time-to-Veto or Autonomous Execution Design solution The system provides a configurable execution mode that allows for switching between two execution behaviors:

1. Human-approved execution (L4 veto before execution) for safety- critical escalatory actions: the system executes only after explicit human approval or the expiry of a visible time-to-veto interval. 2. Autonomous execution with post-execution override (L6 autonomous) for non-critical background actions: the system executes autonomously and informs the operator post-execution, allowing override where physically feasible. These execution modes can be configured prior to the mission execution. Critical tasks may also be labeled as hard-locked to human-approved execution that cannot be switched to autonomous mode during mission execution.

Use when When missions involve both safety critical and routine actions, operators supervising multiple autonomous assets must carefully manage their workload to ensure mission success.

Design rationale

Execution authority should vary with action sensitivity, rather than being fixed at a single LoA. Allowing controlled switching between Level 4 and Level 6 supports responsiveness and scalability while preserving human ownership of sensitive actions. Making the execution mode explicit prevents automation surprise and clarifies responsibility at the moment of action.

Example This example shows (left) a “human-approved task” and (right) an “autonomous execution with post-execution override”.

Status Concept.

CONCLUSION

We present an integrated LoA framework that explicitly supports the systematic development of TDPs for human–automation collaboration. By supplementing the NASA LoA framework with named levels, visual responsibility mapping, and explicit human-loop status, our framework offers a clear and easily communicable structure for reasoning about roles and responsibilities for human-automation teams. Importantly, it can function as a shared frame of reference across designers, technology developers,

Designing by Degree: Levels of Automation and Team Design Patterns 841 operators, and norm-setters. By developing TDPs that combine different LoA across OODA-loop steps, the framework provides actionable guidance for designing flexible and situation-appropriate AA-solutions. Allowing for a combination of LoA also enables the description of systems in which e.g. the Observe and Action steps are carried out by a high-LoA automation, while the Orient and Decide steps are the responsibility of the human. Furthermore, we have provided an example of an IDP, demonstrating how these principles can be applied in practice. However, it should be noted that additional IDPs could be developed for the mixed TDP. Additionally, the claims regarding the mixed TDP have not yet been fully evaluated, and further assessment is required to establish their effectiveness and validity.

Looking forward, we are confident that as this framework keeps evolving, it can support in building appropriate trust calibration and smoother teamwork between people and automated systems. In the end, this should lead to safer, better solutions across a wide range of contexts.

ACKNOWLEDGMENT

The authors would like to thank the domain experts who participated in the evaluation sessions for their time, expertise, and valuable feedback, which substantially contributed to the quality of this work. This work was supported by a Dutch Ministry of Defense’s exploratory research program.

REFERENCES

Beuker, T., Mioch, T., A. Neerincx, M., 2021. Team Design Patterns for participatory development of First Response Human-Agent Teaming. Presented at the Human Systems Engineering and Design (IHSED2021) Future Trends and Applications. https://doi.org/10.54941/ahfe1001151 Borchers, J.O., 2000. Interaction Design Patterns: Twelve Theses, in: Workshop, the

Hague. de Greef, T.E., Arciszewski, H.F.R., Neerincx, M.A., 2010. Adaptive Automation Based on an Object-Oriented Task Model: Implementation and Evaluation in a Realistic C2 Environment. Journal of Cognitive Engineering and Decision Making 4, 152–182. https://doi.org/10.1518/155534310X522851 European Defence Agency, 2022. Best practice guide Unmanned maritime systems

handling, operations, design and regulations. Fischer, J.E., Greenhalgh, C., Jiang, W., Ramchurn, S.D., Wu, F., Rodden, T.,

2021. In-the-loop or on-the-loop? Interactional arrangements to support team coordination with a planning agent. Concurrency and Computation: Practice and Experience 33, e4082. https://doi.org/10.1002/cpe.4082 Kaber, D.B., Endsley, M.R., 2004. The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task. Theoretical Issues in Ergonomics Science 5, 113–153. https://doi.org/10.1080/1463922021000054335 Miller, C., Funk, H., Wu, P., Goldman, R., Meisner, J., Chapman, M., 2005. The

PlaybookTM Approach to Adaptive Automation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 49, 15–19. https://doi. org/10.1177/154193120504900105

Neerincx, M.A., Van Diggelen, J., Van Breda, L., 2016. Interaction Design Patterns for Adaptive Human-Agent-Robot Teamwork in High-Risk Domains, in: Harris, D. (Ed.), Engineering Psychology and Cognitive Ergonomics, Lecture Notes in Computer Science. Springer International Publishing, Cham, pp. 211–220. https:// doi.org/10.1007/978-3-319-40030-3_22 Neerincx, M.A., van Vught, W., Blanson Henkemans, O., Oleari, E., Broekens, J.,

Peters, R., Kaptein, F., Demiris, Y., Kiefer, B., Fumagalli, D., Bierman, B., 2019. Socio-Cognitive Engineering of a Robotic Partner for Child’s Diabetes Self- Management. Front. Robot. AI 6. https://doi.org/10.3389/frobt.2019.00118 Parasuraman, R., Sheridan, T.B., Wickens, C.D., 2000. A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 30, 286–297. https://doi. org/10.1109/3468.844354 Proud, R.W., Hart, J.J., Mrozinski, R.B., 2003. Methods for Determining the Level

of Autonomy to Design into a Human Spaceflight Vehicle: A Function Specific Approach. Richards, C., 2020. Boyd’s OODA Loop. 142–165. Richardson, L.S., Fidock, J., Gunawan, I., 2025. Systematic Literature Review of Levels of Automation (Autonomy) Taxonomy: Critiques and Recommendations. International Journal of Human–Computer Interaction 41, 15824–15843. https:// doi.org/10.1080/10447318.2025.2502978 Ruscio, D., Metge, A., Schopp, M., Dreyer, D., Petermeier, B., 2024. The General

Automation Level Allocation (GALA) Framework, or: Why Do We Need Another Level of Automation Framework? https://doi.org/10.54941/ahfe1005372 Scerbo, M.W., 1996. Theoretical Perspectives on Adaptive Automation, in: Automation and Human Performance. CRC Press, pp. 37–63. Van Diggelen, J., Johnson, M., 2019. Team Design Patterns, in: Proceedings of the

7th International Conference on Human-Agent Interaction. Presented at the HAI ’19: 7th International Conference on Human-Agent Interaction, ACM, Kyoto Japan, pp. 118–126. https://doi.org/10.1145/3349537.3351892 van Diggelen, J., Neerincx, M., Peeters, M., Schraagen, J.M., 2019. Developing Effective and Resilient Human-Agent Teamwork Using Team Design Patterns. IEEE Intelligent Systems 34, 15–24. https://doi.org/10.1109/MIS.2018.2886671

Cover of the proceedings volume Human Systems Engineering and Design (IHSED2026): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED2026): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 979-8-950676-13-0

Explore this ISBN in the AHFE Bookstore Use the ISBN to find the volume · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →