Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Orchestrating Humans and Non-Human Teammates to Counter Security Threats: Human-Autonomy Teaming in High and Low Environmental Complexity and Dynamism

The Norwegian Defence Research Establishment, Norway ↗
Publication year
Digital Object Identifier10.54941/ahfe1002830

Abstract

The changes in the security environment run parallel to changes in Humans and Artificial Cognitive Systems to meet these challenges. In military setting novel technologies in terms of high-speed missiles and the threat of anti-access areal denial capabilities, run parallel to more sophisticated fighter jets and air defense systems to counter such threats. Adverse conditions could involve loss of communication among some of the entities taking part in the mission and the sudden increased threat. Such changes require increased information processing (e.g. to understand the threat and properly sequence actions of each team) and sometimes a change of who coordinate the mission (e.g. because a formal leader is no longer available due to communication loss). While novel Humans and Artificial Cognitive Systems may be important to handle such situations, it is important to enable the use of the technologies so that they will actually have the effect of reducing threats. In this paper, we discuss some theoretical models for how Humans and Artificial Cognitive Systems can be orchestrated to enable the reduction of threats. We focus on the way the use of technologies are integrated. In this way, we keep the focus on the organizing dimension of Humans and Artificial Cognitive Systems use. Drawing on theoretical perspectives of organizational environment, we discuss some models of integration the use of Humans and Artificial Cognitive Systems. The organizational environment may vary along the following characteristics: uncertainty and ambiguity (Scott & Davis, 2007), where uncertainty can be divided into complexity (number of elements and number of relations among elements in an environment) or dynamics (the rate of change in elements in the environment; for a summary see Valaker et al., 2020; Grote, Kolbe & Waller, 2018; Luciano, Nahrgang & Shropshire, 2020). We discuss both human-to-human and human-machine-teaming as ways of handling such environmental contingencies. The following hypothesis are suggested: Clearly defining what situation is to be tackled could ensure using the available structures. E.g. not using a decentralized structure in high stakes situations. This criteria need however to be weighed against the practicality of using a centralized structure (Hollenbeck et al., 2018; Johansson et al., 2018).Utilizing Human-Machine teammates and developing organizational structures that incorporates their use in adverse environments (e.g. allocating Drones to Dull, Dirty and/or Dangerous tasks). This could also include increasing numbers and diverse Tech use i.e. from tool to team mate. This will impact: the conduct of operations; and the supervisors, coordinators and operators, who collaborate with these smart unmanned systems at individual level (e.g. adaptable working agreements), team level (peer to peer collaboration) and organization level (hierarchical collaboration).

INTRODUCTION

Handling an external environment is key to many types of human-machine teams: e.g. the use of unmanned surveillance drones in difficult rescue operations and the widespread use of such drones in a military setting to ensure endurance and reach. In these cases human-autonomy teaming is

© 2023. Published by AHFE Open Access. All rights reserved. 131

needed where human-autonomy teams (HATs) are defined as “at least one human working cooperatively with at least one autonomous agent (McNeese et al., 2018), where an autonomous agent is a computer entity with a partial or high degree of self-governance with respect to decision-making, adapta- tion, and communication (Demir et al., 2016; Mercado et al., 2016; Myers et al., 2019).” (O’ Neill et al., 2022, p. 904). The problem is how to be able to adjust own actions taking into account environmental characteristics, in such a way that the HAT is continued and able to perform their tasks despite various environmental conditions. Prior research have suggested that when complexity of the environment is low more decisions may be delegated to the computer entity but when the complexity increases human control is needed (Abbink et al., 2018). Furthermore, prior research indicate that task resolu- tion, of both human-to-human and human-machine teams, is impaired, in general, when complexity increases (O’Neill, 2022).

However the degree to which more complex situations can be handled by machine and humans may depend on the different designs of HATs ran- ging from planned levels of automation (LOA) via mixed-initiative (MI) to enacted coactive designs (COAD) (Jiang & Arkin, 2015; Kaber, 2018a). Some research is also more optimistic concerning the ability of technology to handle unforeseen situations (Goodrich, Adams & Scheutz, 2021; Lundberg et al., 2022). A concrete example is the development of drones that over time have developed to more adaptable entities. We suggest that environmental characteristics, specifically complexity and dynamics and their influence on coordination and performance (Mouloua et al., 2020) will vary according to HAT design used. In the face of environmental characteristics we suggest that the three HAT designs have strengths and weaknesses, and to overcome such tradeoffs one may use different HAT in concert, e.g. LOA may reduce com- munication costs, while COAD may increase synergies among human and computer entities. Thus, interfaces (Rico et al., 2018) need to be refined in parallel to advancements in the capabilities of autonomy, especially with the vision of a human operator and intelligent agent dynamically collaborating to solve problems and share task completion in a manner similar to effe- ctive human teams e.g. supporting peer-to-peer type communication between human-autonomy team members (Schraagen et al., 2022). We ground these suggestions on dynamic decision theory (DDM) and organization theory which points to the complementarity of feedforward (planned coordination), and feedback (coordination through adjustments) (Brehmer, 1992; Simon, 1957) as they handle different problems, e.g. problems of scheduling and problems of unforeseen events.

On this background the purpose of this article is to elucidate the follow- ing research question: How do different HAT designs contribute to support the coordination of task under various environmental characteristics? In this article, we thus discuss how HAT designs, specifically levels of automation, mixed-initiative and coactive design, may support changes to the workflow between man and machine (Fitts, 1947; Sherdian, 1978; Parasuraman, She- ridan & Wickens, 2000), due to different environmental characteristics. This article thus explores some parts of a research gap identified by O’Neill et al. (2022) who call for investigating the role of different task conditions for HAT

Orchestrating Humans and Non-Human Teammates to Counter Security Threats 133

designs. We do so by focusing on certain characteristics of the task environ- ment. We also build on and extend the prior work that we have done on HAT designs. In a 2021 paper we discussed the prospect of HAT teaming in the context of unmanned combat aircrafts collaborating with fighter jets. Stensrud et al. (2021) indicated that the dynamic of tasks would influence the type of coordination between human and non-human entities requiring a mix of formal and informal mechanisms, but here we add the influence of environmental complexity and look upon a less controllable empirical setting. We discuss a use case building on prior empirical and conceptual work that we and others, have done regarding F-35 and loyal wingman (e.g. Stensrud, Mikkelsen & Valaker, 2020; Stensrud, Betten, Mikkelsen & Valaker, 2021; Bjerke & Valaker, 2022). Finally we discuss future research and practical implications.

EFFECTS OF DIFFERENT SYSTEM DESIGN APPROACHES ON COORDINATION UNDER DIFFERENT ENVIRONMENTAL CHARACTERISTICS We now discuss how LOA, MI and COAD may support coordination in two types of environmental characteristics: complexity and dynamism. Coordi- nation we define as the integration of interests, understanding and activities to reach a common goal (Mathieu et al., 2018; Van de Ven, Delbeq & Koe- nig, 1976; Kouchaki et al., 2012; Grote et al., 2018). We also discuss some potential costs of using the designs, e.g. communication cost. Complexity we define as the number of elements and number of relations among elements in an environment (Schneider et al., 2017) and dynamics as the rate of change in elements in the environment (Dess and Beard, 1984). On this background we briefly present some key assumptions regarding the designs and the exter- nal environment, and then discuss the influence of complexity and dynamics specifically.

LOA concern prescribed “levels” where the human does “everything to a level where the computer does everything”(Verplank & Sheridan, 1978, p. 8- 5) and the research have been concerned with how LOA changes the working conditions for human operators (Fitts, 1947; Sherdian, 1978; Parasuraman, Sheridan & Wickens, 2000). Several extensions and modifications to the ori- ginal framework have been made (Vagia, Transeth & Fjerdingen, 2016) a recent example is Cabrall et al. (2018), and it has been used in several practi- cal applications (Hopkins & Schwanen, 2021). One key observation is that automation may hamper visibility and as a consequence situation awareness of humans if used extensively. Miller and Parasuraman (2007) for exam- ple suggest that LOA could be changed if the environment demanded, and that automating decision-making functions “may reduce human operators’ awareness of system and environmental dynamics” (Miller & Parasuraman, 2007). On the other hand, extensive automation and delegation may help perform tasks in difficult environments and when speed is essential (Kaber, 2018 b). The research to date indicates that flexible use of LOA could be used to handle environmental demands.

The MI approach distinguishes itself from LOA by positing a more equal role for the machine and has been described as “a flexible interaction strategy in which each agent (human or computer) contributes what it is best suited at the most appropriate time.” (Hearst, 1999, p. 14). Later on Jiang and Arkin (2015) have developed this definition to encompass robots. Jiang and Arkin (2015) suggests that feedback from an external environment or inferred state of environment can trigger initiative in a reactive or deliberate way, however uncertainties of the environment can make initiative reasoning challenging (Kirlik, 1993).

One of the key components of COAD is to emphasize collaboration rather than delegation as principles of man-machine interaction, it emphasizes the observation of and sharing of knowledge of own status and knowledge of internal interdependencies and external environment, as well as the predi- ctability of own actions in order for others to rely on them and directing behavior and be directed (Johnson, Bradshar & Feltovich, 2014). Johnson and Bradshaw (2021) explain the problem of people becoming unaware of changing in environment and system states when changes are under control by other agents, by the idea that the system is opaque to the users. They claim that support for interdependence could mitigate such issues.

On the basis of this short summary of the three HAT designs we now suggests ways in which they may support coordination under complexity and dynamics (Brehmer, 2010). Detailed indications of the potential outcomes of the HAT designs are shown in table 1 below.

In less complex situations LOA may be preferred because less feedback is required. A dominant feedforward (command based) coordination mecha- nism based on a high degree of pre-programmed and role-based assignment of authority. On the other hand it requires that formal roles and capabilities (of human and machine) must be defined prior to task resolution, something that could be increasingly difficult to foresee when complexity and dynamics increase. MI and to an even larger extent COAD may be (at least initially) to reactive to fully function in time critical tasks.

On the other hand: In highly complex situations, more elements need to be observed, processed and potentially communicated about in order to coor- dinate between man and machine. There may be a likelihood that the tasks cannot be as easily divided in high complexity and high dynamic situations as in low complex and low dynamic situations. We may posit that increased interdependence among coordinating elements in an extended socio-technical multiteam (Luciano et al., 2020) that includes non-human actors, leads to increased coordination requirements on the boundary of this arrangement. As the degree of interdependence between entities increases, the need to pro- cess more information increases. In such situations HAT designs that rely on feedback should function better and MI and perhaps even more COAD could be chosen rather than LOA. A dominant feedback (e.g. threat based model) coordination mechanism based on a high degree of reactive feed- back and operator-based assignment of authority, will promote this type of arrangement (informal control).

In other words our brief analysis indicate that there are trade-offs between the HAT designs. To compensate for the trade-offs using the different HAT

Orchestrating Humans and Non-Human Teammates to Counter Security Threats 135

Figure 1a: Pre-scribed socio-technical multiteam that includes non-human actors. (Adapted from Stensrud et al., 2020).
Original PDF Figure 1a:

designs in concert could be a solution. A feedback and feedforward inte- grated model fully integrating feedforward and feedback will promote this evolution (formal and informal) adjusting who coordinates based on both (pre-programmed) role and (individually and situated selection of) operator most suitable to mitigate any gap of authority. (Johnson 2018). For example MI (and/or COAD) could be used together with dynamic LOA levels (Petou- sakis et al, 2020; Schmitt et al., 2018; Lindner et al., 2022). Playbooks such as suggested by Miller and Parasuranam could be used to ensure predictabi- lity in a COAD design. In this way planned and emergent activities could be joined.

USE CASE We are now illustrating and discussing the suggestions made in the theory section in a use case. We focus on fighter aircraft (e.g. F-35) that collaborate with loyal wingmen (e.g. surveillance drones) to solve missions (Rebensky et al., 2022). There is a need for an Interaction mechanisms, i.e. HAT design, in order to ensure coordination. These tactical man-machine systems is embedded in a larger organization (Multinational military organization) which directs the activities of the fighter aircraft and loyal wingmen, illu- strated in figure 1a and 1b. Figure 1a illustrate a pre-scribed socio human capability that includes non-human actors and limitations are considered, and figure 1b illustrate an emergent and dynamic handover take-over event with a changed socio human capability that expand ability to include non- human actors. The lower part of the Figure 1 a and b illustrates the fighter aircraft and loyal wingman system.

We draw on prior unclassified reports by RAND to form a use case: “(…) low-observable, multirole F-35 Lightning II Joint Strike fighter could be used as both a sensor and a shooter in a SEAD campaign.7 Still, air plan- ners will likely want to reduce the amount of time that F-35 aircraft spend in

Table 1. Evaluation of coordination through HAT designs under different environmental characteristics.

Table available in the original PDF paper.

Mixed-initative Coactive-design

Environmental characteristics

Levels of automation Low complexity High coordination possible because it easy to define what can be done by man and machine respectively

High coordination, but may be an overly complex method of teaming since complementarity may be less needed.

High coordination, but may incur a cost as to much emphasis is put on feedback, when it is not realy needed to have feedback

High complexity

Good coordination, but relatively few mechanisms for feedback and adjustments may hamper operations.

Difficult to coordinate because the number of elements and their interrelations are many and therefore it is hard to decompose and delegate tasks.

Good coordination because feedback is processed concerning the interrelations. However, dependent on capability to observe and predict and handle concurrent tasks.

Low dynamics Good coordination because it is easy to predict the upcoming tasks and hence to predict to whom to delegate to.

Good coordination and easy to predict the complementarity (e.g. sequencing of human and machine actions is easy)

Good coordination, but may incur a cost as to how much feedback is needed, because the environment is predictable and hence feedback is not needed frequently.

Good coordination because ongoing feedback can aid prediction even in high dynamics.

Good coordination, but few mechanisms to adjust for sudden changes in who do what.

High dynamics Difficult to coordinate because of the many changes in environment that makes it hard to predict what to delegate.

highly contested airspace by leveraging space-based ISR to help locate adver- sary SAMs [surface-to-air-missiles].8 Using long-range precision ground fires would also increase the firepower available to strike targets, offer a redundant capability to strike SAMs if aircraft need to leave the area, and complicate an enemy’s defense planning (…)” (Priebe et al., 2021: p. 37). We extend this case to include loyal wingman (Stensrud et al., 2020) that augment and are complementary to the F-35. The loyal wingman are drones that could but

Orchestrating Humans and Non-Human Teammates to Counter Security Threats 137

Figure 1b: Emergent socio-technical multiteam that includes non-human actors (Ada- pted from Stensrud et al., 2020).
Original PDF Figure 1b:

used in a forward position both to surveil and attack an enemy air defence system. In this case we focus on a loyal wingman that can provide updated information about the enemy situation and that can be in a forward position and closer to the enemy than the fighter aircraft.

We simplify the environment to two environmental characteristics discus- sed above theoretically: the number of elements in the enemy integrated air defence (e.g. number of radars, missile launchers, C2 nodes etc), which is the complexity of the environment, and the variability of an additional air threat (e.g. number of incoming enemy fighter aircraft) which is the dynamic cha- racteristic. We assume that in all conditions a set number of elements in the SAMs should be striked (eventhough the total number of SAMs increase) and we keep the number of friendly airframes constant. The low complexity low dynamic conditions are illustrated in the planned mode in figure 1a, while the high complexity and high dynamics are illustrated in the emergent mode in figure 1b.

We evaluate coordination, as in the theory section, but also add conside- rations of costs of coordination in the different HAT designs. The results of our analysis based on experience from workshops with military officers are presented in Table 2. We do not present results for the low dynamics as we focused on the critical issue of handling high dynamics. Largely our analysis of the empirical use case conformed to the theoretical discussions, although it provided some granularity to the theory. Overall, similar to theory the discus- sion of the case highlighted that LOA were preferred in less complex less dynamic situations and MI and COAD was preferred when complexity and dynamics increased. With respect to LOA it highlighted the relatively high cost of planning (i.e. deciding beforehand who do what and delegate) even in low complexity. With respect to MI the cost of making interfaces betw- een man and machine to accommodate transactions was highlighted. In high

dynamics situations it also suggested that the humans in the MI mode need to prioritize tasks on a high level rather than do detailed interaction with techno- logy. Prioritizing tasks would require advanced technology. Regarding COAD the case illustrated a cost of using this method for low complexity low dyna- mics because of the lack of delegated and (initially) planned task allocation. It should be noted that these suggestions are preliminary and need further refinement through subject-matter expert input as well as experimentation.

DISCUSSION

Our preliminary theoretical analysis and analysis of a use case indicate that LOA, MI and COAD could be complementary in terms of supporting coor- dination under high and low complexity and high and low dynamics. This largely confirm prior research, but our study may open up the discussion of how the different HAT designs could be used in concert to support difficult missions. We now point to some avenues for future research and practical implications.

Future Research A key extension to this work is to define more clearly what the different HAT designs could offer in the particular case discussed and in other relevant cases. Such an analysis could for instance detail the communication requirements, the rules for delegation, and the mechanisms for updating among entities offered by the designs. More work is needed on defining how the trade-offs between HAT designs could be overcome, e.g. through developing COAD and its interface to LOA and MI. We did only a very general discussion of the costs of different designs in different situations. It is likely that the cost will also change over time as the actors learn how to operate in the environment.

Crucially other characteristics of environments need to be discussed in more detail such as the different interpretations that could arise regarding an environment. Related to this the filtering and transmission of environ- mental cues could impact the operation of both human and machine and thus determine their interdendencies and coordination. On a general note the interaction of different characteristics need to be spelled out in more detail, e.g.: what is the consequence of a situation with high complexity and low dynamics versus a situation with high complexity and high dynamics?

The technical interface that could support the different HAT designs under different environmental characteristics is also in need of discussion, and rela- ted to this the trust among entities. In particular how to achieve coordination in congested and contested environments is in need of more study.

Practical Implications A suggested practical approach to implement different HAT designs is to use a supportive framework for design and modelling interfaces to evaluate the consequences of different HAT designs in a practical context (HSI 711th

AFRL, 2022; Park et al., 2020). In development of the collaboration between man and machine it seems to be important to support different needs, and that not one HAT design (that we know of) seem to support all needs. Thus

Orchestrating Humans and Non-Human Teammates to Counter Security Threats 139

Table 2. Preliminary evaluation of coordination and communication cost with different HAT designs under different environmental characteristics.

Table available in the original PDF paper.

F-35 with loyal wingman Mixed-initiative Coactive design

Environmental characteristics

Levels of automation

Coordination: Very high Cost: High because of requirement to interface during task resolution

Coordination: Medium. Require a supervisor role for the human. Cost: High Need to mitigate lack of planning through discover interdependencies.

Low complexity Coordination: High Cost: Medium Due to some planning needed to define task allocation and delegation authority throughout the mission in detail.

High complexity

Coordination: High Cost: High Need to define in much detail what to delegate

Coordination: Medium But better than in low com- plexity since the system is able to process the high number of elements Cost: Medium Scalability is challenging

Coordination: High But more likely to result in transaction overload. Cost: High Need to define “smart” interaction mechanisms to reduce transaction overload.

High dynamics Coordination: Medium Cost: High. Very detailed updating and definition of task allocation mechanisms needed

Coordination: High Cost: High. Requires detailed transaction mechanisms, and well-known procedures that allow the human operator to prioritize between tasks (in particular in low complexity and high dynamics)

Coordination: Medium Cost: Medium. Needed to have a repertoire of playbooks and select among these (in particular when high complexity is also present (Perkins, 2017)). Support team collaboration by modern technology (AI). (Seeber et al.,2020)

in designing and supporting such collaboration one may need to orchestrate different designs, such as making elaborate ways of delegating to the mach- ine, yet also be able to get feedback from the machine and its environment when needed to adjust and integrate on the fly.

CONCLUSION

While complexity may suggest that more control is to be held by the human, different HAT designs may ensure that control and goal achievement can be achieved also in complex and dynamic environments. A start point for inte- gration of teammates (e.g. loyal wingmen) and existing human controlled capabilities (e.g. fighter aircraft) could be to levelling up automation (buil- ding adapters, and interfaces) in order to design approaches that utilize the best from LOA, MI and COAD. The article presented a set of environmen- tal characteristics that could be important to consider e.g. complexity and dynamics, and pointed to tradeoffs between the designs in supporting human- machine coordination in varying environmental conditions. We hope that our preliminary discussion will enable teams to collaborate better by providing a common language and process to distribute models and share information about complementarities among the HAT designs.

ACKNOWLEDGMENT

The authors would like to acknowledge other colleagues at the Norwegian Defence Research Establishment that have contributed in discussions and rea- ding of the paper, as well as officers who have provided us with insight into military problems.

REFERENCES

Abbink, D. A., Carlson, T., Mulder, M., de Winter, J. C. F., Aminravan, F., Gibo, T. L., & Boer, E. R. (2018). A Topology of Shared Control Systems—Finding Common Ground in Diversity. IEEE Transactions on Human-Machine Systems, 48(5), 509–525. https://doi.org/10.1109/THMS.2018.2791570 Allen, J. E., Guinn, C. I., & Horvtz, E. (1999). Mixed-initiative interaction. IEEE

Intelligent Systems, 14(5), 14–23. https://doi.org/10.1109/5254.796083 Bjerke, H. M., & Valaker, S. (2022). Command and Control in a Fifth Generation Air Force: Coordination Requirements of Air Operations with F-35 and the Com- mand and Control-System of the Norwegian Armed Forces. Scandinavian Journal of Military Studies, 5(1), 14–30. https://doi.org/10.31374/sjms.116 Brehmer, B. (1992). Dynamic decision making: Human control of complex systems. Acta Psychologica, 81(3), 211–241. https://doi.org/10.1016/0001- 6918(92)90019-A Brehmer, Berndt,. (2010). Command and control as design. Cabrall, C. D. D., Sheridan, T. B., Prevot, T., de Winter, J. C. F., & Happee, R. (2018).

The 4D LINT Model of Function Allocation: Spatial-Temporal Arrangement and Levels of Automation. In W. Karwowski & T. Ahram (Eds.), Intelligent Human Systems Integration (Vol. 722, pp. 29–34). Springer International Publishing. https://doi.org/10.1007/978-3-319-73888-8_6 Committee on Human-System Integration Research Topics for the 711th Human Performance Wing of the Air Force Research Laboratory, Board on Human- Systems Integration, Division of Behavioral and Social Sciences and Education, & National Academies of Sciences, Engineering, and Medicine. (2022). Human- AI Teaming: State-of-the-Art and Research Needs. National Academies Press. https://doi.org/10.17226/26355

Orchestrating Humans and Non-Human Teammates to Counter Security Threats 141

Demir, M., Cooke, N. J., & Amazeen, P. G. (2018). A conceptual model of team dynamical behaviors and performance in human-autonomy teaming. Cognitive Systems Research, 52, 497–507. https://doi.org/10.1016/j.cogsys.2018.07.029 Dess, G. G., & Beard, D. W. (1984). Dimensions of Organizatio- nal Task Environments. Administrative Science Quarterly, 29(1), 52. https://doi.org/10.2307/2393080 Fitts, P. M., and R. E. Jones. (1947). Analysis of Factors Contributing to 460 “Pilot

Error” Experiences in Operating Aircraft Controls. Dayton, OH: Aero Medical Laboratory, Air Material Command, Wright-Patterson Air Force Base. Frazier, S. N. (2022). MEASURING COGNITIVE WORKLOAD IN AUTO- MATED KNOWLEDGE WORK ENVIRONMENTS (p. 141) [A Dissertation Submitted to the Faculty of Purdue University In Partial Fulfillment of the Requi- rements for the degree of Doctor of Philosophy]. Faculty of Purdue University. Goodrich, M. A., Adams, J. A., & Scheutz, M. (2022). Autonomy Reconsidered:

Towards Developing Multi-agent Systems. In K. Arai (Ed.), Intelligent Systems and Applications (Vol. 294, pp. 573–592). Springer International Publishing. https: //doi.org/10.1007/978-3-030-82193-7_38 Grote, G., Kolbe, M., & Waller, M. J. (2018). The dual nature of adaptive coordination in teams: Balancing demands for flexibility and stability. Organizational Psychology Review, 8(2–3), 125–148. https://doi.org/10.1177/2041386618790112 Hearst, M. (1999, October). Mixed-initiative interaction. IEEE Intelligent Systems.

https://www.researchgate.net/publication/3420505 Hopkins, D., & Schwanen, T. (2018). Experimentation with vehicle automation. In Transitions in Energy Efficiency and Demand. Routledge. Johnson, M., Bradshaw, J. M., Feltovich, P. J., Hoffman, R. R., Jonker, C., Riemsdijk,

B. van, & Sierhuis, M. (2011). Beyond Cooperative Robotics: The Central Role of Interdependence in Coactive Design. IEEE Intelligent Systems, 26(3), 81–88. https://doi.org/10.1109/MIS.2011.47 Johnson, M., Bradshaw, J. M., & Feltovich, P. J. (2018). Tomorrow’s Human– Machine Design Tools: From Levels of Automation to Interdependencies. Journal of Cognitive Engineering and Decision Making, 12(1), 77–82. https://doi.org/10. 1177/1555343417736462 Kaber, D. B. (2018a). Issues in Human–Automation Interaction Modeling: Presumptive Aspects of Frameworks of Types and Levels of Automa- tion. Journal of Cognitive Engineering and Decision Making, 12(1), 7–24. https://doi.org/10.1177/1555343417737203 Kaber, D. B. (2018b). A conceptual framework of autonomous and auto- mated agents. Theoretical Issues in Ergonomics Science, 19(4), 406–430. https://doi.org/10.1080/1463922X.2017.1363314 Kirlik, A., Miller, R. A., & Jagacinski, R. J. (1993). Supervisory control in a dynamic

and uncertain environment: a process model of skilled human-environment inte- raction. IEEE Transactions on Systems, Man, and Cybernetics, 23(4), 929–952. https://doi.org/10.1109/21.247880 Kouchaki, M., Okhuysen, G. A., Waller, M. J., & Tajeddin, G. (2012). The Treatment of the Relationship Between Groups and Their Environments: A Review and Cri- tical Examination of Common Assumptions in Research. Group & Organization Management, 37(2), 171–203. https://doi.org/10.1177/1059601112443850 Lindner, S., Mund, D., & Schulte, A. (2022, January 3). How Human-Autonomy

Teams change the Role of future Fighter Pilots: An Experimental Assessment.

AIAA SCITECH 2022 Forum. AIAA SCITECH 2022 Forum, San Diego, CA & Virtual. https://doi.org/10.2514/6. 2022–2551 Luciano, M. M., Nahrgang, J. D., & Shropshire, C. (2020). Strategic Leadership Systems: Viewing Top Management Teams and Boards of Directors from A Mul- titeam Systems Perspective. Academy of Management Review, 45(3), 675–701. https://doi.org/10.5465/amr.2017.0485 Lundberg, J., & Johansson, B. J. E. (2021). A framework for describing interaction between human operators and autonomous, automated, and manual control systems. Cognition, Technology & Work, 23(3), 381–401. https://doi.org/10.1007/s10111-020-00637-w M. Johnson, J. M. Bradshaw, P. J. Feltovich, C. M., Jonker, M. B. van Riemsdijk,

& M. Sierhuis, “Coactive Design: Designing Support for Interdepdence in Joint Activity”, Journal of Human-Robot Interaction, vol. 3, 1, pp. 43–69, 2014 McNeese, N. J., Demir, M., Cooke, N. J., & Myers, C. (2018). Teaming With a Synthetic Teammate: Insights into Human-Autonomy Teaming. Human Factors: The Journal of the Human Factors and Ergonomics Society, 60(2), 262–273. https://doi.org/10.1177/0018720817743223 Mercado, J. E., Rupp, M. A., Chen, J. Y. C., Barnes, M. J., Barber, D., & Procci, K.

(2016). Intelligent Agent Transparency in Human–Agent Teaming for Multi-UxV Management. Human Factors: The Journal of the Human Factors and Ergonomics Society, 58(3), 401–415. https://doi.org/10.1177/0018720815621206 Miller, C. A., & Parasuraman, R. (2007). Designing for Flexible Interaction Between Humans and Automation: Delegation Interfaces for Supervisory Control. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(1), 57–75. https://doi.org/10.1518/001872007779598037 Mouloua, M., Hancock, P. A., & Ferraro, J. (Eds.). (2020). Human performa-

nce in automated and autonomous systems. Current theory and methods. CRC Press/Taylor & Francis Group. Myers, C., Ball, J., Cooke, N., Freiman, M., Caisse, M., Rodgers, S., Demir, M., & McNeese, N. (2019). Autonomous Intelligent Agents for Team Training. IEEE Intelligent Systems, 34(2), 3–14. https://doi.org/10.1109/MIS.2018.2886670 O’Neill, T., McNeese, N., Barron, A., & Schelble, B. (2022). Human–Autonomy

Teaming: A Review and Analysis of the Empirical Literature. Human Factors: The Journal of the Human Factors and Ergonomics Society, 64(5), 904–938. https: //doi.org/10.1177/0018720820960865 Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 30(3), 286–297. https://doi.org/10.1109/3468.844354 Park, S., Grosser, T. J., Roebuck, A. A., & Mathieu, J. E. (2020). Understan-

ding Work Teams From a Network Perspective: A Review and Future Research Directions. Journal of Management, 46(6), 1002–1028. https://doi.org/10.1177/ 0149206320901573 Perkins, D. G. (2017). Multi-domain battle: The advent of twenty-first century war. Military Review, 97(6), 8. Petousakis, G., Chiou, M., Nikolaou, G., & Stolkin, R. (2021). Human operator cognitive availability aware Mixed-Initiative control. https://doi.org/10.48550/ARXIV.2108.11885

Orchestrating Humans and Non-Human Teammates to Counter Security Threats 143

Priebe, M., Douglas C. Ligor, Michael Spirtas, Karen Schwindt, Caitlin Lee, Ashley L.

Rhoades, Derek Eaton, & Bruce McClintock,. (2020). Multiple Dilemmas: Chal- lenges and Options for All-Domain Command and Control. RAND Corporation. https://doi.org/10.7249/RRA381-1 Rebensky, S., Carmody, K., Ficke, C., Carroll, M., & Bennett, W. (2022). Teammates Instead of Tools: The Impacts of Level of Autonomy on Mission Performance and Human–Agent Teaming Dynamics in Multi-Agent Distributed Teams. Frontiers in Robotics and AI, 9, 782134. https://doi.org/10.3389/frobt.2022.782134 Rico, R., Hinsz, V. B., Davison, R. B., & Salas, E. (2018). Structural influences

upon coordination and performance in multiteam systems. Human Resource Management Review, 28(4), 332–346. https://doi.org/10.1016/j.hrmr.2017.02. 001 S. Jiang and R. C. Arkin, “Mixed-Initiative Human-Robot Interaction: Definition, Taxonomy, and Survey,” in IEEE International Conference, on Systems, Man, and Cybernetics (SMC), 2015, pp. 954–961. Schmitt, F., Roth, G., Barber, D., Chen, J., & Schulte, A. (2018). Experimental Vali-

dation of Pilot Situation Awareness Enhancement Through Transparency Design of a Scalable Mixed-Initiative Mission Planner. In W. Karwowski & T. Ahram (Eds.), Intelligent Human Systems Integration (Vol. 722, pp. 209–215). Springer International Publishing. https://doi.org/10.1007/978-3-319-73888-8_33 Schneider, A., Wickert, C., & Marti, E. (2017). Reducing complexity by creating complexity: a systems theory perspective on how organizations respond to their environments. Journal of Management Studies, 54(2), 182–208. Schraagen, J. M., Barnhoorn, J. S., van Schendel, J., & van Vught, W. (2022). Sup-

porting teamwork in hybrid multi-team systems. Theoretical Issues in Ergonomics Science, 23(2), 199–220. https://doi.org/10.1080/1463922X.2021.1936277 Seeber, I., Bittner, E., Briggs, R. O., de Vreede, T., de Vreede, G.-J., Elkins, A., Maier, R., Merz, A. B., Oeste-Reiß, S., Randrup, N., Schwabe, G., & Söllner, M. (2020). Machines as teammates: A research agenda on AI in team collaboration. Informa- tion & Management, 57(2), 103174. https://doi.org/10.1016/j.im.2019.103174 Sheridan, T. B., & Ferrell, W. R. (1974). Man-machine systems: information, control,

and decision models of human performance. MIT Press. Sheridan, T., Verplank, W., & Brooks, T. (1978a). Human and Computer Control of Undersea Teleoperators. Sheridan, T., Verplank, W., & Brooks, T. (1978b). Human and Computer Control of

Undersea Teleoperators. Simon, H. A. (1957). Models of Man, Social and Rational. Oxford, England: Wiley. Stensrud, R., Mikkelsen, B., Betten, S., & Valaker, S. (2021). A proposal for a simple evaluation method in support of the initial concept phase assessing a future unmanned Loyal Wingman for Royal Norwegian Air Force (RNoAF). Stensrud, R., Mikkelsen, B., & Valaker, S. (2020). 5th generation fighter aircraft and

future command and control: A note on some possible changes. 25th international command and control research and technology symposium. Stensrud, R., Valaker, S., Haugen, R., & Mikkelsen, B. (2021). Possible consequences for command and control of shifting coordination form: Navigating the murky waters of adverse situations and complex missions. International command and control research and technology symposium. Stensrud, R., Valaker, S., & Haugen, T. (2020). Interdependence as an Element of

the Design of a Federated Work Process. 2020 IEEE International Conference on Human-Machine Systems (ICHMS), 1–6. https://doi.org/10.1109/ICHMS49158. 2020.9209423

Vagia, M., Transeth, A. A., & Fjerdingen, S. A. (2016). A literature review on the levels of automation during the years. What are the different taxonomies that have been proposed? Applied Ergonomics, 53, 190–202. https://doi.org/10.1016/j.apergo.2015.09.013 Valaker, S., Stensrud, R., Haugen, T., Eikelboom, A., & van Bemmel, I. (2020). The

Influence of Harmonization Levels on Multi-Domain Operations Effectiveness: the Moderating Roles of Organizational Environment and Command and Con- trol Variety. 25th International command and control research and technology symposium. Ven, A. H. V. D., Delbecq, A. L., & Koenig, R. (1976). Determinants of Coordination Modes within Organizations. American Sociological Review, 41(2), 322. https://doi.org/10.2307/2094477

Cover of the proceedings volume Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems

Explore the complete digital volume

Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-45-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →