Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Artificial Social Intelligence in Action: Lessons Learned from Human-Agent Hybrid Search and Rescue

1 Team Performance Laboratory, Institute for Simulation and Training, University of Central Florida, Orlando, FL 32826, United States ↗
2 Cognitive Sciences Laboratory, Institute for Simulation and Training, University of Central Florida, Orlando, FL 32826, United States ↗
Publication year
Digital Object Identifier10.54941/ahfe1004190

Abstract

Socially intelligent artificial agents have recently shown some evidence of improving team performance when advising human teammates during the execution of time-pressured, complex missions. These agents, imbued with a form of social intelligence supported by Artificial Theory of Mind, have also demonstrated some negative outcomes associated with their approaches to delivering advice and motivating teammates to succeed. Here, we closely examine team performance outcomes associated with a simulated team Urban Search and Rescue mission in the context of interventions delivered by artificial socially intelligent agents that served as advisors to the human teammates engaged in task execution. The task studied here required some individual taskwork effectiveness as well as a notable amount of interdependent teamwork coordination. The interdependent activities provided the advising artificially intelligent teammates an opportunity to observe and intervene to improve aspects of team process. Some of the interventions delivered by the socially intelligent agents were found to positively impact performance, notably those that targeted objective data and the dissemination of information to the right individual at appropriate timepoints; however, other interventions negatively impacted team outcomes. Results showed that Motivation interventions aimed solely at bolstering the motivation of team members did not yield positive outcomes; in fact, they were found to have adverse effects on overall team performance and task execution.

INTRODUCTION

The effectiveness of teams is dependent on interactions among team mem- bers, and the role of Artificial Intelligence (AI) agents in teams is a topic of growing interest. Largely, the role that AI has typically played in teams has been limited to static models executing certain tasks and performing human- defined functions (Kaplan and Haelein, 2019). Fundamental to our ability to interact, humans rely on social intelligence and Theory of Mind (ToM; Chen et al., 2021), a core socio-cognitive process that enables us to recognize

© 2023. Published by AHFE Open Access. All rights reserved. 176

Lessons Learned From Human-Agent Hybrid Search and Rescue 177

and attribute mental states, beliefs, desires, and intentions to others. ToM is critical to effective team interactions, which requires team members be able to understand, interpret, and predict one another’s behavior (Fiore et al., 2013). Thus, an agent will require social intelligence to engage as a coop- erative and collaborative team member – and, because it is foundational to human social intelligence, to effectively interact with humans in human-agent teams, an artificial social intelligence (ASI) will need to have an Artificial Theory of Mind (AToM; Williams et al., 2022). An ASI will need to engage in team interaction that supports teamwork processes. Teamwork processes have been described in terms of recurring processes and phases that teams go through over the course of teamwork, that support effective team inter- action – including transition phase processes, action phase processes, and interpersonal processes (Marks et al., 2001). Thus, ASI will need to be able to develop and maintain social cognition and AToM in complex, evolving scenarios.

In this paper, we report the findings of analyses exploring ASI interventions in human-agent teams. We are interested in exploring the types of interactions pertaining to certain teamwork processes - namely the impact of interventions targeting teamwork processes in the action processes and transition pro- cesses, which are more task and goal-oriented, and interpersonal processes that are designed to be motivational. Details of specific ASI architectures are outside of our current scope (see Huang et al., 2022b). In these analy- ses, we explore the impacts of ASI interventions across all ASIs employed in the reported study that were related to Information Sharing, Mission Status Updates, and Motivation content. The ASI were able to monitor team mem- bers, track experimental task objectives, and generate interventions with the goal of supporting teamwork and team processes. Interventions were text- based messages delivered to a single team member, a dyad, or all three team members. This manuscript follows our previous work on intervention anal- yses, which focused on explicit Theory of Mind interventions and detailed the methods used and briefly described here (readers interested in our ToM- related intervention analyses are referred to: Williams et al., 2023). In this paper, we continue our exploration of the content of these interventions as they relate to certain teamwork processes, and report our analyses of these in the context of mission outcomes.

METHOD

Experimental Task The data used in the analyses of this manuscript were collected by Arizona State University in the third study of the “Artificial Social Intelligence for Suc- cessful Teams” (ASIST; ASIST, 2023), an initiative of the Defense Advanced Research Projects Agency (DARPA) (see Huang et al., 2022a). The data used in these analyses can be accessed in the publicly available dataset (avail- able here: Huang et al., 2022b). The experimental task in this study utilized a Minecraft-based testbed to simulate gamified Urban Search and Rescue (USAR) missions in which teams of three human members had to gather information in the virtual environment of a partially collapsed building. The

testbed utilized a heads-up display to show objective-related information to participants, including a mission timer and regular and critical victim rescue statuses. Each team member was given a role with particular capabilities: the Medic role was able to determine the types of injuries a victim may have sustained so that they can be directed to the appropriate rescue point, or extraction point; the Engineer role was able to break fallen rubble that may block the path or rooms in different parts of the environment but had the slowest movement speed; and the Transporter role was able to investi- gate rooms using a signal device to determine if there were victims trapped in the room, and had the fastest movement speed. Every role was able to move, or transport, victims found in the environment, though the Transporter role moved more quickly so they were optimally suited towards moving the triaged victims to the extraction point. The teams’ overarching goal in the USAR missions was to locate injured victims of the building collapse, triage them, and get them to the correct extraction point based on their injuries. There were two types of victims, critical victims and regular victims, and three possible injury types, critical (type C; which were exclusively present in critical victims) and either abrasions (type A) or bone damage (type B) injuries for the regular victims.

Figure 1: Testbed interface from the Medic’s perspective, who is watching the Engineer.
Original PDF Figure 1:

Agent Role and Capabilities In addition to the three human team members executing the experimental USAR task, each team in these analyses were paired with an Artificial Social Intelligence agent (ASI) that was able to observe the teams’ actions within the virtual environment and communicate to the team through interventions, or text-based messages, that could be sent to a single team member, a dyad, or all three team members. There were six unique ASI developed as part of the ASIST program (e.g., Li et al., 2023; Wang et al., 2023) that interacted with team members as an advisor where they may make suggestions, provide advice and reminders, or motivate the teams. Each advisor’s interventions

Lessons Learned From Human-Agent Hybrid Search and Rescue 179

differed across all teams, such that no single advisor provided the exact same interventions in the same way to each team they assisted. The ASI were given access to pre-experimental trial surveys taken by the participants that were aimed at capturing aspects of teamwork and taskwork potential. These sur- veys include a video game experience measure (Bendell et al, 2020; Williams et al., 2023), the Reading the Mind in the Eyes Task (RMET; Baron-Cohen et al., 2002), and Santa Barbara Sense of Direction (SSOD; Hegarty et al., 2002). The results of the surveys were made available to the ASI agents so that they may use them toto inform their internal models of the team they were working with. The ASI interventions given could be informed by actions taken in the virtual environment, mission/task status, as well as the individ- ual differences in participants captured in these pre-trial surveys. ASI were not given omniscient complete advance knowledge of the environment, such as the specific locations of mission objects and objectives but were able to observe every action team members took as well as the field of view of the team members.

Agent Interventions Due to the fact that the six ASI’s were developed independently of each other and that the interventions given by even a single ASI varied with each team, we needed a way to discuss the content of the interventions given over the course of the study in a way that was agnostic to which ASI gave the inter- vention. In this paper we briefly discuss the methods used, but an interested reader is directed to Williams et al. (2023) for a more detailed description of the coding process. We first identified and extracted all unique interventions given in this study across the ASI, and included interventions that differed only by specific player reference or numerical values (e.g., “there are [n] min- utes remaining in this mission”) to ensure we were inclusive of all possible unique interventions. Then, we collaborative conducted an initial reflexive thematic analysis where we generated categories to label the interventions based on their content. We reviewed these categories and the unique inter- ventions list separately, and then came back to discuss the categories together. The first three authors discussed and further refined the coding categories until they agreed on the coding categories scopes and definitions. After the categories were set, the coders reviewed the interventions list and applied a primary code to each intervention first, and then, only where it was agreed by all coders that it was necessary, a secondary code to further describe the content of an intervention. The primary and secondary codes assigned to the unique list of interventions were then carried through to the full list of inter- ventions such that every instance of an intervention was given the appropriate category codes which was reviewed and agreed upon.

Here, we focus on a subset of interventions that are related to certain teamwork processes, and how they impacted the completion of mission objectives. ASI agents provided interventions that supported team members’ awareness of the status of different tasks and metrics. These types of interven- tions were designed to support team processes specifically related to Action Processes, including monitoring progress towards goals, systems monitoring. motivation and confidence building (Mathieu et al., 2020).

Table 1. Coding categories subset list.

Table available in the original PDF paper.

Category Category Description % of All Interventions Given

3.22%

Information Sharing …can help the team by telling about the C8 shortcut, there are … stabilized victims

6.50%

Mission Status Update you have transported N victims,... critical and... noncritical victims Motivation Bravo!, Great job, congratulations, keep going 30.20%

Note: These categories, definition and distributions represent a subset of those originally identified in (Williams et al., 2023).

Examples of interventions given by ASI agents that were assigned a pri- mary code or secondary code of Information Sharing, Mission Status Update, and Motivation are provided below in Table 2. To demonstrate the con- tent of interventions assigned these codes, the examples shown in Table 2 were selected to illustrate the variability in interventions given, even within a single category. To further emphasize this, some of these examples were assigned additional codes (Strategy General and Implicit Communication: Marker) to provide examples of the other intervention categories identified (see: Williams et al., 2023).

Table 2. Examples of interventions coded with information sharing, mission status update, or motivation.

Table available in the original PDF paper.

Intervention Primary Code Secondary Code

Motivation

Great job adapting your behavior! Motivation Keep going! Do your best teamwork. Motivation Great job getting that victim into the triage area, Green! Motivation Your team is doing pretty well, and you really showed some great effort towards the end of the previous trial. Keep that energy up for the next trial. You can do this!

Motivation Mission Status Update

11 non-critical and 10 critical victims stabilized.17 victims transported. Victims being stabilized now = 1. Victims being transported now = 2. GO TEAM!

Team, the mission is nearing its end; let’s finish triaging and moving victims instead of trying to find more.

Strategy General Information Sharing

Strategy General Mission Status Update There are 3 minutes left. Mission Status Update There are two care sites for each type, A, B, and Critical. You carried the victim to the more distant location of the two for your victim type. USE THE CLIENT MAP to find the closest care area to save time.

You just missed marking that last room. There was a critical victim there.

Information Sharing

Implicit Communication: Marker If you are not sure what to do next, there are currently 5 stabilized victims that need transport.

Information Sharing

Information Sharing Strategy Sequence If our teammates do not yet know, you can help the team by telling about the C8 shortcut.

Information Sharing

Blue, there is a victim nearby that can be moved to the evacuation point. It is visible on your map.

Lessons Learned From Human-Agent Hybrid Search and Rescue 181

RESULTS

These exploratory analyses examine only the intervention types listed in Table 1, Information Sharing, Mission Status Update, and Motivation, and how they may impact mission outcomes. The ASI interventions assigned these codes reflect an ASI’s engagement in supporting team processes, including monitoring progress towards goals, goal specification, systems monitoring, team monitoring, as well as motivation and confidence building (Mathieu et al., 2020). It is important to keep in mind that these analyses are agnostic to the agent with which a team was paired, and instead look at the relation- ships between mission outcomes and three different intervention types. These include interventions where the ASI provided the team with motivation, such as by directly cheering the team on or through praise for specific actions taken, which is related to motivation and confidence building in the Mathieu et al. framework. Additionally, instances where the ASI made interventions related to information sharing, which included updating teammates on the environmental changes, such as the locations of passages and victims, and is related to systems monitoring and team monitoring in the taxonomy of team processes (Marks et al., 2001). Finally, Mission Status Update interventions included instances of the ASI providing the team updates on the state of mis- sion goals and objectives, such as reporting the number of victims rescued, and when to transition priorities or strategies to more effectively complete mission objectives over the course of the mission – relating to transition pro- cesses, as well as monitoring progress towards goals and systems monitoring (Mathieu et al., 2001).

Note that these analyses are at the intervention-type level rather than at the individual-ASI level, as ASI delivered interventions at varying frequency and type even within one ASI across the teams they participated with. Addi- tionally, we are not interested in examining the learning effects over the course of two missions, so our analyses explore the impact of interventions on Mission 2 outcomes and metrics.

Count of Interventions The overall counts of a given intervention category, Information Sharing, Mission Status Update, and Motivation, correlated with Total Intervention Count is shown in Table 3. Notably, we found that the count of Motivation interventions correlated significantly Total Intervention Count (Pearson’s r = 0.905, p < 0.001), meaning that for teams that received a large number of interventions a larger portion of those interventions were of the Motiva- tion category. This is important to note to contextualize the analyses below because there are differences between the intervention patterns of the ASIST agents that are not captured in this analysis. Here, we are interested not in the individual agent impacts but instead in the impacts of the intervention content types regardless of the agents with which a given team worked.

Type of Interventions and Mission Outcomes First, we focus on interventions related to teamwork processes in the action processes and transition processes, specifically the Information Sharing and

Table 3. Correlations between total intervention count and count of intervention type.

Table available in the original PDF paper.

Total Intervention Count Intervention Category Pearson’s r p −0.107 0.297 Intervention Count: Total Intervention Count: Information Sharing

−0.092 0.375

Intervention Count: Total Intervention Count: Mission Status Update

0.905*** <.001

Intervention Count: Total Intervention Count: Motivation Note: all tests one-tailed, for positive correlation. *p<.05, **p<.01, ***p<.001

Mission Status Update interventions, which differ from Motivation interven- tions as they occurred with far less frequency and were primarily task and utility focused, rather than encouraging and motivational. Results of these analyses (see Table 4), indicate that Mission Score was significantly corre- lated with both Information Sharing (Pearson’s r = 0.322, p <.001) and Mission Status Update (Pearson’s r = 0.252, p = .007). With regards to vic- tim extractions, Information Sharing was positively correlated with Critical Victim Extractions and Mission Status Updates was positive correlated with Regular Victim Extractions.

Table 4. Correlations between information sharing and mission status update intervention types and mission outcomes.

Table available in the original PDF paper.

Intervention Category Mission Outcome Pearson’s r p 0.322*** <.001 Intervention Percent: Information Sharing

Metric: Mission Score

0.310** .003

Intervention Percent: Information Sharing

Metric: Critical Victim Extractions

0.069 .521

Intervention Percent: Information Sharing

Metric: Regular Victim Extractions

0.252** .007

Intervention Percent: Mission Status Update

Metric: Mission Score

0.173 .105

Intervention Percent: Mission Status Update

Metric: Critical Victim Extractions

0.220* .019

Intervention Percent: Mission Status Update

Metric: Regular Victim Extractions Note: all tests one-tailed, for positive correlation. *p<.05, **p<.01, ***p<.001

Looking at the remaining intervention type, Motivation (see Table 5), results indicated that Motivation interventions were significantly negatively correlated with Mission Score (Pearson’s r = −0.241, p = .010) and Critical Victim Extractions (Pearson’s r = −0.261, p = .007).

DISCUSSION

We explored the impact of interventions generated by Artificial Social Intelli- gence (ASI) agents that were given to teams during a complex gamified USAR tasks through the analysis of a subset of interventions in a research program

Lessons Learned From Human-Agent Hybrid Search and Rescue 183

Table 5. Correlations between motivation intervention types and mission outcomes.

Table available in the original PDF paper.

Intervention Category Mission Outcome Pearson’s r p −0.241* .010 Intervention Percent: Motivation

Metric: Mission Score

−0.261** .007

Intervention Percent: Motivation

Metric: Critical Victim Extractions

−0.038 .362

Intervention Percent: Motivation

Metric: Regular Victim Extractions Note: all tests one-tailed, for negative correlation. *p<.05, **p<.01, ***p<.001

on the development of artificial social intelligence. Our analyses focus on cer- tain intervention types that are related to teamwork processes, and include the intervention categories Information Sharing, Mission Status Update, and Motivation. Results indicate that ASI supporting teamwork through provid- ing task progress updates, tracking environmental conditions, identifying and prioritizing objectives, and providing backup behaviors were helpful to team performance on the complex, interdependent task – Information Sharing and Mission Status Update interventions positively impacted mission outcomes including overall mission score as well as critical and regular victim rescues. Further, the Information Sharing and Mission Status Update interventions were generated much less frequently than Motivation interventions – though it should be noted that the frequency of interventions varied across the six ASI, and across teams within a single ASI. The amount of Motivation type interventions generated, on the other hand, was significantly positively cor- related with the total intervention count, and comprised just under a third of all interventions given. However, results demonstrated that Motivation inter- ventions negatively impacted team performance on mission score and critical victim rescues. Though the ASI interventions related to Motivation were gen- erated at a high rate, they did not improve the performance on this complex, interdependent task.

These findings provide preliminary support that ASI agents can benefit team performance through supporting teamwork processes with interven- tions Though we are not able to directly examine this, it may be that the number of motivation interventions given interfered with team processes, and that they found it distracting, irrelevant, or the rate of interventions given disrupted other team processes. There were some trials where the ASI gener- ated many redundant motivation messages within a single trial (e.g., in one trial the “Keep going! Do your best teamwork”intervention was delivered 38 times over the course of 15 minutes, and “Great job adapting your behavior!” was additionally delivered 11 times in this same trial). It may be that there are specific motivation interventions that are primarily responsible for driv- ing these effects, or it may be that motivation-related interventions in general are less helpful in a complex, interdependent task. The intervention types of Information Sharing and Mission Status Update typically contained content that was more objective and utility focused, helping the team maintain aware- ness of task status and prompting team communication of task-relevant infor- mation (e.g., “If our teammates do not yet know, you can help the team by

telling about the C8 shortcut.” and “If you are not sure what to do next, there are currently 5 stabilized victims that need transport.”). Though occurring at a far less rate, these interventions seemed to help support teamwork processes directly by participating in the action and transition processes (Marks et al., 2001) rather than the interpersonal processes. We suggest that future research investigating the implementation of agent architectures that are capable of Artificial Theory of Mind for supporting human-agent teaming should exam- ine how and when to use these intervention types based on the team they are working with, and how a team is responding to the use of certain intervention types. This would involve behavior analysis of the team mem- bers before and after interventions are delivered to determine the response of team members with consideration for the context of the situation/task. Additionally, future research should investigate conditions related to individ- ual differences among team members that impact intervention receptions in teams – for example, teams with reduced confidence, based on either self- report or ASI observation and mental state attribution, could benefit from Motivation interventions. Finally, the interventions, or interactions that took place between agents and humans, were one directional, and future research should examine how agents may learn from bidirectional interactions with human teammates to tailor interventions dynamically and across team members.

ACKNOWLEDGMENT

This research was sponsored by the Defense Advanced Research Projects Agency (DARPA) and the Army Research Office and was accomplished under Grant Number W911NF-20-1-0008. The views and conclusions contained in this document are those of the authors and should not be interpreted as repre- senting the official policies, either expressed or implied, of DARPA, the Army Research Office, or the U.S. Government. The Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.

REFERENCES

ASIST (2023). Artificial Social Intelligence for Successful Teams. DARPA. https://ww w.darpa.mil/program/artificial-social-intelligence-for-successful-teams Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I. (2001). The “Read-

ing the Mind in the Eyes” Test revised version: a study with normal adults, and adults with Asperger syndrome or high-functioning autism. The Journal of Child Psychology and Psychiatry and Allied Disciplines, 42(2), 241–251. Bendell, R., Williams, J. Fiore, S. M., & Jentsch, F. (2021). Supporting Social Interactions in Human-AI Teammates from Sparse Data. Proceedings of the Human Factors and Ergonomics Society Annual Meeting (665-669). Los Angeles, CA: SAGE Publications. Chen, B., Vondrick, C., and Lipson, H. (2021b). Visual behavior modelling for

robotic theory of mind. Sci. Rep. 11, 1–14. doi: 10.1038/s41598-020-77918-x.

Lessons Learned From Human-Agent Hybrid Search and Rescue 185

Fiore, S. M., Wiltshire, T. J., Lobato, E. J., Jentsch, F. G., Huang, W. H., and Axelrod, B. (2013). Toward understanding social cues and signals in human-robot inter- action: effects of robot gaze and proxemic behavior. Front. Psychol. 4, 859. doi: 10.3389/fpsyg.2013.00859. Hegarty, M., Richardson, A. E., Montello, D. R., Lovelace, K., & Subbiah, I. (2002).

Development of a self-report measure of environmental spatial ability. Intelligence, 30(5), 425–447. Huang, L., Freeman, J., Cooke, N., Colonna-Romano, J., Wood., M., Buchanan, V., & Caufman, S. J. (2022a). Exercises for Artificial Social Intelligence in Minecraft Search and Rescue for Teams. OSF. https://osf.io/project/jwyvf/files/osfstorage/ 62549a7c28f940000fa2455e Huang, L., Freeman, J., Cooke, N., Colonna-Romano, J., Wood, M., Buchanan, V.,

& Caufman, S. (2022b). Artificial Social Intelligence for Successful Teams (ASIST) Study 3 (ASU Library Research Data Repository; V3) [Data set, study procedure and materials]. ASU Library Research Data Repository. https://doi.org/10.48349/ ASU/QDQ4MH Kaplan, A., and Haenlein, M. (2019). Siri, Siri, in my hand: who’s the fairest in the land? On the interpretations, illustrations, and implications of artificial intelligence. Bus. Horizons 62, 15–25. doi: 10.1016/j.bushor.2018.08.004. Li, H., Le, L., Chis, M., Zheng, K., Hughes, D., Lewis, M., & Sycara, K. (2023,

January). Sequential theory of mind modeling in team search and rescue tasks. In Computational Theory of Mind for Human-Machine Teams: First International Symposium, ToM for Teams 2021, Virtual Event, November 4–6, 2021, Revised Selected Papers (pp. 158–172). Cham: Springer Nature Switzerland. Marks, M. A., Mathieu, J. E., & Zaccaro, S. J. (2001). A temporally based framework and taxonomy of team processes. Academy of management review, 26(3), 356–376. Mathieu, J. E., Luciano, M. M., D’Innocenzo, L., Klock, E. A., & LePine, J.

A. (2020). The development and construct validity of a team processes survey measure. Organizational Research Methods, 23(3), 399–431. Wang, Y., Gurney, N., Zhou, J., Pynadath, D. V., & Ustun, V. (2023, January). Route Optimization in Service of a Search and Rescue Artificial Social Intelligence Agent. In Computational Theory of Mind for Human-Machine Teams: First International Symposium, ToM for Teams 2021, Virtual Event, November 4–6, 2021, Revised Selected Papers (pp. 220–228). Cham: Springer Nature Switzerland. Williams, J., Bendell, R., Fiore, S., Jentsch, F. (2023). The Role of Artificial The-

ory of Mind in Supporting Human-Agent Teaming Interactions. In: Julia Wright and Daniel Barber (eds) Human Factors and Simulation. AHFE (2023) Inter- national Conference. AHFE Open Access, vol 83. AHFE International, USA. http://doi.org/10.54941/ahfe1003561 Williams, J., Fiore, S. M., & Jentsch, F. (2022). Supporting artificial social intelligence with theory of mind. Frontiers in artificial intelligence, 5, 750763.

Cover of the proceedings volume Artificial Intelligence, Social Computing and Wearable Technologies

Explore the complete digital volume

Artificial Intelligence, Social Computing and Wearable Technologies

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-89-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →