Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Are Structured Analytic Techniques (SATs) the Missing Component in Cognitive Warfare? The Future of ISR Military Operations

711 th Human Performance Wing, Airman Systems Directorate, WPAFB, OH 45433, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1005374

Abstract

U.S. Intelligence, Surveillance, and Reconnaissance (ISR) operators collect critical information with respect to our adversaries ground movement patterns, weapon capabilities, and strategic framework to support future military direction and enhance Joint All-Domain Command and Control (JADC2) situational awareness. However, the ability for ISR operators to (1) detect and identify essential elements of information (EEI) within vague or ill-defined content and (2) fuse and disseminate collected information across the JADC2 enterprise is extremely challenging. To combat these issues, ISR tools have been developed to assist and facilitate the comprehension of collected intelligence in an effort to augment and enhance decision-making efficacy. More recently, Structured Analytic Techniques (SATs) has been an area of interest within military applications in an effort to support processing, exploitation, and dissemination (PED) of collected intelligence. SATs comprise of a systematic process that enhances critical thinking and logical reasoning by reducing cognitive biases. Previous research has discovered that implementing SATs when providing vague/ill-defined content has been shown to improve decision-making performance. Therefore, the objective of this study was to add to the body of knowledge by evaluating the effectiveness of Sphinx, an ISR decision-support tool focused on SAT methodology, when providing vague/ill-defined content. Four groups of 10 active-duty military operators (N=40) were randomly assigned to one of two analytic techniques (Sphinx or Control) and provided textual content in either incremental or complete sections with the objective of correctly detecting embedded EEIs resulting in improved performance accuracy. The findings discovered that providing active-duty military operators with Sphinx coupled with an incremental workflow methodology resulted in the highest solution accuracy compared to all other conditions (i.e., 6 of 10 operators identified the correct solution – 60%). Moreover, operators that were provided Sphinx detected significantly more EEIs compared to the Control condition (p<0.01). This discovery provides new evidence that equipping active-duty military operators with Sphinx, an ISR decision-support tool, can enhance the detection of EEIs resulting in improved performance. More specifically, Sphinx enabled the operators to better understand vague content leading to greater detection of EEIs. The processing, exploitation, and dissemination (PED) of EEIs within Sphinx can greatly benefit the JADC2 enterprise by enhancing situational awareness and future military direction and recommendations.

INTRODUCTION

Intelligence, Surveillance, and Reconnaissance (ISR) operators are continu- ally tasked to process, exploit, and disseminate (PED) collected intelligence on our adversaries ground movement patterns, weapon capabilities, and troop formations in near-real time to support the strat-to-task framework for future military direction (see Figure 1). Intelligence data can be collected from a multitude of sources such as human intelligence (HUMINT), signals intelligence (SIGINT), imagery intelligence (IMINT), geospatial intelligence (GEOINT), measurement and signature intelligence (SIGINT), and open- source intelligence (OSINT) (Kami´nski, 2019). Nevertheless, regardless of the collection methodology, all the intelligence has a single commonality – essential elements of information (EEI). EEIs are embedded, mission-relevant information that provides underlying evidence to support a high-level ques- tion or Priority Intelligence Requirement (PIR) based on commanders’ intent (Frame et al., 2023). If EEIs are correctly identified, they can provide rea- soning and justification to inform strategic planners and military leadership

© 2024. Published by AHFE Open Access. All rights reserved. 268

Are Structured Analytic Techniques (SATs) the Missing Component in Cognitive Warfare? 269

on future planning guidance (Nelson et al., 2023). However the question still remains, is there an optimal methodology to augment and enhance the under- standing and interpretation of vague collected intelligence resulting in higher efficacy for EEI detection.

Figure 1: Processing, exploitation, and dissemination (PED) phase of the intelligence cycle across INTs and multi-INTs with a focus on how to detect subtle essential elements of information (EEI) (Nelson et al., 2023).
Original PDF Figure 1:

Within the literature, Structured Analytic Techniques (SATs) has been shown to augment and enhance critical thinking and logical reasoning when confronted with vague textual content by reducing pre-existing cognitive biases resulting in improved performance (Whitesmith, 2019). Moreover, SATs has had a positive influence on performance outcomes when processing, exploiting, and disseminating information from multiple users also known as crowdsourcing (Estellés-Arolas, 2012). With a capability gap identified with the processing and exploitation of collected ISR intelligence, SATs could be the missing component. As a result, ISR subject matter experts (SMEs) from Wright-Patterson AFB developed Sphinx. Sphinx is an ISR collaborative workspace tool focusing on structured analytic techniques to systematically support the breakdown of high-level PIRs into detectable EEIs in an effort to enhance future prediction and recommendation accuracy. Therefore, the objective is study was to evaluate the effectiveness of Sphinx on military oper- ators performance when providing an ambiguous narrative in complete or incremental sections.

METHODS AND MATERIALS Study Participants

The study was approved by the Air Force Research Laboratory Institutional Review Board (AFRL IRB) on human participants to evaluate the efficacy of Sphinx, an ISR decision-support tool focused on SAT methodology, when providing an ambiguous narrative. Four groups of 10 active-duty military operators (N = 40) were randomly assigned to one of two analytic techniques (Sphinx or Control) and provided textual content in either incremental or

270 Nelson

complete sections. The operators objective was to correctly identify embed- ded EEIs within the content and provide a single solution for the problem statement.

TASK Before evaluating the effectiveness of Sphinx in a real-world ISR environment that could have significant impact on our military posture, our objective was to evaluate the analytic approach and performance outcomes with an unclas- sified scenario. Therefore, a task was adapted from an online mystery-solving game ‘The Haunted Portrait’ (5minutemystery.com). The task provides an opportunity for the active-duty military personnel to decipher subtle EEIs embedded within the content. The detection of the EEIs can provide under- lying reasoning and justification for their assessment. Within the content, there are 4 embedded EEIs that when detected, can provide valuable informa- tion leading to the correct solution. The task was selected and supported by ISR subject matter experts (SMEs) as being similar to real-world ISR mission environments that require processing, exploiting, and disseminated (PED) critical intelligence regarding problem solving and future decision-making recommendations.

PROCEDURES The active-duty military operators were randomly assigned to either Sphinx or a Control group and provided the same content. If an operator was assigned to Sphinx, they were required to read through the content and develop the high-level question (PIR), identify the subtle clues within the text (EEIs), and provide justification and reasoning (Indicators and Observables). Following the implementation of their inputs into Sphinx, the operators would provide a single solution. If an operator was assigned to Control, they were required to read through the content and provide a single solution with justification and reasoning.

Moreover, each SAT was coupled within a workflow methodology (i.e., all at once or incremental sections). If the operators were provided all the content at once, they would read through the information and follow the structured approach assigned. If the operators were provided incremental sections, they would read through the first section provided and follow the structured analytic approach. This process was repeated for each of the five sections. The purpose of the experimental design was to determine if pro- viding Sphinx, an ISR SAT capability, with incremental or complete sections of information would enhance the detection of EEIs resulting in enhanced situational awareness and improved solution accuracy (see Figure 2).

RESULTS

As shown in Table 1, there was a statistically significant main effect between SAT and information workflow methodology for EEI detection (F3,156=11.05, p<0.01). As a result, an ANOVA was conducted compar- ing each SAT and information workflow methodology for EEI detection. As shown in Table 2, there was a statistically significant difference between

Are Structured Analytic Techniques (SATs) the Missing Component in Cognitive Warfare? 271

ControlAll and SphinxAll (F1,78=15.24, p<0.01) and ControlAll and SphinxInc (F1,78=22.56, p<0.01). As well, there was a statistically significant difference between ControlInc and SphinxAll (F1,78=10.65, p<0.01) and ControlInc and SphinxInc (F1,78=17.19, p<0.01). SphinxInc correctly detected 30 of the 40 EEIs (75%), SphinxAll correctly detected 27 of the 40 EEIs (68%), ControlInc correctly detected 13 of the 40 EEIs (33%), and ControlAll correctly detected 11 of the 40 EEIs (28%) as shown in Figure 3.

Figure 2: Textual content provided to the participants and how that information can be processed, exploited, and disseminated (PED) based on the experimental condition to enhance detection of EEIs and performance accuracy.
Original PDF Figure 2:
Table 1. Analysis of variance (ANOVA) depicting the main effect of structured analytic techniques (SATs) and information workflow methodology for EEI detection.

Table available in the original PDF paper.

Metric Source df SS MS F p Between-Conditions 3 6.97 2.32 11.05 <0.01 EEI Detection Within-Conditions 156 33.02 Total 159

Statistical Significance at alpha level of 0.05

Table 2. Analysis of variance (ANOVA) comparing each structured analytic techniques (SATs) and information workflow methodology for EEI detection.

Table available in the original PDF paper.

Approach Source df SS MS F p ControlAll Between-Conditions 1 0.05 0.05 0.24 0.63 vs Within-Conditions 78 16.75 0.21 ControlInc Total 79 16.80 ControlAll Between-Conditions 1 3.20 3.20 15.24 <0.01 vs Within-Conditions 78 16.75 0.21 SphinxAll Total 79 19.95 ControlAll Between-Conditions 1 4.51 4.51 22.56 <0.01 vs Within-Conditions 78 15.48 0.20 SphinxInc Total 79 19.99

(Continued)

272 Nelson

Table 2. Continued Approach Source df SS MS F p

Table available in the original PDF paper.

ControlInc Between-Conditions 1 2.45 2.45 10.65 <0.01 vs Within-Conditions 78 17.55 0.23 SphinxAll Total 79 20.00 ControlInc Between-Conditions 1 3.61 3.61 17.19 <0.01 vs Within-Conditions 78 16.28 0.21 SphinxInc Total 79 19.89 SphinxAll Between-Conditions 1 0.11 0.11 0.52 0.46 vs Within-Conditions 78 16.28 0.21 SphinxInc Total 79 16.39 Statistical Significance at alpha level of 0.05

Figure 3: Essential elements of information (EEI) correctly detected by structured analytic technique (SAT) and information workflow methodology.
Original PDF Figure 3:

With respect to solution accuracy, there was not a statistically significant difference between SAT and information workflow methodology. However, SphinxInc provided the highest solution accuracy compared to all other conditions (i.e., 6 out of 10 operators identified the correct solution – 60%).

Are Structured Analytic Techniques (SATs) the Missing Component in Cognitive Warfare? 273

CONCLUSION

ISR operators have a daunting task of processing, exploiting, and dissemi- nating (PED) collected intelligence in near-real time in order to provide an optimal recommendation regarding future military direction. The findings from this study have shown that incorporating Sphinx, an ISR SAT capa- bility, can enhance the detection of critical information resulting in better performance accuracy. Future research needs to be conducted to determine how effective Sphinx may be pertaining to real-world military intel. How- ever, we are progressing in the right direction and developing appropriate tools based upon structured analytic techniques (SATs) that will support our warfighters and ensure we maintain military superiority.

ACKNOWLEDGMENT

The views, opinions, and/or findings contained in this paper are those of the author and should not be interpreted as representing the official views or policies, either expressed or implied, of the Air Force, Air Force Research Laboratory or the Department of Defense.

REFERENCES

Capitan, B., 2008. Assuring mission success: Systems security engineering and assurance. IEEE: Military Communication Conference, San Diego, CA, 16–19 November 2008. Estellé s-Arolas, E., Goná lez-Ladrón-de-Guevara, F., 2012. Towards an integrated

crowdsourcing definition. Journal of Information Science, 38(2), 189–200. Gioia, D., Thomas, J., 1996. Identify, image, and issue interpretation: Sensemaking during strategic chance in academia. Administrative Science Quarterly, 370–403. Kami´n ski, M. A., 2019. Intelligence sources in the process of collection of informa-

tion by the US Intelligence Community. Security Dimensions: International and National Studies, 32, 82–105. Frame, M. E., Acker-Mills, B., Maresca, A., Patterson, R., Curtis, E., Buccello-Stout, R., Nelson, J., 2023. Evaluation of a decision support system using Bayesian network modelling in an applied multi-INT surveillance environment. Military Psychology, 10, 1–13. Nelson, J., Heggedahl, T., Frame, M., Maresca, A., Schlessman, B., 2023. Fighting

the next war with Sphinx: An Intelligence, Surveillance, and Reconnaissance (ISR) collaborative tool to support Joint All-Domain Command and Control (JADC2) operations. Journal of DoD Research and Engineering, C4, 65–77. Nelson, J., Maresca, A., Schlessman, B., 2024. Can structured analytic techniques enhance intelligence, surveillance, and reconnaissance (ISR) assessments when providing incremental information. IEEE Intelligent Human Systems Integration (IEEE IHSI), 69. Whitesmith, M., 2019. The efficacy of ACH in mitigating serial position effects and

confirmation bias in an intelligence analysis scenario. Intelligence and National Security, 34(2), 225–242.

Cover of the proceedings volume Human Factors and Systems Interaction

Explore the complete digital volume

Human Factors and Systems Interaction

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-30-4

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →