Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Does Military Experience Influence Intel Collection Efficacy when Providing Chatter Locations on a Geographical Map

1 711 th Human Performance Wing, Airman Systems Directorate, WPAFB OH 45433, USA ↗
2 Parallax Advanced Research, Beavercreek, OH 45431, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1006283

Abstract

Future military planning relies heavily on the information collected from Intelligence, Surveillance, and Reconnaissance (ISR) operations to support data-driven decision-making. In particular, ISR collections that utilize imagery intelligence (IMINT) can detect, track, and target our adversaries ground movement behaviors and headquarter locations in near-real time. However, understanding when and why IMINT collections should be conducted is a challenging problem intel analysts are facing. To combat this issue, the 711th Human Performance Wing at Wright-Patterson Air Force Base developed Intrage. Intrage is a strategic decision-making game with the premise of accelerating the understanding of ISR operations. Methods: The study consists of two groups, 25 military participants and 25 non-military participants from WPAFB. Participants were provided with an overview of Intrage and requested to complete two Phases of the game. In Phase I, participants were provided the Intrage map with chatter locations and requested to conduct four intel collections. Following Phase I, participants were informed that their collections were inconclusive. In Phase II, participants were provided the Intrage map with the same chatter locations and requested to conduct two new intel collections. The objective was to determine if a correlation exists between military and non-military participants regarding intel collection efficacy when providing chatter locations on the fictional map of Intrage. Results: An analysis of variance was performed depicting conducted collections when the collection encompassed four or less chatter locations, five to seven chatter locations, and eight or more chatter locations. There was not a statistically significant difference detected between groups when conducted collections consisted of four or less chatter locations. However, there was a statistically significant difference between groups when conducted collections consisted of five to seven chatter locations (p=0.02). Military participants conducted significantly less intel collections compared to non-military participants. In addition, there was a statistically significant difference between groups when conducted collections consisted of eight or more chatter locations (p=0.03). Military participants conducted significantly more intel collections compared to non-military participants. Moreover, in Phase II there was not a significant difference between groups with respect to conducted collections and provided chatter locations. Conclusion: The findings provide underlying evidence that military experience does influence intel collection efficacy when provided chatter locations on a geographical map. Nevertheless, as both military and non-military participants engaged in additional phases of Intrage, a learning effect was observed resulting in similar performance metrics.

© 2025. Published by AHFE Open Access. All rights reserved. 78

Does Military Experience Influence Intel Collection Efficacy 79

INTRODUCTION

Military leadership relies heavily on Intelligence, Surveillance, and Reconnaissance (ISR) operations to support future military direction and guidance. If intel analysts improperly process, exploit, or disseminate (PED) collected intelligence, it could hinder our warfighters capabilities in the battlefield and reduce military superiority (Nelson et al., 2023). Moreover, collected intelligence can be derived across multiple disciplines including geospatial intelligence (GEOINT), measurement and signature intelligence (MASINT), signals intelligence (SIGINT), and imagery intelligence (IMINT) (Clark, 2013). Although each discipline can provide valuable insight on our adversaries’ military posture, imagery intelligence (IMINT) can detect, track, and target high-value targets and headquarter locations in near-real time. Yet, understanding when and why IMINT collections should be conducted is a challenging problem our intel analysts are currently facing.

To combat this issue, the 711th Human Performance Wing at WPAFB has developed Intrage with the support of ISR subject matter experts (SMEs).

Intrage is a two team strategic decision-making game consisting of a fictional map divided into three geographical regions. The northern region of the map is assigned to team one and consists of Regions A, B, and C. The southern region of the map is assigned to team two and consists of Regions E, F, and G. In the middle of the map is neutral territory controlled by allies. To begin the game, each team will assign their headquarters to a single hexagon within one of their regions. After the headquarters has been placed, each team will conduct intel collections to gain insight on their oppositions ground defense and headquarter locations. In this version of the gameplay, the participants focused on IMINT collections. To conduct an IMINT collection, the participant was requested to select a single hexagon within their adversary’s territory. The collection will capture the single hexagon where the conducted collection was placed and the initial surrounding hexagons–honeycomb layout (see Figure 1). The premise of Intrage is to enhance and accelerate the understanding of ISR operations.

Previous literature has discovered that military experience can enhance performance metrics regarding decision-making (Knighton, 2004), logical reasoning (Vrijkotte et al., 2016), and working memory (Jha et al., 2010). Therefore, we are interested in determining if a correlation exists between military and non-military experience and intel collection efficacy when providing chatter locations on the geographical map of Intrage. As well, with Intrage being developed as a training mechanism to accelerate and enhance the understanding of ISR operations, will engaging in multiple phases of Intrage result in similar performance metrics between military and non-military participants.

METHODS AND MATERIALS Study Participants

The study protocol was approved by the U.S. Air Force Research Laboratory (AFRL) Institutional Review Board (IRB). The objective of this effort was

to determine if any military experience influences intel collection efficacy when providing military and non-military participants chatter locations on the fictional map of Intrage. Two groups of 25 participants completed the research study (Group 1–25 military participants / Group 2–25 non-military participants).

Participants were excluded from the study if they did not meet the following criteria: must be 18 years old or older, speak fluent English, retain basic computer skills, and be located within the U.S. All participants completed the task online via a Qualtrics survey link.

Procedures All participants were provided with an overview of the study objectives including the Intrage map and areas of interest (i.e., chatter locations). Participants were then requested to conduct 4 intel collections in the southern region of the map (i.e., Region E, F, or G) to gather additional insight into the chatter locations. Each collection was placed within a single hexagon and collected information on that hexagon and the nearest surrounding hexagons (i.e., honeycomb design). Following the initial collections, participants were informed that the collections were inconclusive at providing additional information and the participants were instructed to conduct 2 new intel collections. Again, these collections were placed within a single hexagon and collected information on that hexagon and nearest surrounding hexagons. Following the completion of the task, each participant was provided a demographic questionnaire to capture age, gender, and education for post- hoc analysis. In addition, each participant was provided with the need for cognition (NFC) survey to assess the individual’s engagement and enjoyment in effortful cognitive activities (Nelson et al., 2025; Petty et al., 2009).

Hypothesis Within the literature, it has been discovered that military experience influences decision-making assessments and performance metrics (Nelson et al., 2024; Mangos & Arnold, 2008). Based on this information, three hypotheses were developed for each phase of the task. In the first phase of the study where participants were instructed to conduct 4 intel collections based on provided chatter locations we expect that (1) non-military participants would display greater density of intel collections within a specific region (i.e., Region E, F, or G) compared to military participants (2) non-military participants intel collections would encompass fewer chatter locations within a specific region (i.e., Region E, F, or G) compared to military participants (3) overall non-military participants intel collections will encompass fewer chatter locations compared to military participants. In the second phase of the study where participants were informed that the collections were inconclusive and instructed to conduct 2 new intel collections, we expect that (1) non- military participants would display greater density of intel collections within a specific region (i.e., Region E, F, or G) compared to military participants (2) non-military participants intel collections would encompass fewer chatter locations within a specific region (i.e., Region E, F, or G) compared to

Does Military Experience Influence Intel Collection Efficacy 81

military participants (3) overall non-military participants intel collections will encompass fewer chatter locations compared to military participants.

Data Analysis Data analysis was performed using R Statistical Analysis Software (R version 4.1.2.). R is an open-source programming language with downloadable packages from the Comprehensive R Archive Network (CRAN) repository. R can perform statistical computation, data modelling, and data representation on ingestible datasets. To test our hypotheses, an analysis of variance (ANOVA) was conducted comparing conducted intel collections with respect to geographical locations (i.e., Region E, F, and G) and provided chatter locations.

RESULTS

Phase I–Four Conducted Intel Collections To begin, we will discuss the results from the first phase of the study where participants were instructed to conduct 4 intel collections on the southern region of the Intrage map. With respect to the first hypothesis that non- military participants would display greater density of intel collections within a specific region (i.e., Region E, F, or G) compared to military participants, it was discovered that there was not a statistically significant difference detected for Region E between groups (p = 0.14). Military participants conducted forty-two of one hundred collections in Region E compared to thirty-two of one hundred for non-military participants (42% for military participants / 32% for non-military participants). In addition, there was not a statistically significant difference detected for Region G between groups (p = 0.65). Military participants conducted thirty-three of one hundred collections in Region G compared to thirty of one hundred for non- military participants (33% for military participants / 30% for non-military participants). However, there was a statistically significant difference detected for Region F between groups (p = 0.04). Military participants conducted twenty-five of one hundred collections in Region F compared to thirty-eight of one hundred for non-military participants (25% for military participants / 38% for non-military participants) (see Table 1 and Figure 1).

Table 1: Analysis of variance (ANOVA) comparing regional intel collection locations and groups (military vs. non-military participants).

Table available in the original PDF paper.

Source df SS MS F p Region E Between-Conditions 1 0.50 0.50 2.17 0.14 Within-Conditions 198 46.12 0.23 Total 199 46.62 Region F Between-Conditions 1 0.85 0.85 4.05 0.04 Within-Conditions 198 42.31 0.21 Total 199 43.16

Continued

Table 1: Continued Source df SS MS F p

Table available in the original PDF paper.

Region G Between-Conditions 1 0.05 0.05 0.23 0.65 Within-Conditions 198 43.11 0.22 Total 199 43.16 Statistical Significance at alpha level of 0.05.

Figure 1: Geographical representation for regional conducted intel collections between groups.
Original PDF Figure 1:

For the second hypothesis that non-military participants intel collections would encompass fewer chatter locations within a specific region (i.e., Region E, F, or G) compared to military participants, it was discovered that there was not a statistically significant difference detected for Region F between groups (p = 0.08). Military participants conducted collections over sixty- nine chatter locations compared to eighty-eight chatter locations for non- military participants. In addition, there was not a statistically significant difference detected for Region G between groups (p = 0.23). Military participants conducted collections over sixty-two chatter locations compared to fifty-one chatter locations for non-military participants. However, there was a statistically significant difference detected for Region E between groups (p = 0.05). Military participants conducted collections over eighty chatter locations compared to sixty-one chatter locations for non-military participants. Overall, military participants conducted collections with greater efficacy capturing two-hundred and eleven chatter locations compared to two-hundred for non-military participants (see Table 2 and Figure 2).

Does Military Experience Influence Intel Collection Efficacy 83

Table 2: Analysis of variance (ANOVA) comparing regional conducted intel collection over chatter locations and groups (military vs. non-military participants).

Table available in the original PDF paper.

Source df SS MS F p Region E Between-Conditions 1 0.80 0.80 3.81 0.05 Within-Conditions 448 96.00 0.21 Total 449 96.80 Region F Between-Conditions 1 0.60 0.60 3.16 0.08 Within-Conditions 598 115.32 0.19 Total 599 115.92 Region G Between-Conditions 1 0.27 0.27 1.42 0.23 Within-Conditions 448 84.36 0.19 Total 449 84.63

Statistical Significance at alpha level of 0.05.

Figure 2: Geographical representation for regional conducted intel collections over chatter locations between groups.
Original PDF Figure 2:

For the third hypothesis that overall non-military participants intel collections would encompass fewer chatter locations compared to military participants, it was discovered that there was not a statistically significant difference detected when intel collections encompassed four or fewer provided chatter locations between groups (p = 1.00). Two of the twenty- five military and non-military participants conducted intel collections over four or fewer provided chatter locations (8% for military participants / 8% for non-military participants). There was a statistically significant difference detected when intel collections encompassed five to seven provided chatter locations between groups (p = 0.02). Two of twenty-five military participants conducted intel collections over five to seven provided chatter locations compared to nine of twenty-five for non-military participants (8% for military participants / 36% for non-military participants). In addition,

there was a statistically significant difference detected when intel collections encompassed eight or more provided chatter locations between groups (p = 0.03). Twenty-one of twenty-five military participants conducted intel collections over eight or more provided chatter locations compared to fourteen of twenty-five for non-military participants (84% for military participants / 56% for non-military participants) (see Table 3 and Figure 3).

Table 3: Analysis of variance (ANOVA) comparing regional conducted intel collection containing chatter locations and groups (military vs. non-military participants).

Table available in the original PDF paper.

Source df SS MS F p 2–4 Between-Conditions 1 0.00 0.00 0.00 1.00 Within-Conditions 48 3.68 0.08 Total 49 3.68 5–7 Between-Conditions 1 0.98 0.98 6.13 0.02 Within-Conditions 48 7.60 0.16 Total 49 8.58 8–10 Between-Conditions 1 0.98 0.98 4.90 0.03 Within-Conditions 48 9.52 0.20 Total 49 10.50

Statistical Significance at alpha level of 0.05.

Figure 3: Geographical representation for conducted intel collections containing chatter locations between groups.
Original PDF Figure 3:

Phase II—Two Conducted Intel Collections Next, we will discuss the results from the second phase of the study where the participants were informed that the collections in Phase I were inconclusive. The participants were then requested to conduct two new intel collections on the southern region of the Intrage map. With respect to the fourth hypothesis that non-military participants would display greater

Does Military Experience Influence Intel Collection Efficacy 85

density of intel collections within a specific region (i.e., Region E, F, or G) compared to non-military participants, it was discovered that there was not a statistically significant difference detected for Region E (p = 0.49), Region F (p = 0.76), and Region G (p = 0.41) between groups. Military participants conducted twenty-one of fifty collections in Region E compared to nineteen of fifty for non-military participants (42% for military participants / 38% for non-military participants). Military participants conducted eighteen of fifty collections in Region F compared to seventeen of fifty for non- military participants (36% for military participants / 34% for non-military participants). Military participants conducted eleven of fifty collections in Region G compared to fourteen of fifty for non-military participants (22% for military participants / 28% for non-military participants) (see Table 4 and Figure 4).

Table 4: Analysis of variance (ANOVA) comparing regional intel collection locations and groups following inconclusive discoveries (military vs. non-military participants).

Table available in the original PDF paper.

Source df SS MS F p Region E Between-Conditions 1 0.08 0.08 0.47 0.49 Within-Conditions 48 7.92 0.17 Total 49 8.00 Region F Between-Conditions 1 0.02 0.02 0.09 0.76 Within-Conditions 48 10.48 0.22 Total 49 10.50 Region G Between-Conditions 1 0.18 0.18 0.69 0.41 Within-Conditions 48 12.32 0.26 Total 49 12.50

Statistical Significance at alpha level of 0.05.

Figure 4: Geographical representation for regional conducted intel collections between groups following inconclusive discoveries.
Original PDF Figure 4:

For the fifth hypothesis that non-military participants intel collections would encompass fewer chatter locations within a specific region (i.e., Region E, F, or G) compared to military participants, it was discovered that there was not a statistically significant difference detected for Region E (p = 0.72), Region F (p = 0.58), and Region G (p = 0.74) between groups. Military participants conducted collections over forty-four chatter locations compared to forty-one for non-military participants for Region E. Military participants conducted collections over fifty-one chatter locations compared to forty-six for non-military participants for Region F. Military participants conducted collections over twenty chatter locations compared to eighteen for non-military participants for Region G. Overall, military participants conducted collections with greater efficacy capturing one-hundred and fifteen chatter locations compared to one hundred and five for non-military participants (see Table 5 and Figure 5).

Table 5: Analysis of variance (ANOVA) comparing regional conducted intel collection over chatter locations and groups following inconclusive discoveries (military vs. non-military participants).

Table available in the original PDF paper.

Source df SS MS F p Region E Between-Conditions 1 0.02 0.02 0.13 0.72 Within-Conditions 448 68.92 0.15 Total 449 68.94 Region F Between-Conditions 1 0.04 0.04 0.29 0.58 Within-Conditions 598 81.28 0.14 Total 599 81.32 Region G Between-Conditions 1 0.01 0.01 0.13 0.74 Within-Conditions 448 34.78 0.08 Total 449 34.79

Statistical Significance at alpha level of 0.05.

Figure 5: Geographical representation for regional conducted intel collections over chatter locations between groups following inconclusive discoveries.
Original PDF Figure 5:

Does Military Experience Influence Intel Collection Efficacy 87

For the sixth hypothesis that overall non-military participants intel collections would encompass fewer chatter locations compared to military participants, it was discovered that there was not a statistically significant difference detected when intel collections encompassed two or fewer provided chatter locations (p = 0.23), three to four provided chatter locations (p = 0.39), and five or more provided chatter locations (p = 1.00). Two of twenty-five military participants conducted intel collections over two or fewer provided chatter locations compared to five of twenty-five for non- military participants (8% for military participants / 20% for non-military participants). Eleven of twenty-five military participants conducted intel collections over three to four provided chatter locations compared to eight of twenty-five for non-military participants (44% for military participants / 32% for non-military participants). Twelve of twenty-five military and non- military participants conducted intel collections over five or more provided chatter locations (48% for military participants / 48% for non-military participants) (see Table 6 and Figure 6).

Table 6: Analysis of variance (ANOVA) comparing regional conducted intel collection containing chatter locations and groups following inconclusive discoveries (military vs. non-military participants).

Table available in the original PDF paper.

Source df SS MS F p 1–2 Between-Conditions 1 0.18 0.18 1.50 0.23 Within-Conditions 48 5.84 0.12 Total 49 6.02 3–4 Between-Conditions 1 0.18 0.18 0.75 0.39 Within-Conditions 48 11.60 0.24 Total 49 11.78 5–6 Between-Conditions 1 0.00 0.00 0.00 1.00 Within-Conditions 48 12.48 0.26 Total 49 12.48

Statistical Significance at alpha level of 0.05.

Figure 6: Geographical representation for conducted intel collections between groups following inconclusive discoveries.
Original PDF Figure 6:

CONCLUSION

Intelligence, Surveillance, and Reconnaissance (ISR) operations are vital in collecting information on our adversaries’ military posture. However, requesting the appropriate intel collection discipline is a difficult challenge that our intel analysts are facing every day. Therefore, the 711th Human Performance Wing at WPAFB developed Intrage to enhance and accelerate basic knowledge on ISR operations. The findings in this study focusing on imagery intelligence (IMINT) were very promising showing that military experience did display a significant correlation with respect to effectively conducting IMINT requests. Moreover, as both groups of participants (military and non-military) played additional phases of the game, a learning curve was observed resulting in similar performance metrics. This finding strengthens the argument that Intrage has the potential to enhance the understanding of ISR operations.

It is important to note that in the current study, no additional information was provided to the participant to distinguish how regions may differ from one another with respect to threat or adversary activity. The only information that was provided was chatter locations across Regions E, F, and G. Future research will be conducted providing threat and adversary activity information to determine why and how intel collections are conducted and to elaborate on the findings from this study. In addition, future research will be conducted incorporating additional intel discipline options coupled with complex scenario events.

ACKNOWLEDGMENT

The views, opinions, and/or findings contained in this paper are those of the author and should not be interpreted as representing the official views or policies, either expressed or implied, of the Air Force, Air Force Research Laboratory or the Department of Defense.

REFERENCES

Clark, R., 2013. Perspectives on intelligence collection. Journal of US Intelligence Collection, 20, 47–52. Jha, A., Stanley, E., Kiyonaga, A., Wong, L., and Gelfand, L., 2010. Examining

the protective effects of mindfulness training on working memory capacity and affective experience. Emotion, 10(1), 54. Knighton, R., 2004. The psychology of risk and its role in military decision-making.

Defence Studies, 4(3), 309–334. Nelson, J., Heggedahl, T., Morgan, J., Johnston, S., and Cotter, J., 2025.

Does military experience influence strategic decision-making with respect to geographical headquarter placement. IEEE Intelligent Human Systems Integration (IEEE IHSI), IHSI 2025, Rome, Italy, Feb 22–24, 2025. Springer. Nelson, J., Heggedahl, T., Frame, M., Maresca, A., and Schlessman, B., 2023. Building the future of intelligence, surveillance, and reconnaissance (ISR) collections: The development and evaluation of a collaborative ISR tool to support intel analysts. Intelligence and Security Informatics: IEEE International Conference on Intelligence and Security Informatics, ISI 2023, Charlotte, NC, USA, Oct 2–3, 2023. Springer.

Does Military Experience Influence Intel Collection Efficacy 89

Nelson, J., Maresca, A., Schlessman, B., Holt, J., Kegley, J., and Boydstrum, A.,

2023. Evaluating the efficacy of structured analytic techniques (SATs) as a support system to enhance decision-making within ISR mission environments. International Conference on Human Factors in Design, Engineering, and Computing, Honolulu, HI, USA, Dec 4–6, 2023. Springer. Mangos, P., and Arnold, R., 2008. Enhancing military training through the application of maximum and typical performance measurement principles. Performance Improvement, 47(3), 29–35. Petty, R., Brinol, P., Loersch, C., and McCaslin, M., 2009. The need for cognition.

Handbook of Individual Differences in Social Behavior, 318-329. The Guilford Press. Vrijkotte, S., Roelands, B., Meeusen, R., and Pattyn, N., 2016. Sustained military operations and cognitive performance. Aerospace Medicine and Human Performance (AMHP), 87(8), 718–727.

Cover of the proceedings volume Human Error, Reliability, Resilience, and Performance

Explore the complete digital volume

Human Error, Reliability, Resilience, and Performance

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-49-6

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →