Linda R. Elliott, Bruce Mortimer, Anna Skinner (2019). Tactile Displays for Soldier Systems: Progress and Issues. In: Advances in Cognitive Engineering and Neuroergonomics. AHFE Open Access, vol. 7. https://doi.org/10.54941/ahfe100211
Multisensory tactile displays have enabled Soldiers to communicate covertly during strenuous movements and to navigate in low visibility conditions, while allowing Soldiers to keep their hands on their weapons and their eyes on their surroundings. The full potential of these multisensory systems to reduce Soldier cognitive load and enhance performance has yet to be determined, but shows great promise, particularly in situations where there is degraded visual acuity, high noise, and/or need for audio silence. Improvements in tactor technology include more distinctive and varied tactile sensations that are expected to allow recognition of a greater range of tactile cues and simultaneous presentation of two types of signals (e.g., navigation and alerts). The current study assessed the operational effectiveness of a tactile display integrated with a gesture recognition glove for automated detection of Soldier hand and arm signals, which were transmitted and displayed as tactile patterns on a haptic feedback vest. This study indicated that the integration of glove-based gesture recognition and a tactile display resulted in faster and more accurately perceived communications than traditional Army hand and arm signals. Given these recent technology developments and their potential, there is a corresponding opportunity for basic and applied research to address issues arising from these multisensory displays.
Human factors studies of Soldier roles have shown significant overloading of the visual and auditory sensory modes in jobs such as Abrams tank commanders and drivers (Mitchell, 2009), ground robot controllers, and unmanned aircraft operators. Dismounted Soldiers consistently experience heavy cognitive and visual workload, particularly during navigation and patrol, and under conditions of high stress and time pressure (Mitchell et al., 2004; Mitchell,
2005; Mitchell & Brennan, 2009a, 2009b; Pomranky & Wojciechowski, 2007). In addition, a review of emerging technologies assessed for Infantry Soldier combat teams during the Army Expeditionary Warrior Experiment (AEWE), included aerial and ground vehicles with sensor arrays, small stationary sensors, more robust communication capabilities, and improved visual capabilities encompassing weapon sights, binoculars, night vision, and targeting aids (Scalsky, Meshesha & Struken, 2009; U.S. Army Evaluation Center, 2013). From this, we see clearly that cognitive task demands on dismounted Soldiers are increasing.
Review articles (Jones & Sarter, 2008; van Erp, 2007) provide comprehensive information with regard to several different tactile and haptic devices, and discuss the characteristics of these devices that affect perception and localization. In addition, a review of the tactile display literature resulted in the identification of 64 studies that compared performance data arising from comparisons of tactile, visual, and combination (tactile and visual) displays (Elliott, Coovert, Redden, 2009) and met the criteria for inclusion in meta-analytic investigations. Results were consistent with Wickens’ multiple resource theory (Wickens, 2008), which predicts that distribution of information across sensory channels can result in better performance when one channel is overloaded. Tactile cues were found to be particularly helpful when added to an existing system to provide additional information, such as alerts, direction cues, and spatial information. Multimodal presentation of both visual and tactile cues that represent the same (i.e., redundant) information, was also significantly associated with better performance. In addition, the investigation demonstrated moderating effects of task, such that effectiveness of a tactile or tactile/visual cueing system depended in part on whether the task was that of simple alerts, direction cueing, spatial orientation, or communications where different tactile patterns indicated different meanings. Results show overwhelming evidence for the effectiveness of tactile cues when added to a task situation to guide and direct attention or to support spatial orientation.
Given the increasing cognitive demands associated with the role of the Soldier and the promising results arising from a growing literature on tactile displays that are also consistent with theories of multitasking workload (Wickens, 2008) and pre-attentive nature of tactile cues (van Erp, 2007) the US Army Research Laboratory, Human Research and Engineering Directorate (ARL/HRED) initiated experiments with tactile displays for Soldier navigation and communication specifically to assess their impact on workload and performance. Several HRED studies have been conducted within the context of Soldier land navigation to investigate effects of tactile cues in context (Elliott, Redden, Krausman, & Carstens, 2005). The studies demonstrated that Soldiers could detect not only single alerts but also patterns of multiple tactors to represent different messages. It is particularly promising that the Soldiers could perceive these patterns during strenuous movements (Merlo, Stafford, Gilson, Hancock, 2006). Redden, Carstens, Turner, Elliott, 2006). Three additional HRED experiments demonstrated the efficacy and suitability of a torso-mounted tactile belt for Soldier navigation (Elliott, van Erp, Redden, & Duistermaat, 2010). Given this series of results from land navigation studies, it is evident that tactile navigation displays can be used in strenuous outdoor environments and can outperform visual displays under conditions of high cognitive and visual workload. In addition, Soldier feedback (e.g., after-action reviews, comments, and structured rating scales) was very positive, indicating that the core advantage of the system was that it was “hands-free, eyes-free, and mind-free.”
The experiments described above establish the potential of tactile systems for supporting Soldier performance while easing workload and gaining high user acceptance. At the same time, Soldiers have provided many suggestions for device design before a system can be practically used in combat. Specifically, the device must be made to be lightweight, comfortable, rugged, easy to use, and easy to maintain. The device must enable reliable communication among Soldiers. Currently, Soldiers use visual hand signals to communicate and coordinate movements and target detection. Tactile systems can build upon these techniques, by enabling commanders to easily and covertly signal Soldiers regarding alerts or movements. To pursue further development of these capabilities, a Small Business Innovative Research (SBIR) topic was developed, funded, and administered as a collaboration between Army Research Laboratory and the Army Research Office. This resulted in two projects that will be described here. One, performed by AnthroTronix, Inc., is focused on integration of the tactile display capability with gesture-based hand and arm signals. The other, performed by Engineering Acoustics is focused on advanced tactor capabilities integrated with GPS navigation. Here we will summarize progress to date.
INTEGRATION OF TACTILE DISPLAY WITH GESTURAL GLOVE Hands-free covert communications
Soldier communications, within and across teams, are essential for Soldier effectiveness. Communications must be rapid, concise, and immediately understood. However, the use of handheld communication devices poses some challenges. While handheld devices such as radios or more recently, smart-phones and smart-tablets, provide critical capabilities, these handheld devices can also distract leader’s visual attention away from the tactical battlefield environment and in some cases, hinder their ability to use their weapons or increase the response time to engage a target when having to transition from the device to the weapon. When the communication is speech-based, there is also the common interference problem associated with noisy environments. Furthermore, such devices may violate noise or light discipline, posing the risk of giving away an individual or team’s location. There will always be situations where the handheld device is best left stowed and there is a need for Soldier communications that are covert and relatively hands-free, allowing immediate access to weapons.
A fundamental form of this kind of communication among Soldiers is the use of hand and arm signals. Dismounted Soldiers in the field often utilize an established set of hand and arm signals in order to communicate with others while maintaining noise discipline (e.g., when approaching an objective) or at times when noise levels exceed what can be heard via voice and radio. Most military personnel are familiar with these signals. Soldier hand and arm signals are documented in sources such as the US Army Field Manual No. 21-60 and U.S. Marine Corps Rifle Squad manual (FMFM 6-5). These commands are often relayed from one team member to the next, reaching team members not within line of sight of the initial team member issuing the command; however, this takes time and requires visual, and sometimes aural, attention in order to receive commands. Therefore, both verbal and visual hand and arm signals are limited in combat situations that are noisy or visually degraded.
COMMAND System
The COMMAND (Communication-based Operational Multi-Modal Automated Navigation Device) system, in development by AnthroTronix, builds upon their previous work for Office of Naval Research, which led to the development of a Haptic Automated Communication System (HACS) prototype that utilizes an instrumented glove for real-time communications based on hand signals. The instrumented glove includes 6 embedded accelerometers, a gyroscope, and a digital compass for automated recognition of standard hand and arm signals, gestures, pointing, and weapon firing. It also includes a torso-mounted accelerometer and digital compass for detecting Soldier location and stance (e.g., upright, prone). Also included is a haptic display vest with 20 tactors for pattern-based communications, and a GPS-enabled handheld computer. Figure 1 shows the tactile display vest. Figure 2 shows the gestural glove with an example pattern. Four different gestures with corresponding tactile patterns were developed for evaluation by Soldiers in the current field study.
Figure 1. Front and back diagrams of placement of embedded tactorsFigure 2. Example of gesture and corresponding tactile pattern
Soldier-based Evaluation A Soldier-based evaluation was conducted with thirty one enlisted Soldiers from Infantry-related occupational specialties. Two routes were used. One had the soldier perform combat-relevant movements such as walking, climbing, and crawling, using an Individual Movement Technique (IMT) course (e.g., obstacles). During this event, each Soldier would walk the course, guided by a data collector and followed by another Soldier who would generate the signals either with traditional hand and arm signals or by using the gestural glove. Each Soldier would experience both the gestural glove signals and the traditional hand and arm signals. Half the signals would be generated when the Soldier was walking, the other half when the Soldier was performing a more strenuous movement. See Figure 3.
Figure 3. IMT course movement
Figure available in the original PDF paper.
In addition, each Soldier experienced the hand and arm signals, and the gestural system, while moving through wooded terrain, while simultaneously searching for visual targets (i.e., small orange flags).
Figure 4. IMT course movement
For detection of signals, the difference between Glove and Hand-arm means were significantly different in the IMT condition (F 1, 30 = 20.13, p < 0.001, ηρ² = 0.40) and also for tactical movement in wooded terrain, (F 1, 30 = 36.25, p = 0.00, ηρ² = 0.547), where detection rates were higher for the glove/tactile system. Figure 5 provides a graph representing the difference pertaining to detection rate for the glove/tactile system vs. traditional hand and arm signals, for the IMT and tactical movement task demands.
Figure 5. Detection rate using the glove/tactile system vs. hand and arm signals, for IMT and tactical movement.
Differences between the systems in time for detection were also significant for IMT maneuvers (F 1, 30 = 214.84, p = 0.00, ηρ² = 0.877) and tactical movement (F 1, 30 = 455.479, p = 0.00, partial eta square ηρ² = 0.938), and can be seen in Figure 6. Error bars represent one standard deviation above and below the mean. Detection of signals using the glove/tactile system was 100% regardless of signal (i.e. tactile pattern), across all task demands.
Figure 6. Time to detect signal using the Glove vs. Hand and arm signals, for IMT and tactical movement.
Differences in accuracy rate were not significant for IMT tasks (F 1, 30 = 3.95, p = 0.056, ηρ² = 0.116) or for tactical movement (F 1, 30 = 0.616, p = 0.439, ηρ² = 0.02). These effects are represented below in Figure 7. It should be noted that the percent correct was only calculated on those that were detected, such that the difference in detection rates is not reflected here. Cognitive Engineering and Neuroergonomics (2019)
Figure 7. Percent correct signal identification using the glove/tactile system and for the hand and arm signals, for IMT and tactical movement.
The detection of flags during the wooded terrain course provided a secondary visual task. Soldiers detected an average of 10.61 flags with the glove/tactile system (standard deviation = 2.70) and 9.71 flags with the hand-arm signals (standard deviation = 2.77). While the mean number was higher with the glove system, the difference did not meet significance criteria, though it did come close (F 1,30 = 3.64, p = 0.07, ηρ² = 0.11). There was considerable variance among the Soldiers with regard to performance of this task.
Soldier feedback was generally positive regarding the comfort of the glove. The glove received a mean rating of 5.50 (7 pt. scale; between Positive and Very Positive) for comfort. At the same time, many suggestions were offered as improvements, many of which have to do with improving form and fit through having a range of different sizes, with different fit between women and men. Other suggestions were to reduce the bulk, increase elasticity, and make the sensors interchangeable with different types of gloves. There was complete agreement that the hand signals were easy to learn and the glove was easy to use. Soldiers also agreed that the glove was a good concept for Soldier operations, and allowed more attention to be spent on surroundings. When asked the type of situations in which the glove concept could be useful, assuming combat-readiness, Soldiers emphasized covert operations that require silence, whether for dismounted patrol or reconnaissance. They also mentioned situations in which a larger number of Soldiers split up into squads that are out of line of sight, and specifically when they have to coordinate on the fly. Night operations were also listed, as well as any situations in which visibility is impaired. The mean overall rating for operational relevance for Soldier missions was 5.91 on a 7pt scale, where 6.00 is “Very likely”.
INTEGRATION OF TWO TACTOR TYPES WITH GPS NAVIGATION
Figure 8 shows a block diagram of the prototype Engineering Acoustics, Inc. (EAI) NavCom system. The NavCom system includes a Soldier worn handheld tablet or smart-phone display, GPS inertial sensor, and a dual-row tactile belt. A separate, remote display is connected to a networked computer (laptop or tablet) that is used for mission planning, task management and mission analysis.
Figure 8. Block diagram for the EAI-NavCom system
Figure available in the original PDF paper.
The NavCom User hardware comprises either a smart phone or tablet (or similar) with integral visual display and touch screen interface, the EAI dual-belt tactor controller and belt array shown in Figure , and a COTS GPS / inertial sensor shown in Figure 9. The chest mounted visual display provides real time, first-person navigation including waypoints, exclusion zones and points of interest together with situational awareness information. The smart phone is connected via a USB hub to a small controller box, battery and the dual row or hybrid tactor belt. The hybrid belt is worn under the Soldier’s tactical vest and provides navigation and situational awareness information through tactile cue symbology. The NavCom system provides Soldier’s with simple access to visual information when it is needed (as shown in Figure 10), but can also be operated with tactile cueing only in situations where the visual modality is being used.
Figure 9. EAI prototype NavCom dual-row tactor belt, battery controller, and GPS sensorFigure 10. Soldier with prototype NavCom and smart-phone visual display There are several challenges in the design of wearable tactor arrays. Tactors must be lightweight, punctuate (easily localizable) and provide vibratory stimuli under mechanical loading. Further, the tactile stimuli must be readily perceived. Figure 11 shows the EAI C-2 tactor that has been proven effective in previous experiments (Redden et al., 2006)., along with the newer, and smaller, C-3 tactor, and a low frequency motor based tactor, the EMR tactor. Cognitive Engineering and Neuroergonomics (2019)
The C-2 and C-3 are almost equivalent in vibratory output and our studies (Elliott, Mortimer, Cholewiak, Mort, Zets, Pittman, 2013) have shown that, in quiet environments, they provide equivalent Soldier performance in tactile orientation and pattern recognition tasks. For direction cues, the accuracy (percent correct) for the C-2 was 97.02% and the EMR was 97.62%. For combinations of direction and command cues, the accuracy for the C-2 was 99.40% and 99.40% for the EMR. Further investigations are planned to investigate performance in more operational context. The C-3 (6g) is substantially lighter than the C-2 (18g), therefore it can lower the system weight (especially in large arrays). The EMR is a new motor-based design with an operating frequency range of 60-250 Hz. This design is able to produce a wide range of perceivable tactile features ranging from a strong “alert” to a “soft” pressure pulse or “nudge”. The EMR can produce substantial peak displacements of up to 1.2 mm p-p (as measured against a phantom with the mechanical impedance of skin). In contrast, the C-2 or C-3 would typically only be driven to peak displacements of about 0.5 mm p-p owing to the relatively high PC channel displacement sensitivity.
Figure 11. The EAI EMR, C-2, and C-3 tactor transducers (left to right).
The C-2, C-3, and EMR were designed to create a strong localized sensation on the body and works like a plunger. For the C-2 and the C-3 tactor, the contact with the skin is from the predominant moving mass (Figure 12), driving the skin with perpendicular sinusoidal movement that is independent of the loading on the housing (Mortimer et al. 2007). Only the “inner circle” vibrates, while the outer ring acts as a reaction mass, thus stopping the “spread” of vibrations and improving the stimulus localization.
Figure 12. C-2 linear actuator: photograph (left) and operational schematic (right).
Figure available in the original PDF paper.
The EAI dual row, hybrid belt potentially allows the presentation of multiple types of tactile symbology as the tactor belt row may be comprised of C-2/C-3 and EMR tactors. The EMR tactors can be programmed over a range of sensations including thresholds much lower than the C-2 / C-3 so that different tactors can portray different kinds of information. The EMR can, and in this study, will be programmed to a lower frequency, creating a sensation that is less “sharp” than the C-2 or C-3. The C-2 / C-3 will be programmed at the frequency that is optimal for human perception (250Hz). The sensation is particularly “sharp” in the C-2 and C-3 due to its structure and designed resonance. The C-2 and C-3 tactors have a rise time of less than 2 ms while the EMR has a rise time of about 12 ms.
In an upcoming Soldier-based evaluation, we will investigate simultaneous presentations of navigation and robot communication/monitoring cues using tactile patterns during scenarios developed for operational relevance (Elliott, Redden, Schmeisser, Rupert, 2012). The experimental condition will replicate the task demands of the combined direction and robot alert cues while the Soldier is on the move, during night operations. The Soldier will receive GPS-driven waypoint navigation cues (single direction cues, using the C-2 tactors) that will guide Soldiers to waypoints and around exclusion zones. The Soldier will also receive incoming communications (i.e., four commands) from a hypothetical autonomous robot. The soldier will walk outside, alongside a data collector, following the direction cues. When the direction changes, he should change direction accordingly. When he receives an incoming robot alert, he notifies the data collector and states which alert is perceived. During part of the course, he will also have to maneuver around an exclusion zone and in another part, he will be searching for visual targets. This experiment is planned to occur at night in order to evaluate effectiveness during degraded visual conditions. Soldiers will use a standard military monocular night vision system.
In conjunction with the night navigation /communications experiment, data will also be collected for further investigation of the psychological construct of tactile salience. In our theoretical model, the construct of tactile salience is proposed to be mediated by three general factors: characteristics related to the user (e.g., individual differences), characteristics related to the technology (e.g., tactor characteristics), and characteristics related to the environment (e.g., task demands) (Elliott, Mortimer, Cholewiak, Mort, Zets, Pittman, 2013). Consistent with the research on visual salience, data will be gathered to explore characteristics of tactile salience under a baseline condition. Soldiers will experience and compare a variety of tactile sensations and provide feedback.
Figure 13. Model for Tactile Salience.
CONCLUSIONS
There is a large and growing literature of experiments that support the theory-based predictions regarding advantages of tactile cues to support performance in high-workload situations, particularly multi-tasked situations with high demands for focal visual attention. It has been established, through cognitive task analyses, that Soldiers have very high demands for visual attention, particularly when they are moving or shooting. As tactile displays are increasingly used for communication of more complex and multiple concepts, it will become evident that tactile and multisensory systems in general must be designed for salience (i.e. rapid and easy comprehension). This paper described the iterative efforts, starting with cognitive task analyses of Soldier roles and consideration of prevalent theories for reduction of cognitive workload through multisensory displays, and extending to execution of a large scale review of the literature, meta-analytic investigations around core distinguishing factors (i.e., alerts, direction cues, spatial orientation, and communications), to Soldier-based experiments. Finally, SBIR-funded efforts are developing new tactile system capabilities, associated with increased effectiveness, lower workload, and greater ease of use. These projects have resulted in intuitive gestural user interfaces for the system and a tactile display that has the potential to produce a wide range of tactile pattern characteristics. These developing systems also represent capabilities for the researcher, enabling the pursuit of innovative research at both basic and applied levels.
REFERENCES
Elliott, L., Coovert, M., Redden, E. (2009). Overview of meta-analyses investigating vibrotactile versus visual display options.
Proceedings of the 14th International conference of Human Computer Interaction. Elliott, Mortimer, Cholewiak, Mort, Zets, & Pittman (2013). Development of dual tactor capability for a soldier multisensory navigation and communication system. Proceedings of the 2013 International Human Computer Interface Conference, July, Las Vegas. Elliott, L. R., Redden, E., Krausman, A, Carstens, C. (2005). Multi-modal displays to support Army Infantry Decisionmaking
and performance. Proceedings of the 2005 International Conference on Naturalistic Decisionmaking, June 2005, Amsterdam, NL. Elliott, L., Redden, E., Schmeisser, E., Rupert, A. (2012). Tactical scenario development for user-based performance evaluations. Proceedings of the International Applied Human Factors and Ergonomics Conference, San Francisco, 21- 25 July. Elliott, L., van Erp, J.B.F., Redden, E., Duistermaat, M. (2010). "Field-Based Validation of a Tactile Navigation Device," IEEE
Transactions on Haptics, vol. 3, no. 2, pp. 78-87, Apr.-June Jones, L.A., Sarter, N.B. , Tactile Displays: Guidance for Their Design and Application, Human Factors: The Journal of the Human Factors and Ergonomics Society February 2008 vol. 50 no. 1 90-111 Merlo, J., Stafford, S., Gilson, R., Hancock, P. (2006). The effects of physiological stress on tactile communication.
Proceedings of the Human Factors and Ergonomics Society Annual Meeting. October 2006 vol. 50 no. 16 1562-1566 Mitchell, D. K. (2009). Workload Analysis of the Crew of the Abrams V2 SEP: Phase I Baseline IMPRINT Model. (Technical Report, ARL-TR-5028). Aberdeen Proving Ground, MD: US Army Research Laboratory. Mitchell, D.,Samms, C., Glumm, M., Krausman, A., Brelsford, M. and Garrett, L. (2004). Improved Performance Research
Integration Tool (IMPRINT) Model Analyses in Support of the Situational Understanding as an Enabler for Unit of Action Maneuver Team Soldiers Science and Technology Objective (STO) in support of Future Combat Systems (FCS). Aberdeen Proving Ground, MD: US Army Research Laboratory. Mitchell, 2005; Mitchell, D. K. and Brennan, G. (2009a). Infantry Squad Using the Common Controller to Control an ARV-A (L) Soldier Workload Analysis. (Technical Report, ARL-TR-5029). Aberdeen Proving Ground, MD: US Army Research Laboratory, Mortimer B, Zets G, Mort G and Shovain C, Implementing Effective Tactile Symbology for Orientation and Navigation, 14th International Conference on Human Computer Interaction, HCI (2011). Mitchell, D. K., and Brennan, G. (2009b). Infantry Squad Using the Common Controller to Control a Class 1 Unmanned aerial
vehicle system (UAVS) Soldier Workload Analysis. (Technical Report, ARL-TR-5012). Aberdeen Proving Ground, MD: US Army Research Laboratory. Mortimer, B., Zets, G., Mort, G., and Shovain, C., (2011). Implementing Effective Tactile Symbology for Orientation and Navigation. Proceedings of the 14th International Conference on Human Computer Interaction, HCI (2011). Pomranky, R. and Wojciechowski, J. (2007). Determination of Mental Workload During Operation of Multiple Unmanned
Systems. (Technical Report, ARL-TR-4309). Aberdeen Proving Ground, MD: US Army Research Laboratory. Redden, E. S, Carstens, C. B., Turner, D. D. and Elliott, L. R. (2006). Localization of Tactile Signals as a Function of Tactor Operating Characteristics. (Technical Report ARL-TR-3971.) Aberdeen Proving Ground, MD: US Army Research Laboratory Scalsky, D., Meshesha, D. and Struken, S. (2009). Army expeditionary warrior experiment (AEWE) spiral E final report. U.S.
Army Test and Evaluation Command: Alexandria, VA. U.S. Army Evaluation Center (2013). Army Expeditionary Warrior Experiment (AEWE) Spiral H Final Report. Request from Commander, U.S. Army Test and Evaluation Command (CSTE-AEC-FFE), 2202 Aberdeen Boulevard, Third Floor, Aberdeen Proving Ground, MD 21005-5001. U.S. Army Field Manual No. 21-60. (1987). Visual signals. http://armypubs.army.mil/doctrine/DR_pubs/dr_a/pdf/fm21_60.pdf Cognitive Engineering and Neuroergonomics (2019)
U.S. Marine Corps Rifle Squad manual (FMFM 6-5). http://www.amazon.com/Century-Marine-Marines-Training-Manuals/dp/1422052672/ref=sr_1_1? ie=UTF8&qid=1387307357&sr=8-1&keywords=fmfm6-5. Van Erp, J. (2007). Tactile displays for navigation and orientation: Perception and behavior. Leiden, The Netherlands: Mostert & Van Onderen Wickens, C. (2008). Multiple resources and mental workload. Human Factors, 50, 3, 449-454.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Cognitive Engineering and Neuroergonomics
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2101-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional