Geert De Cubber, Emile Le Flécher, Alexandre La Grappe - Dominicus, Daniela Doroftei (2023). Human-agent teaming between soldiers and unmanned ground systems in a resupply scenario. In: Human Factors in Robots, Drones and Unmanned Systems. AHFE Open Access, vol. 93, pp. 43–49. https://doi.org/10.54941/ahfe1003746
Thanks to advances in embedded computing and robotics, intelligent Unmanned Ground Systems (UGS) are used more and more in our daily lives. Also in the military domain, the use of UGS is highly investigated for applications like force protection of military installations, surveillance, target acquisition, reconnaissance, handling of chemical, biological, radiological, nuclear (CBRN) threats, explosive ordnance disposal, etc. A pivotal research aspect for the integration of these military UGS in the standard operating procedures is the question of how to achieve a seamless collaboration between human and robotic agents in such high-stress and non-structured environments. Indeed, in these kind of operations, it is critical that the human-agent mutual understanding is flawless; hence, the focus on human factors and ergonomic design of the control interfaces.The objective of this paper is to focus on one key military application of UGS, more specifically logistics, and elaborate how efficient human-machine teaming can be achieved in such a scenario. While getting much less attention than other application areas, the domain of logistics is in fact one of the most important for any military operation, as it is an application area that is very well suited for robotic systems. Indeed, military troops are very often burdened by having to haul heavy gear across large distances, which is a problem UGS can solve.The significance of this paper is that it is based on more than two years of field research work on human + multi-agent UGS collaboration in realistic military operating conditions, performed within the scope of the European project iMUGS. In the framework of this project, not less than six large-scale field trial campaigns were organized across Europe. In each field trial campaign, soldiers and UGS had to work together to achieve a set of high-level mission goals that were distributed among them via a planning & scheduling mechanism. This paper will focus on the outcomes of the Belgian field trial, which concentrated on a resupply logistics mission.Within this paper, a description of the iMUGS test setup and operational scenarios is provided. The ergonomic design of the tactical planning system is elaborated, together with the high-level swarming and task scheduling methods that divide the work between robotic and human agents in the fieldThe resupply mission, as described in this paper, was executed in summer 2022 in Belgium by a mixed team of soldiers and UGS for an audience of around 200 people from defence actors from European member states. The results of this field trial were evaluated as highly positive, as all high-level requirements were obtained by the robotic fleet.
Thanks to advances in embedded computing and robotics, intelligent Unman- ned Ground Systems (UGS) are used more and more in our daily lives. Also in the military domain, the use of UGS is highly investigated for applications like force protection of military installations (Carroll et al., 2014), surveillance, target acquisition, reconnaissance, handling of chemical, biological, radiolo- gical, nuclear (CBRN) threats (Schneider et al., 2016), explosive ordnance disposal, search and rescue SAR (De Cubber et al., 2017) or crisis manage- ment (Doroftei et al. 2009). A pivotal research aspect for the integration of these military UGS in the standard operating procedures is the question of how to achieve a seamless collaboration between human and robotic agents in such high-stress and non-structured environments. Indeed, in these kind of operations, it is critical that the human-agent mutual understanding is flaw- less; hence, the focus on human factors and ergonomic design of the control interfaces.
The objective of this paper is to focus on one key military application of UGS, more specifically logistics, and elaborate how efficient human-machine teaming can be achieved in such a scenario. While getting much less attention than other application areas, the domain of logistics is in fact one of the most important for any military operation, as it is an application area that is very well suited for robotic systems (Ergene, 2016). Indeed, military troops
are very often burdened by having to haul heavy gear across large distances, which is a problem UGS can solve.
This paper is based on more than two years of field research work on human and multi-agent UGS collaboration in realistic military operating con- ditions, performed within the scope of the European project iMUGS (Le Flecher et al., 2022). In the framework of this project, not less than six large-scale field trial campaigns were organized across Europe. In each field trial campaign, soldiers and UGS had to work together to achieve a set of high-level mission goals that were distributed among them via a planning & scheduling mechanism. This paper will focus on the outcomes of the Belgian field trial, which concentrated on a resupply logistics mission.
PREVIOUS WORK & MAIN CONTRIBUTIONS While the all-terrain mobility (Odedra et al. 2009) and environmental perce- ption requirement (De Cubber et al., 2011) still poses a serious problem for deployment in many environments, multiple tracked, wheeled and even leg- ged robotic systems have been developed for accompanying troops, while carrying their equipment (Byers, 2008). Other developments include unman- ned trucks using a follow-the leader protocol or serving as a mule for carrying heavy resupply material across sometimes large distances. However, there still exists a large gap between what is tested in research laboratories or even during field trials and the reality on the terrain. This disparity exists because of the highly complex nature of military operations, which require robust operation in all weather conditions and on all types of terrain, which is very difficult. The iMUGS project therefore started from an operationally proven platform (Milrem Themis, see Figure 1) and worked on the development of the human-agent teaming capabilities for this vehicle, in order to come to a solution that can be deployed for real operations.
Human-agent teaming between multiple highly autonomous unmanned systems and human operators in a complex military context requires the care- ful consideration of the design choices for the interface between the humans and the robots (Doroftei et al., 2012). Multiple types of human operators can come in contact with the robotic agents and each of them needs to commu- nicate with the robotic agents using an appropriate control interface. Within the iMUGS project, this was tackled by adopting a multi-layered approach, designing high-level command and control interfaces for field commanders that enable mission planning, but also lower-level control interfaces for field agents that enable basic human-to-vehicle interaction.
When deploying a large fleet of unmanned systems with human operators, especially in an international coalition context, one of the major difficul- ties is always the interoperability between the devices. In (Serrano Lopez et al., 2017), Serrano Lopez et al. propose a multi-robot interoperability fra- mework, enabling multiple robots to operate together, in synergy, enabling multiple teams to share data, intelligence and resources in realistic search and rescue missions. Mathiassen et al. expands on this concept in (Mathiassen et al., 2021), by demonstrating interoperability between UGS and multiple operator control units for military operations. Within the iMUGS project, an
Human-Agent Teaming Between Soldiers and Unmanned Ground Systems 45
open architecture was developed that allows for the integration of multiple interoperability protocols, in order to maximize the usability.
iMUGS TEST SETUP AND OPERATIONAL SCENARIOS The main goal of the iMUGS trials in Belgium was to demonstrate the swar- ming capabilities of the unmanned assets, meaning that they can work as a coordinated team in collaboration with the human operators and soldi- ers. Therefore, a realistic urban warfare scenario was created where a village is assaulted by enemy troops. The ‘friendly’ forces then move in to liberate the village, helped by the support of iMUGS unmanned systems. During the ensuing gunfight between the enemy & friendly troops, swarms of unman- ned ground systems are used again to resupply the friendly forces with new ammunition. This paper focuses on the resupply phase of the scenario.
The objective of the resupply scenario is to demonstrate the swarming system’s ability to be used in logistics. The fleet of UGS is used to automa- tically get supplies in a predetermined zone and drive to the delivery points. The assets involved in this trial are shown on Figure 1 and consist of:
3 Milrem Themis UGS for intelligence gathering & reconnaissance
1 Boxer Mission Control Vehicle
5 Robotnik Summit UGS, acting as a swarm system for resupply operati- ons
Figure 1: iMUGS robotic assets involved in the scenario. Top left: Milrem themis unmanned ground vehicle. Top right: boxer mission control vehicle; bottom: Robotnik Summit UGS swarm for resupply operations.
The different vehicles are equipped with heterogeneous sensor kits, such that each one of them brings extra information towards the mission controller installed in the Boxer control vehicle.
HUMAN MACHINE INTERFACE DESIGN CHOICES DISCUSSION A human – centred design approach (Doroftei et al., 2017) was followed within the iMUGS project in order to take into consideration the needs & requirements of the military end users during each step of the design process. As the needs for interaction modalities with the robotic assets are very diffe- rent for e.g. field operatives and mission planners, a multi-layered approach was adopted.
At the highest level, mission planners within the control vehicle are pre- sented with a command and control interface aimed at enhancing as much as possible the situational awareness of the operator, such that informed decisi- ons can be made, as shown on the top of Figure 2. It is also to be stressed that –using this command and control system - this mission planner also exerts at all-time meaningful human control over all aspects of the robot operati- ons. An important aspect for the mission planning system is minimizing the cognitive overload for the human operator. Indeed, due to the many data stre- ams that are available from the fleet of unmanned systems, there is a need to intelligently combine this data and present it to the operator in a com- prehensible manner. The UGS are able to execute planned missions totally autonomously. In this context, it consists of delivering goods to soldiers’ posi- tions, where these waypoint positions are automatically tracked using a blue force tracking battle management system.
Figure 2: iMUGS human robot interfaces. Top: mission planning system in the boxer control vehicle. Bottom: one-hand control interface.
Human-Agent Teaming Between Soldiers and Unmanned Ground Systems 47
As a mid-level control modality, UGS can also be tasked to follow soldiers, carrying goods for the soldiers. In order to do this, person detection (Lahouli et al., 2018) and person tracking tools (Enescu et al., 2006) are implemented that enable the UGS to follow a soldier from a safe distance.
For low-level control, operators in the field are able to take over control e.g. for complex manoeuvring operations using a one-hand control interface which keeps the other hand free for other tasks, as shown on the bottom of Figure 2.
FIELD VALIDATION WITH MILTARY END USERS The iMUGS Belgian field trial was executed on June 2nd 2022 at the military base of Marche-en-Famenne, Belgium. This location was chosen because it provides an excellent mix of multiple environmental characteristics: from forested areas to open grasslands to urban areas, and from hilly terrain to more flat areas. This variety allowed to validate the iMUGS concept of operation under multiple environmental conditions.
The field trial was performed in close collaboration with the 1/3L Com- pany from the Belgian Land component. These military operatives served both as actors in the mission scenario as first users and evaluators of the UGS tools put at their disposal. The integration of novel tools required a review of standard operating procedures and tactics, which did require serious re-planning with respect to normal operations. This shows that the integration of robotic tools in military operations is not an easy or straight- forward operation, but one that requires careful planning and training of all actors involved.
In the end, the military operatives succeeded to work very well as a coor- dinated team together with the robotic assets, as the quantitative evaluation of the mission performance showed that all technical requirements that were imposed by the stakeholders were successfully achieved. Figure 3 shows some
Figure 3: Snapshots of the iMUGS field trial campaign in Belgium, showing soldiers from the Belgian land component interacting with the iMUGS robots during an urban warfighting resupply scenario mission.
snapshots of the trial operation, while a video of the campaign is available here: https://www.youtube.com/watch?v=HBV3jSi9xd4.
CONCLUSION
In this paper, we presented the concept of the human-agent teaming system developed within the European project iMUGS. The concept adopts a multi- layered approach to ensure that each user is presented with an appropriate user interface.
The concept was validated using a complex military resupply scenario mis- sion, executed in summer 2022 in Belgium by a mixed team of soldiers and UGS for an audience of around 200 people from defence actors from Euro- pean member states. The results of this field trial were evaluated as highly positive, as the robotic fleet achieved all high-level requirements.
ACKNOWLEDGMENT
The research presented in this paper has received funding from the European Union’s EDIDP (European Defence Industrial Development Programme) programme under the iMUGS grant agreement.
REFERENCES
Byers, Brian. (2008) Multifunctional Utility/Logistics and Equipment (MULE) Vehicle Will Improve Soldier Mobility, Survivability and Lethality. ARMY AL&T. pp. 27–29. Carroll, D. Mikell, K. Denewiler, T. (2004) Unmanned Ground Vehicles for Integrated
Force Protection. In: Proc. SPIE 5422, Unmanned Ground Vehicle Technology VI. De Cubber, Geert. Doroftei, Daniela. Sahli, Hichem. Baudoin, Yvan. (2011) Outdoor Terrain Traversability Analysis for Robot Navigation using a Time-Of-Flight Camera. RGB-D Workshop on 3D Perception in Robotics. De Cubber, Geert. Doroftei, Daniela. Rudin, Konrad. Berns, Karsten. Serrano,
Daniel. Sanchez, Jose. Govindaraj, Shashank. Bedkowski, Janusz. Roda, Rui. (2017) Search and rescue robotics-from theory to practice. InTechOpen. Doroftei, Daniela. De Cubber, Geert. Colon, Eric. Baudoin, Yvan. (2009) Behavior based control for an outdoor crisis management robot. Proceedings of the IARP International Workshop on Robotics for Risky Interventions and Environmental Surveillance. pp. 12–14. Doroftei, Daniela. De Cubber, Geert. Chintamani, Keshav. (2012) Towards col-
laborative human and robotic rescue workers. 5th International Workshop on Human-Friendly Robotics (HFR2012). Doroftei, Daniela. De Cubber, Geert. Wagemans, Rene. Matos, Anibal. Silva, Eduardo. Lobo, Victor. Cardoso, Guerreiro. Chintamani, Keshav. Govinda- raj, Shashank. Gancet, Jeremi. (2017) User-centered design. “Search and rescue robotics. From theory to practice.” IntechOpen. London, UK. pp. 19–36. Enescu, Valentin. De Cubber, Geert. Cauwerts, Kenny. Sahli, Hichem. Demee-
ster, Eric. Vanhooydonck, Dirk. Nuttin, Marnix. (2006) Active stereo vision- based mobile robot navigation for person tracking. Integrated Computer-Aided Engineering. Vol. 13. No 3. pp. 203–222. IOS Press. Ergene, Y (2016) Analysis of unmanned systems in military logistics. Master’s thesis, Naval Postgraduate School, USA.
Human-Agent Teaming Between Soldiers and Unmanned Ground Systems 49
Lahouli, Ichraf. Karakasis, Evangelos. Haelterman, Robby. Chtourou, Zied. De Cubber, Geert. Gasteratos, Antonios. Attia, Rabah. (2018) Hot spot method for pedestrian detection using saliency maps, discrete Chebyshev moments and sup- port vector machine. IET Image processing. Vol. 12. No. 7. pp. 1284–1291. The Institution of Engineering and Technology. Le Flecher, E. La Grappe, A. and De Cubber, G. (2022) iMUGS – A ground multi-
robot architecture for military Manned-Unmanned Teaming. in 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems workshop on Human- Multi-Robot Systems Challenges for Real World Applications. IEEE. Mathiassen, K. Schneider, FE. Bounker, P. Tiderko, A. De Cubber, G. Baksaas, M.
Główka, J. Kozik, R. Nussbaumer, T. Röning, J. Pellenz, J. and Volk, A. (2021) Demonstrating interoperability between unmanned ground systems and command and control systems. International Journal of Intelligent Defence Support Systems. Vol. 6. Iss. 2. pp. 100–129. Nielsen, CW. Gertman, DI. Bruemmer, DJ. Hartley, RS. Walton, MC (2008) Evaluating Robot Technologies as Tools to Explore Radiological and Other Hazardous Environments. In: Proceedings of the 2nd Joint Topical Meeting Emergency Pre- paredness & Response and Robotics & Remote Systems. Emergency Management & Robotics for Hazardous Environments. Odedra, S. Prior, S. Karamanoglu, M. Shen, S (2009) Increasing the traf- ficability of unmanned ground vehicles through intelligent morphing. In: ASME/IFToMM International Conference on Reconfigurable Mechanisms and Robots. pp. 674–681. Serrano López, Daniel. Moreno, German. Cordero, Jose. Sanchez, Jose. Govin-
daraj, Shashank. Marques, Mario Monteiro. Lobo, Victor. Fioravanti, Stefano. Grati, Alberto. Rudin, Konrad. (2017) Interoperability in a heterogeneous team of search and rescue robots. Search and Rescue Robotics-From Theory to Practice. pp. 93–125. InTech. Schneider, FE. Gaspers, B. Tiderko, A. Katorgin, O., Wildermuth, D. (2016) Unmanned systems for radiological and nuclear measuring and mapping. In: Proceedings of the International Scientific and Technological Conference “Extreme Robotics”.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Factors in Robots, Drones and Unmanned Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-69-8
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional