Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Certifying unmanned systems

The Norwegian Defence Research Establishment, Norway ↗
Publication year
Digital Object Identifier10.54941/ahfe1006788

Abstract

The paper presents a review that summarizes and importantly synthesizes the status of the certification for the time being a Norwegian drone system. Focus is on how this certification process generates new insights or creates an integrated analytic approach (critical view on) the activity of certifying based on feedback from Norwegian military end users. Main question to be discussed when planning and implementing a test set-up for the certification i.e., a course syllabus, is how to facilitate a course syllabus and activities that most efficiently qualify and certify an unmanned system. The test bed, developed by the Norwegian Defense Research Establishment (FFI), are supported by a framework of various software configurations adapted for behaviors in various military applications across different domains (underwater, ground, air). We investigate how operators and instructor trigger dialogue in observational tasks during tutored physical flying session certifying an air drone system and how do debriefing and learning create cognitive comprehension that is the competency developed in a team during certifying a military unmanned system.

INTRODUCTION

The deployment of Unmanned Aerial Systems (UAS), i.e. drones, for military applications is growing, offering numerous benefits for squad- level operations, particularly for Special Operations Forces (SOF). Special Reconnaissance (SR) refers to highly specialized military reconnaissance activities, often conducted in sensitive or hostile environments to collect strategic intelligence. It involves a unique blend of skills, including clandestine operations, covert surveillance, and advanced technology to enable global awareness and support military operations. The objective for training and certification in this context is Special Force squads need to train and certify for SR operations supported by Unmanned Aerial Systems (UAS) to build trust (Lyons et al., 2021) and skills (Cools & Maathuis, 2024). Focus of the training is based on mitigating risks tasking UAS, building agent and team situational awareness (SA), and coordination levels required for SR missions (Endsley, 2015-2024; Stensrud et al., 2024).

Endsley (2023, s. 5): “In addition, across many domains and systems there is a need for people to have SA of how well both they and their

© 2025. Published by AHFE Open Access. All rights reserved. 343

teammates are performing (i.e., agent SA). In an effective human-human team, people monitor their teammates to determine if they are overloaded or able to perform properly (e.g., fatigued, experienced on the task), and adjust task assignments accordingly (Endsley et al., 1998; Endsley & Robertson, 1996; W. M. Jones, 1997; Salas, Burke, & Cannon-Bowers, 2000). They also monitor their own performance and ask for help from their teammates or take other actions to compensate for deficiencies, such as taking a break from rest or deferring tasks to others. They similarly attempt to have an understanding of how complete their own SA is (meta-SA) so that they can act to gather more information and delay decision making if needed and possible (Endsley, 2020; Endsley & Jones, 2012).” The theme of the UAS system development is to build resilience in the UAS system and should be viewed as a layered system of systems. Key threat against UAS system operations is denied and degraded cyberspace (Alberts, 2024) and electromagnetic environments (Theron & Kott, 2019). We present a template for the certification process for unmanned systems, emphasizing the importance of structured roles, tasks, and processes to support communication and collaboration among involved actors (Mathieu et al., 2018; McNeese et al., 2018; Stensrud, Valaker & Haugen, 2020). The system design and certification section explain the work conducted, starting with the context of ground surveillance and challenges certifying a swarm system, as well as the operators of this system. The paper includes methods in use, samples and examples and lessons learned from the certification, highlighting the importance of well-structured procedures and technical aviation challenges. Further on, we will discuss why the certification process needs qualified military operators to accomplish a regulatory regime for unmanned systems. The lessons learned section presents feedback from military end users (i.e. recruited from SOF units) documenting their experiences and observations during the certification process.

METHODS

The paper describes interview activities, feedback, and results of unmanned systems. We will introduce methods that are fully dependent upon a UAS testbed developed by FFI for military purposes. This includes the vehicles themselves, ground control stations, support systems, communication and data links, networks, and personnel.

Analysis Analysis we define as a sequence of arguments that show compliance between the results (theory examination) of operators and the given expectation(s) of the certification of unmanned system overall results (in team settings, and qualitative feedback). For example, we apply an analysis of time spent during check-out of system technical components, i.e. evaluation of methods validating and verifying system status of the UAS-system components leading to safe flight (Stensrud, Valaker, & Haugen, 2023). One step of analysis is measuring time used on different activities during certification.

Certifying Unmanned Systems 345

Table 1: Tracking the time spent on various activities during the certification program (Salmon et al., 2024).

Table available in the original PDF paper.

Access to Tools and Expertise Results Classroom, preparation Time to complete, theory and examination: 8 lessons one hour each. Simulator-based training Time to complete: four hours per team Physical flying (temperature, pressure, humidity, challenges, quality, etc.)

Time to complete, pre- and final examination: three hours per team.

Based upon time estimated measurements we will propose a baseline for certification of drone systems and a recommended distribution of time between types of activity during preparation of pilots for unmanned system certification.

Demonstration Demonstration we define as bringing forward proof to support a given conclusion; for example, using a simulator-based training application mixed with flying a UAS-system in various configurations, for training and in support of resolving a reconnaissance task. And to prove functionality of UAS Valkyrie for the Norwegian MAA at the site during the course. Demonstrating and exploring what happens when Norwegian operator teams certifying for doing physical and practical task solution in Norwegian winter climate, exposed for scenes with various day-light and darkness, to resolve use cases emulating non-collaborating soldiers and cars in a military field range. The demonstration of safe flying skills will certify the operators to qualify for flights of UAS in military exercises.

Test A test defines an event, or a series of events performed on the UAS- system, sub-systems, or components (like the “Flamingo” dance to configure magnetic compass) that involves the candidates and what they must check before flying. Routines of checking system status, batteries, power, etc. are included in the course syllabus tests. Tests may also be applied to gain understanding and evidence for the specified system limits, the operational envelope of conduct of safe flight for the UAS system, tests of sensor payload, weatherproof tests of control link, EMC testing etc. These tests are documented (in reports, in calibration sheets etc.). These documents (operator manual, system manuals) given Norwegian MAA before the examination of drone operators for approval of UAS-system. Instrumented tests, inspections done by the candidates, maintenance, etc. were trained according to the course syllabus.

Simulation Simulation or simulation-based training we define as a one-to-one representation of UAS-functions supported by a man-machine interface (MMI) to emulate procedures and activating functions for safe flying. The interactive physical flight and simulator training played a significant role in the course syllabus. Interactive learning, defined as two-way interactions

between learner and instructor rather than one-way information sharing (Moreno & Mayer, 2007), has been associated with increased engagement and improved performance compared to passive classroom sessions (Russell et al., 2016). Hands-on flying time is essential for developing a solid understanding and the ability to operate the Valkyrie system safely and responsibly. However, physical flight training can be time-consuming and requires access to various resources, such as instructors, UAVs, and training areas. It would not be an effective use of time if operators had to wait passively for their turn due to a limited number of instructors and drones.

Simulation-based training helps increase overall training time and provides more opportunities for operators to engage in learning contingencies. Instead of passively waiting for their turn in physical flight training, they can actively participate in simulation-based training. The incorporation of simulation- based training impacts operators’ learning outcomes. When introduced before physical UAV training, simulation-based training has been shown to significantly improve flight performance compared to physical UAV training alone (Somerville et al., 2024). Additionally, interactive simulation-based training has been linked to better short- and long-term retention of learning objectives compared to lecture-based learning or more passive simulation- based training (Lohmann, 2020). Theory related to learning methods and structuring the certification course will be supportive of the discussion later in the article, and the research questions below.

Figure 1: The test bed “Valkyrie ISR” is approved by the Military Airworthiness Authority (MAA) (Adapted from Minos-Stensrud et al. (2025); Norwegian Defence. (2023)).
Original PDF Figure 1:

Certifying Unmanned Systems 347

RESEARCH QUESTIONS Main topic (research question) for the use case is to determine a team composition (who is most suited to be vehicle controller, mission controller etc.) and properties for dealing with different dynamic situations (handling interruptions etc.). To help this selection in the future for certifying a unmanned system we are to investigate the following Research Questions (RQs):

RQ1: How do operators and instructor trigger dialogue in observational tasks during certification of unmanned systems (Lundberg and Johansson, 2021) (i.e. tutored physical flying session) testing functions and demonstrating functionality during the certification of an unmanned system?

RQ2: How can the results of simulation-based training sessions (e.g. semi- autonomous training in UAS simulator) for observational tasks be described and evaluated during a certification program of drones?

RQ3: How do debriefing and learning create the cognitive comprehension that is the competency developed in a team during a certification process of drones?

RESULTS

The sub-section describes interview activities, feedback, and the certification of drones. The site chosen for study was an experimental range owned by the Norwegian Armed Forces and Norwegian Defence Research Establishment in common and designed for military trials and experiments in Norway (Alberts & Hayes, 2005). The activities observed during the study were a certification process for unmanned systems and examination of drone operators. The data collection and boundaries for the study was a 5-day certifying course with a final exam. The analysis and foundational methods in use in this study primarily based on a qualitative analytic framework (case study) meaning that the procedures used in interpreting data fit into a predetermined structure identified as the protocol or guide (named course syllabus) followed when certifying drone operators.

During the 5-day course certifying drone operators, the dialogue between operators and instructors was monitored by the observatory team and we identified that it is triggered through various structured and spontaneous interactions. The dialogue between operators and instructors typically unfolds in briefing sessions, pre-flight preparations, and in simulation training. Simulation-based training sessions, such as those involving semi- autonomous UAS simulators for observational tasks, can be described and evaluated through several key metrics and methods.

The data collection was done on site by an observer team and formalized as sessions with open-ended questions in that the observers asked general questions of the participants, allowing the participants to provide their views freely. The observers conducted face-to-face interviews on site and these interviews were semi-structured with a generally few open-ended questions intended to elicit views and opinions from the participants after action and during their individual self-study sessions. The data collection and analysis are based on coded observations, interviews, documents,

and digital materials – and anonymized according to security and ethical regulation protocols for Army people in action– which is paramount to allow publication of the research. There were qualitative observations schemas designed for After Action Review (AAR) filled out by the participators during the course syllabus process designed specifically for feedback on the certification of drone pilots (operators) available for the participants during the last day of the course. These written documents were digitally scanned by the supporting observer team, and coded and generated fruitful feedback and results for the Army and Research program. The output was aggregated – then the original scanned material was destroyed by the observer team. Additionally, action reports and qualitative interviews of the participants have been coded according to qualitative interpretation techniques and anonymized. Recording and digital material of simulation results of drone operations and interventions of instructors during the certification process has been useful information when talking directly with participants afterwards, and during the 3-days-visit at the research site. Debriefing and learning are critical components in developing cognitive comprehension and competency within a team. However, the structured reflection techniques and methods in debriefing sessions, and through knowledge sharing and learning, and reinforcement of learning objectives supported by training, were fruitful because of the casting of operators.

SYSTEM DESIGN Valkyrie is an unmanned aircraft system (UAS) consisting of several individual components that together deliver a capability that, alone or in a swarm, solves missions with a degree of automation introduced for Norwegian Army as an important tutoring, and training system. Valkyrie units fall into Class 1 A according to EASA rules and regulations. The Valkyrie UAS is a generic unmanned swarm system consisting of several cooperating UAVs Nummedal (2021). The Valkyrie UAS integrates sensors, platforms, and control logic with a user interface that enables an operator to control multiple UAVs simultaneously Minos-Stensrud et al. (2021). The Valkyrie swarm system consists of four types of UAVs: Svale, Flamingo, Hubro (VTOL ISR), and Pappegøye (EW). The swarm UAV system, Valkyrie, supports autonomous drone surveillance with auto-follow, UAV’s calculating the cross-bearing of a target as Digitally Assisted Fire Support (DAFS), where two UAVs calculate cross-tangents of a target, and No-comms Autonomous Search to find targets without communication. The system uses machine vision and machine learning algorithms for various tasks such as hotspot detection and motion detection. Valkyrie UAS was used in military exercises such as Nordic Response 2024, WESC 2024, and Joint Viking 2025. Preparations for these exercises were made in the period 2023 (4th quarter) to 2024 (1st quarter) with intense training of drone operators and preparation of the UAS system.

System in Use The sub-section includes a summary of the use case and mission contexts (Jenkins et al., 2023), such as route reconnaissance, and real-time situational

Certifying Unmanned Systems 349

awareness. The Mission given to the teams was to plan monitoring road axis (north/south) and intersect road simultaneously with an area search. The candidates took turns playing the role of Mission Operator (MO) and Vehicle Operator (VO), respectively. Further on, the operators had to select a deployment pattern (direct command or pre-programmed mission). They had to consider possible icing on the surfaces of the vehicles as well as wind loads in the next few hours that the pre-examination was carried out (about an hour for each team). Home address defined by Drone 1 technically set-up for pre-exam sessions. Test first with deployment of Drone 1, then other drones (Drone 2 and Drone 3). Roles were changed midway. Detections are given over the loudspeaker to the operator from the control system as a voice message: “new detection, drone 1”. Weather conditions (1 Feb 2024): 5–10 M/S and good visibility; medium-high cloud base. The mission was the same for all sessions. However, the weather changed during the afternoon and challenged the late afternoon pre-examination sessions. The dialogues between operators in each team were observed and monitored during sessions of examination. The certification process includes human-in- the-loop data collection and after-action review during system certification with military end users for the period 29th January to 2nd of February 2024. We present results of an analytical post-process of feedback from military end users, documenting their experiences and observations during the certification process. It highlights the importance of simulator training for building predictability and confidence in the behaviours of the Valkyrie UAS system.

DEVELOPING A SYLLABUS TO CERTIFICATE DRONE OPERATORS AND AN UNMANNED AIRCRAFT SYSTEM (UAS) The certification process involves a collaboration between the Norwegian Armed Forces, Norwegian Defence Research Establishment (FFI), the Norwegian Armed Forces School (HVS), and the Norwegian Air Force’s Air Operational Training and Certification Center (LFTS).

Course Content Requirements Course content requirements (duration) to qualify operators and certify Unmanned Aircraft System (UAS) named Valkyrie in situ are dependent upon motivated students (operators) and an UAS approved by the Military Airworthiness Authority (MAA) (The Norwegian MAA). The course syllabus includes simulator-based training, physical flying, and classroom sessions. The program follows a progression-based approach, adding training or learning objectives for each sort of sortie. It emphasizes the need for methods to manipulate complexity and dynamics in the environment and measures for goal attainment.

System Design as a Supportive Framework for Certification of Drones and Operators System experts work within a framework, consisting of processes and methodologies, provided by systems engineering to ensure successful

integration of the UAS swarm system. Methodologies include the familiar, carefully structured approach to meeting the functional and nonfunctional requirements for certification of drone operators.

Most of the requirements for drone operator training and certification are derived from requirements for a balanced set of activities (e.g., various semi-autonomous training activities in simulators, classroom sessions, and physical flight exercises). The certification process includes human-in-the- loop data collection and analysis during system certification with military end users reported in this paper. Based on the performance, efficiency, environmental, operational, maintenance, and training, we will present a baseline of activities and results (see Table 1) enhancing the certification of drone pilots (to promote efficacy and effectiveness) based on lessons learned from certification sessions.

Course Syllabus Course syllabus includes theoretical classroom sessions, self-study, and interactive learning sessions through physical flying and semi-autonomous simulation-based training estimated for a 5-day course program supporting a drone certification process (Table 2). The section labelled white indicates theory/classroom sessions and educational work with final theory exam (i.e. important classroom work briefing/presented by instructor arranged for the candidates during the certification to drone operators.). White lessons were approx. planned for 1-1, 5 days. Sections labelled green are simulator-based training. Green lessons were approx. planned for 1 day. The section labelled blue indicates physical flying with the Valkyrie UAS system and planned for approx. 2-2,5 days.

Table 2: The proposed course syllabus. Total of 5 days: 50% Physical training (incl.

Table available in the original PDF paper.

examination), 30% theory (incl. examination), and 20% simulation training. (Adapted from Minos-Stensrud & Nummedal, 2025).

The course syllabus consisted approx. 50%–30%–20% distribution of activity time, i.e., physical flying tutored by instructor, classroom exercises, and semi-autonomous training in simulation-based tool following

Certifying Unmanned Systems 351

a progression-based approach where for each sortie (take-off, flight and landing) a moment, training or learning objective is added.

It is a requirement that each student familiarizes themselves with the operator’s manual created by FFI before the start of the course. It introduces the following topics, among others: (1) System limitations, (2) Emergency procedures, (3) Flight procedures, (4) System description, (5) System operation, (6) Maintenance, troubleshooting and storage and (7) Risk assessment.

Course Syllabus Day by Day The program follows a progression-based approach where for each sortie (take-off, flight and landing) a training or learning objective is added.

DAY 1 - Visual Line of Sight (VLOS) - Intro Multirotor/Intro Valkyrie SIM Day 1 introduces the use of a single UAV VLOS (Visual Line of Sight) using hand controls and an introduction to the simulator. The simulator is an important and effective tool when it comes to education, training, and practice in the Valkyrie system. The goal for the day is that the student can operate a multicopter safely using hand control and can train independently in the simulator.

DAY 2 – Beyond Visual Line of Sight (BVLOS) - Intro Valkyrie Flight Day 2 introduces ground routines, the swarm system, Vehicle Operator (VO) or Mission Operator (MO) cooperation and emergency procedures. The goal for the day is that the student will independently: set up the system, operate the system safely, take down the system, and perform emergency procedures in the simulator.

DAY 3 – BVLOS - Valkyrie Missions and Theory Day 3 focuses on mission training, an introduction to what happens in the machine and what this means for us, mass training on emergency procedures and dark flying. The goal for the day is that the student should have an idea of how to solve ISR missions using the system, understand how the system works and what practical significance this has, and be able to perform emergency procedures independently.

DAY 4 - BVLOS - Valkyrie Mission Day 4 focuses on mass training in the simulator, pre-exam, and maintenance of the system. The goal for the day is that the student should have an idea of how to use the system effectively in mission solving, carry out a mission independently, and be able to perform 1st line maintenance.

During Day 4 the military end users were exposed for military use case planning and seeking a scene with not-friendly activities and calculating the cross-bearing of a target as Digitally Assisted Fire Support (DAFS). The route (road) that for various reasons was inspected by a swarm of autonomous drones recognizing an area of land supported with auto-follow functions.

DAY 5 - BVLOS - Exam Day 5 is used for the exam and the conclusion of the course. Requirements for passing the course are that the student shows good understanding and ability to operate the system in a safe and responsible manner. This includes a good understanding and risk assessment of factors that can affect the flight.

Examples of themes from the course syllabus and results discussed below.

DISCUSSION

We discuss the certification process for unmanned systems, emphasizing the importance of structured roles, tasks, and processes to support communication and collaboration among involved actors as well as looking at the whole UAS drone eco system need of facilities (strategic directives, governmental acceptance and flexible regulations). The overall military operational concept for the utilization of UAS in the Norwegian Armed Forces says little about requirements for training of operators and technical personnel.

Management of UAS in the Norwegian Armed Forces can be roughly divided according to two principles. The first is based on an approach where UAS are treated administratively, regularly and operationally as traditional aircraft, under the auspices of the Royal Norwegian Air Force. On the other hand, small systems should operate under a common set of regulations, with common basic training, but be acquired and managed at the tactical level as a supply item. Transferred to an organizational perspective, the smaller systems are personal, or team equipment operated by one team member - in support of the team. Larger systems that are vehicle- or runway-dependent are operated by dedicated units and selected teams in support of a unit. Considering the Norwegian Army likely to be the largest user of UAS, there is probably a need to clarify roles, responsibilities and authority for the Norwegian Armed Forces’ UAS ecosystem.

It has become clear that treating the Valkyrie swarm system as a separate entity (i.e. in a laboratory setting) separated from the end users, results in poor performance and potential failure when the UAVs are freed to be operated in an operational setting. However, continued development and growth in training and distributed learning technology (e.g. simulator-based training, and development of in-situ team-training facilities combined with the development of team-training methods) has delivered desired results for Norwegian Army as well as knowledge and expertise of Norwegian Defence Research Establishment. In the future, distributed training and course sessions will be further expanded because of the need for more certified drone operators. The need for expanding and stepping up the amount of and variety of certification courses will impact these activities and will have impact upon the UAS eco system in Norway (field ranges, facilities, organisation and arrangements satisfying the requirements of quality training) including the military-civil cross-collaboration (air traffic control, and air traffic management services). The potential explosion of un- trained users operating in future harsh and dynamic environments impacts the need for multi-task and multi-team coordination training with UAS

Certifying Unmanned Systems 353

swarm systems. And that is why the certification process of military operators needs to accomplish a regulatory regime based on principles for long-lifecycle manned systems because they are a part of a defence system of systems.

FUTURE RESEARCH The paper presents feedback from military end users, documenting their experiences and observations during the certification process of an unmanned system (named Valkyrie), and ideas for future team training. Debriefing and learning sessions are essential for developing cognitive comprehension and competency within a operator team. They provide a platform for reflection, knowledge sharing, feedback, reinforcement of learning objectives, scenario analysis, emotional and social learning, and practical application. These processes ensure that team members continuously improve and develop the skills necessary for effective teamwork and mission success. It highlights the importance of simulator training for building predictability and confidence in the behavior of the Valkyrie system. The paper presents feedback from military end users, documenting their experiences and observations during the certification process. It highlights the importance of simulator training for building predictability and confidence in the behavior of the Valkyrie system. The paper discusses the influence of environmental characteristics on system-level performance and human-robot interaction (HRI). It emphasizes the need for future methods to manipulate complexity and dynamics in the environment and measures for goal attainment. The paper includes examples and lessons learned from the certification course, highlighting the importance of well-structured procedures and technical aviation challenges. The course syllabus includes simulator training, physical flying, and classroom sessions.

The program follows a progression-based approach, adding training or learning objectives for each sort of sortie. Requirements for appropriate team size, hosting instructors, and proper management certifying drone operators; the duration of course should not be shorter than 5 days. The time line for an optimal 5 days course should be approximately 50%–30%–20% – i.e., two and a half day of time spent on physical flying (including examination) – and one and a half day minimum on theory (reviews/presentations in classroom including examination) and at least one day on semi-autonomous work – simulator-based training, necessary for optimal implementation of the course syllabus.

The operational framework for the certification process is fully dependent upon a functional and certified UAS system. For time being, the Valkyrie UAS is in use in Norway that integrates sensors, platforms, and control logic with a user interface that enables an operator to control multiple UAVs simultaneously.

The certification process involves a collaboration between the Norwegian Armed Forces, FFI, the Norwegian Armed Forces School (HVS), and the Norwegian Air Force’s Air Operational Training and Certification Center (LFTS).

The system design and certification include the Valkyrie swarm system consisting of four types of UAVs: Svale (Unmanned Counter Aerial Vehicle), Flamingo (Quad-Copter), Hubro (Vertical Take-Off Landing ISR), and Pappegøye (Electronic Warfare).

The analysis of the certification process includes human-in-the-loop data collection and analysis during system certification with military end users supported by observers from Norwegian FFI. The paper suggests future research framework and human-autonomy-teaming decision support (courses of action generation, optimal route scheduling) related to training in environmental complexity, cooperation design for handling complicated scene-analysis, and the integration of swarm technologies into various crewed and uncrewed units for area projection. A comprehensive future study of relevant techniques and ways of exploring the state of the art of methods and approaches for certification of drones is needed. Comparison of existing methods and future immersive technologies and techniques to handle health and flight safety in certification processes educating UAS-operators, i.e. monitor and report lessons learned, and safety deviations, are needed in future.

ACKNOWLEDGMENT

The authors would like to acknowledge colleagues at Norwegian Defence Research Establishment (FFI) participating in the Autonomy program at the Institute contributing to this research, led by Dr. Rikke Seehus. We are grateful to Six Robotics, for contributing important insights about VALKYRIE swarm system during the experiments in 2024.

REFERENCES

Alberts, D. (2024) ‘Cyber Risk to Mission Exploratory Analysis’, in Cyber.

International Command Control Symposium (ICCRTS), UK, London: The Command and Control Institute, pp. 1–18. Available at: https://docs.google.com/ presentation/d/104lu1irYIS6oHyAwtByZuCse8j8S9D1v/edit#slide=id.p1 (Accessed: 31 January 2025). Alberts, D. S. and Hayes, R. E. (2005) Code of best practice: Campaigns of Experimentation. Pathways to Innovation and Transformation. Office of the Assistant Secretary of Defense (OASD): Command & Control Research Program (CCRP) (CCRP Publication Series). Available at: http://www.dodccrp.org/files/ Alberts_Campaigns.pdf (Accessed: 15 May 2025). Cools, K. and Maathuis, C. (2024) ‘Trust or Bust: Ensuring Trustworthiness

in Autonomous Weapon Systems’, in MILCOM 2024 - 2024 IEEE Military Communications Conference (MILCOM). MILCOM 2024 - 2024 IEEE Military Communications Conference (MILCOM), Washington, DC, USA: IEEE, pp. 182–189. Available at: https://doi.org/10.1109/ MILCOM61039.2024.10773908. Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making. In Individual and group decision making: Current issues (pp. 221–246). Lawrence Erlbaum Associates, Inc. Endsley, M. R. (1988). Design and Evaluation for Situation Awareness Enhancement.

Proceedings of the Human Factors Society Annual Meeting, 32(2), 97–101. https://doi.org/10.1177/154193128803200221 Endsley, M. R. (2015). Situation Awareness Misconceptions and Misunderstandings.

Journal of Cognitive Engineering and Decision Making, 9(1), 4–32. https:// doi.org/10.1177/1555343415572631

Certifying Unmanned Systems 355

Endsley, M. R. (2017). From Here to Autonomy: Lessons Learned From Human– Automation Research. Human Factors: The Journal of the Human Factors and Ergonomics Society, 59(1), 5–27. https://doi.org/10.1177/0018720816681350 Endsley, M. R. (2020). The Divergence of Objective and Subjective Situation

Awareness: A Meta-Analysis. Journal of Cognitive Engineering and Decision Making, 14(1), 34–53. https://doi.org/10.1177/1555343419874248 Endsley, M. R. (2023). Supporting Human-AI Teams: Transparency, explainability, and situation awareness. Computers in Human Behavior, 140, 107574. https:// doi.org/10.1016/j.chb.2022.107574 Endsley, M. R. (2024). Understanding Automation Failure. Journal of Cognitive

Engineering and Decision Making, 15553434231222060. https://doi.org/ 10.1177/15553434231222059 Endsley, M. R., Bolte, B., & Jones, D. G. (2001). Designing for Situation Awareness (0 ed.). CRC Press. https://doi.org/10.1201/9780203485088 Endsley, M. R., & Jones, D. G. (2012). Designing for Situation Awareness (2nd ed.,

(2012)). Taylor & Francis, London. https://doi.org/10.1201/9780203485088 Endsley, M. R., & Jones, W. M. (1998). Situation Awareness Information Dominance & Information Warfare. ((AL/CF-TR-1997-0156)). https://apps.dtic.mil/sti/ citations/ADA347166 Endsley, M. R., & Robertson, M. M. (1996). Team Situation Awareness in Aviation

Maintenance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 40(21), 1077–1081. https://doi.org/10.1177/154193129604002107 Jenkins, M., Kingsley, C., Stone, R., & Kelly, S. (2023). Small UXS-Squad Teaming: Military Operational Use Cases. AHFE 2023 Hawaii Edition. https://doi.org/ 10.54941/ahfe1004270 Lundberg, J. and Johansson, B. J. E. (2021) ‘A framework for describing interaction

between human operators and autonomous, automated, and manual control systems’, Cognition, Technology & Work, 23(3), pp. 381–401. Available at: https://doi.org/10.1007/s10111-020-00637-w. Lyons, J. B. et al. (2021) ‘Human–Autonomy Teaming: Definitions, Debates, and Directions’, Frontiers in Psychology, 12, p. 589585. Available at: https://doi.org/ 10.3389/fpsyg.2021.589585. Mathiassen, K. and Nummedal, O. R. (2022) LandX20 Experiment Report – experiment for future land warfare capabilities with focus on increased situational awareness and unmanned systems. FFI-report 22/ 00274. kjeller: FFI. Available at: https://www.ffi.no/publikasjoner/arkiv/ landx20-experiment-report-experiment-for-future-land-warfare-capabilities- with-focus-on-increased-situational-awareness-and-unmanned-systems. Mathieu, J. E., Luciano, M. M. and DeChurch, L. A. (2018) ‘Multiteam Systems: The

Next Chapter’, in The SAGE Handbook of Industrial, Work & Organizational Psychology. 55 City Road: SAGE Publications Ltd, pp. 333–353. Available at: https://doi.org/10.4135/9781473914957. McNeese, N. J. et al. (2018) ‘Teaming With a Synthetic Teammate: Insights into Human-Autonomy Teaming’, Human Factors: The Journal of the Human Factors and Ergonomics Society, 60(2), pp. 262–273. Available at: https://doi.org/ 10.1177/0018720817743223. Minos-Stensrud, M., Jonas Fossum Moen, H. and Dyre Bjerknes, J. (2021)

‘Information sharing in multi-agent search and task allocation problems’, in 2021 IEEE Symposium Series on Computational Intelligence (SSCI). 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA: IEEE, pp. 1–7. Available at: https://doi.org/10.1109/SSCI50451.2021.9660121.

Minos-Stensrud, M. and O. R. Nummedal (2025) ‘Status Valkyrie 2025’, in 2025 FFI Autonomous Symposium Series, Kjeller, FFI, Norway, pp. 1–7. Available at: https://ffi.no. Moreno, Roxana & Mayer, Richard. (2007). Interactive Multimodal Learning

Environments. Educational Psychology Review, 19(3), 309–326. https://doi.org/ 10.1007/s10648-007-9047-2 Norwegian Defence. (2023). Syllabus for type check on UAV class 1A - Valkyrie - Test and trial Type check Valkyrie - T&P (p. 12) [Instructions for commander Royal Norwegian Air Force’s weapons school with subordinate commanders (commander’s instructions LVS) (2022-08-01) point 3.1 letter f)]. Royal Norwegian Air Defence. Nummedal, O. R. (2021) Flamingo - a UAV for autonomy research. FFI-Report 21/00318. Kjeller: FFI. Available at: https://www.ffi.no/publikasjoner/arkiv/ flamingo-a-uav-for-autonomy-research. Russell, Jae-Eun, Van Horne, Sam, Ward, Adam S., Bettis, E. Arthur, Sipola, Maija, Colombo, Mariana & Rocheford, Mary K. (2016). Large Lecture Transformation: Adopting Evidence-Based Practices to Increase Student Engagement and Performance in an Introductory Science Course. Journal of Geoscience Education, 64(1), 37–51. https://doi.org/10.5408/15-084.1 Salmon, P. M. et al. (2024) ‘When tomorrow comes: A prospective risk assessment

of a future artificial general intelligence-based uncrewed combat aerial vehicle system’, Applied Ergonomics, 117, p. 104245. Available at: https://doi.org/ 10.1016/j.apergo.2024.104245. Somerville, Alexander, Lynar, Timothy, Joiner, Keith & Wild, Graham. (2024). Use of Simulation for Pre-Training of Drone Pilots. Drones, 8(11), 640. https://doi.or g/10.3390/drones8110640 Stensrud, R. et al. (2024) ‘Situation awareness training as a prerequisite for handling complexity in Human-autonomy teaming: Demonstration and experiment proposal’, in. Intelligent Human Systems Integration (IHSI 2024) Integrating People and Intelligent Systems. Available at: https://doi.org/10.54941/ ahfe1004514. Stensrud, R., Valaker, S. and Haugen, T. (2020) ‘Interdependence as an Element of

the Design of a Federated Work Process’, in 2020 IEEE International Conference on Human-Machine Systems (ICHMS). 2020 IEEE International Conference on Human-Machine Systems (ICHMS), Rome, Italy: IEEE, pp. 1–6. Available at: https://doi.org/10.1109/ICHMS49158.2020.9209423. Stensrud, R., Valaker, S. and Haugen, T. (2023) ‘Unified Vision-series of military exercise - the methodological struggles in conducting evaluation of Human Factors and Systems Interaction’, in. 14th International Conference on Applied Human Factors and Ergonomics (AHFE 2023), AHFE, pp. 1–20. Available at: https:// doi.org/10.54941/ahfe1003605. Theron, P. and Kott, A. (2019) ‘When Autonomous Intelligent Goodware Will Fight Autonomous Intelligent Malware: A Possible Future of Cyber Defense’, in MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM). MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM), Norfolk, VA, USA: IEEE, pp. 1–7. Available at: https://doi.org/ 10.1109/MILCOM47813.2019.9021038. Lohmann, Robert. (2020). Effects of Simulation-Based Learning and One Way to Analyze Them. Journal of Political Science Education, 16(4), 479–495. https://doi.org/10.1080/15512169.2019. 1599291

Cover of the proceedings volume Human Systems Engineering and Design (IHSED2025): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED2025): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-74-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →