Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Building an assurance case for assessing novel maritime border surveillance system

VTT Technical Research Centre of Finland Ltd, 02150 Espoo, Finland ↗
Publication year
Digital Object Identifier10.54941/ahfe1006797

Abstract

An assurance case is a documented body of evidence providing a convincing and valid argument that a system is adequately built for a given application in a given environment. It is a requirements-based approach, in which requirements provide a reference for the assessment of determined features of the target system. System adequacy is demonstrated by presenting evidence and by justifying why the evidence supports particular requirement-based claims. As follows, a systems assurance case is a conceptual procedure within which reasoning about system validity takes place. It is a hierarchical ordering of system-related data extending from an abstract understanding of system performance to a concrete proof of the validity system. In this paper, we discuss the development of an assurance case for a maritime border surveillance system which aims at enhancing the situational awareness of border authorities at external maritime borders of the European Union and third countries. This paper discusses the benefits of the assurance case in the context of applied civil security research, considering among others the suitability and comprehensiveness of the approach compared to other methods, such as checklists and compliance matrices. The strength of the approach in organizing information associated with complex systems is also addressed.

ASSURANCE CASE IN SYSTEM EVALUATION Assurance cases are typically constructed to substantiate claims regarding various system attributes, including safety, reliability, maintainability, human factors, operability, and security, and they are applicable to any characteristic of a system. Based on the ISO standard, an assurance case is “auditable artefact that provides a convincing and sound argument for a claim on the basis of tangible evidence under a given context” (ISO/IEC/IEEE 15026-2:2022). Compared to assurance based on guidelines or standards, which primarily define the required evidence, assurance cases introduce the significant innovation of an explicit argument, which links pieces of evidence to claims. This approach can enable assurance cases to be more precisely tailored to the unique conditions of a system and offer greater flexibility in adjusting to new techniques and applications (Rushby 2015).

© 2025. Published by AHFE Open Access. All rights reserved. 437

An assurance case is a document that describes the process and methodology used to provide assurance on the system requirements. To ensure the applicability of the method, a high-level overview of the system, its components and a description of the operating environment should be provided. Cobos et al. (2022) have examined the use of the method addressing the safety of autonomous marine vessels. Any assumptions related to, for instance, development methodology, user interface, or external dependencies, should be taken into account to achieve an acceptable outcome.

Systems and infrastructures analyzed using assurance case are typically large, complex, risk-intensive, and increasingly software-intensive. Regulatory authorities and stakeholders e.g., in licensing processes acknowledge the advantages of assurance case implementation, as it enhances the analytical thoroughness by collecting evidence of important system attributes from various sources. These include among others risk assessments, incident reports, human factors verification and validation, and operational experience. Further benefits of the assurance case include integrating evidence sources, facilitating communication among stakeholders, clarifying implicit issues, and supporting management and governance (Sklyar et al., 2020).

To demonstrate that a system is acceptably safe to operate, it is common to present a safety case for that system. By definition, a safety case is an assurance case addressing safety aspects of a particular system. For systems incorporating software components, the safety case must examine the contribution of the software to the safety of the whole system. Hawkins et al. (2023) have tested safety assurance process to wildfire detection and alerting based on machine learning and satellite data. Safety claims have been broken down into sub-claims, each supported by evidence.

Valkonen et al. (2016) have outlined the features of a good safety demonstration for human-made systems. Although a safety case comprises other documents (artefacts) beyond a safety demonstration, the features of a good safety demonstration provide a solid foundation for a well-made safety case. Clear ownership is crucial for a safety demonstration. The entity having ultimate responsibility for the safety of the plant and its personnel should have clear accountability. The right experts, representing diverse perspectives, should be involved in creating the safety demonstration. The safety demonstration should focus on issues important to key users and stakeholders, such as main hazards and risks, and the level of detail should correspond to the hazards and complexity of the subject. Finally, a useful safety demonstration must be accessible, easy to understand and compact.

Safety cases were initially developed for assuring plant safety in the chemical industry. Their application has since expanded to railway systems, defense systems, nuclear plants, automotive functional safety, and medical devices. In the field of safety case research, many studies have addressed the issue of effectively composing safety cases by reusing or improving previous cases. Napolano et al. (2015) have exploited assurance case pattern and created a post-failure safety case to structure knowledge extracted from accidents or failures and prevent similar accidents. Organizing the knowledge

Building an Assurance Case for Assessing Novel Maritime Border Surveillance System 439

gained from accidents or failures is significantly relevant to prevent similar incidents in both the same and different organizations.

The objective of this paper is to report an assurance case generated for a maritime border surveillance system developed within the context of a jointly funded European research project. The system intends to improve border guard situational awareness and the performance of surveillance tasks at EU’s external maritime borders.

EURMARS ASSURANCE CASE Short Project Overview

The assurance case presented in this paper is generated for a multi- annual EU-funded project called EURMARS1 - An advanced surveillance platform to improve the EURopean Multi Authority BordeR Security efficiency and cooperation. The project aims to create a next-generation surveillance platform for improving European maritime border security through enhanced sensor effectiveness and multi-authority cooperation. Funded under Horizon Europe’s civil security research, the project produces a system-of-systems integrating coastal ground sensing, high-altitude sensing, and satellite systems, managed by border guards and coast guards. The project targets technology readiness levels 7 or 8 for its final outputs, indicating for example a system prototype demonstrated in real-life operational environment. Diverse pilot use cases (PUCs) validate the solution in real-world scenarios, such as maritime and land border surveillance, maritime search and rescue, and monitoring of maritime structures and oil spills. The project launched in October 2022 and ends in September 2025.

Adopting Assurance Case for the Project An assurance case aims to demonstrate what is learned about the overall acceptability and validity of the targeted system throughout the assessment process. The construction of an assurance case proceeds through two kinds of activities, 1) reasoning in two stages and 2) practical testing of the system. In the first reasoning stage before real-life testing, a goal structure is formulated. In the second stage after the tests, a claim structure is established, providing explicit arguments about how the test evidence confirms or contradicts the fulfilment of the claims.

The assurance case adopted for this work is based on EURMARS requirements development (see Salmela et al., 2024) and project system testing at living labs and demonstrations in distinct maritime operational environments across Europe. At the time of writing this paper, all system testing and demonstrations have been completed in four locations within a time frame of approximately two years (i.e., between early autumn 2023 to summer 2025). This includes the organization of two living labs in Bulgaria (years 2023 and 2024), the first demonstration event in Cyprus (2024) and the second demonstration event in the UK (2025). The final demonstration of the project was held in June/July 2025 in Bulgaria at a separate maritime 1https://cordis.europa.eu/project/id/101073985

site in contrast to the earlier living labs. The overall length of each event was approximately one working week, including the set-up, configuration and dismantling of tested technologies and system modules and the execution of the pilot use cases.

Figure 1 illustrates the general assurance case framework adopted for the EURMARS project. To create and manage the assurance case, Adelard’s ASCE (the Assurance and Safety Case Environment) software was adopted. ASCE’s visualization capabilities are claimed to support complex data analysis and user comprehension of the assurance argument as a whole.

Figure 1: General assurance case framework (modified from Koskinen et al., 2021).
Original PDF Figure 1:

The adopted framework is two-sided consisting of a goal structure and a claim structure (modified from Koskinen et al., 2021). The former introduces the reference base for system evaluation, while the latter comprises a set of evidence, arguments, and claims which organize acquired evaluation test information systematically and meaningfully and elucidate the reasoning behind the analysis of the test results. The framework is modified from the standard assurance case model for enhanced consideration of human engineering requirements.

Building an Assurance Case for Assessing Novel Maritime Border Surveillance System 441

The high-level acceptance goal indicated in the above framework describes the evaluation’s primary objective, namely the promotion of demonstrable capabilities and performance of a border surveillance system. The high-level acceptance goal divides into a hierarchy of several sub-level goals, that is, into general and specific requirements. In constructing the goal structure, the specific requirements are thematically organized under different high-level requirements. All specific requirements can be verified (i.e., does the system provide the required capabilities?), but only a certain proportion of them is feasible to validate through tests and demonstrations (i.e., does the system bring added value to border guards performing various border surveillance tasks?).

The acceptance criteria are derived from the requirements and acknowledge the defined pilot use cases as well and the possibilities of measuring critical activities during living labs and demonstrations. As most EURMARS requirements are based on user needs, the acceptance criteria are attentive to the preconditions for the system’s ability to support user task execution. Both objective indicators of task performance (e.g., time, fluency, number of errors) and subjective measures (e.g., situational awareness, mental workload, user experience) ground the acceptance criteria.

An item of evidence describes user or system performance in the context of a particular operational test occasion, scenario or a use case. Both positive and negative evidence can be considered as input to the assurance case. Positive evidence includes signs of unexpected benefits and evidence of future potentials, whereas negative evidence is any indication of problems or inefficiencies regarding system performance, human-system interaction or user performance (Koskinen et al., 2021). Completely new evidence may also emerge that have limited or no association to any predefined acceptance criteria.

A claim represents a generic property of the design solution or its use, defined prior to design. Claims regarding what constitutes a “good” system serve as a benchmark against which the acquired evidence is assessed (Koskinen et al., 2021). An argument explains how the evidence explicitly confirms the fulfilment of a claim, that is, it describes a generic mechanism in the use of the system that may cause the specific consistency or discrepancy. The arguments bridge the gap between a specific piece of evidence and a more generic claim.

RESULTS

In the following, the EURMARS high-level goal structure is depicted utilising the visualization generated with the ASCE software. The following limitations should be taken into account: Firstly, the visualization includes a partial representation of the whole assurance case as the case was incomplete at the time of paper submission. For example, the analysis of human engineering consistencies or discrepancies was unfinished. Secondly, the claim structure of the assurance framework is in focus as the applied framework aligns well particularly with ASCE’s claims-argument-evidence notation (i.e., ASCAD 1.3). Finally, as the EURMARS surveillance platform comprises a

complex system-of-systems, describing the complete assurance case within the limits of a conference paper would be neither desirable nor feasible.

Figure 2 depicts the claim structure of the assurance case in a hierarchical order with the acceptance claim shown at the top and the high-level and specific claims at the bottom of the image. Furthermore, in the selected layout, sources of evidence are placed at the sides providing additional clarity to the figure. In the diagram, claims, the argument and evidence are written in phrases to better describe the application domain instead of reduced expressions (e.g., C, A, E).

Figure 2: Visualization of EURMARS assurance case.
Original PDF Figure 2:

The acceptance claim derives from the primary objectives of the project defined in the EURMARS Grant Agreement (GA), that is, enhancing border authorities’ surveillance capabilities within maritime environment at external borders. The project’s key objectives are not limited to this, but others can be inferred. However, as explained above, in this paper, we focus only on a proportion of the entire assurance case. Together with project GA, the assurance case exploits information from various sources, such as the requirements specifications, the pilot use case definitions and the early versions of system.

As the assurance case is based on requirements specified for the project, the high-level claims mirror the high-level requirements in their definitions. For each individual requirement, some acceptance criteria are specified, and the target system is evaluated against these criteria. Moreover, the argument is primarily based on the requirements and associated with the defined pilot

Building an Assurance Case for Assessing Novel Maritime Border Surveillance System 443

use cases for the project. For specific claims, only identification numbers are given. This is due to the confidential nature of the requirement specifications in the project. Furthermore, providing more detailed information about specific claims would unnecessarily clutter the visualization chosen for this paper.

Finally, below the specific claims, several additional sub-claims could be included according to the EURMARS requirements hierarchy. For example, the high-level requirement group relating to decision-support includes close to forty sub-requirements of which several sub-claims could be derived (i.e., 9 sub-requirements associated with specific claims 3,1–3,4 and 30 with specific claims 5,1–5,11). Overall, the EURMARS requirements development resulted in specifying over 140 requirements (Salmela, 2024a). However, as noted above, for visual and textual clarity and due to confidentiality reasons, these elements were omitted from the figure.

As noted above, project implementation of the high-level and specific claims is supported with various evidence. Arguments, in turn, demonstrate how the collected evidence either supports the fulfilment of the claims or rejects them. For the claims addressed in this paper, the evidence originates from three sources: technical benchmarking specific to EURMARS components and sub-systems; analysed end-user feedback collected during project living labs and demonstrations; and finally, analysed evaluator observations conducted during the same events. An assurance case presents a structured argument that interactions with the intended environment of use, including the services provided, are both intentional and clearly specified (ISO/IEC/IEEE 15026-2:2022). The operational demonstrations and system tests with defined use cases in EURMARS served this purpose. The overall assessment methodologies and specific data collection and analysis methods vary across evidence types, representing also different (scientific) fields particularly with regards to the implemented EURMARS technologies (e.g., measuring technical performance of object detection algorithms vs rule-based decision-support relying on textual data).

For end-user feedback, a survey was implemented, while on-site evaluator observation consisted of gathering information about the execution of pilot use cases from technical, operational and user perspectives and documenting platform use with different means (e.g., screenshots, recordings). Moreover, tested/demonstrated technologies, different phases of the events and event environmental conditions were recorded by taking photographs and videos. To conclude, in the project, the responsibilities were shared between partners: system developers were responsible of technical verification and benchmarking and human factors experts gathered end-user experiences and feedback.

DISCUSSION

System evaluation forms a standard part of any technology development project, and approaches for its adoption and implementation vary across the projects. The assurance case provides a structured argumentation of system assessment. Its main value lies in systematic organisation of

evaluation evidence and in increasing the comprehensibility of evaluation results. Required information for the assurance case is gathered and updated throughout a project’s lifecycle, along with modelling work. Evaluation is performed at different project events (i.e., living labs, demonstrations) which generate various types of data for the assurance case.

The modified EURMARS assurance case model provided sufficient means to evaluate a border surveillance system holistically by addressing comprehensively both technical and human engineering requirements. The formulated assurance case facilitated an in-depth assessment of the claims and the achievement of defined acceptance criteria. The project’s iterative development process gradually accumulated the information needed for the assurance case, including requirements specifications, pilot use case definitions, early descriptions of developed technologies, demonstration plans, and evaluation exercises.

As highlighted in the examined literature, implementing assurance case is a time and expertise-consuming process, particularly when aiming to be comprehensive and detailed. This may question its benefits when compared against the resources used. In cases of assessing regulatory compliance of an IT system, investments on thorough assurance cases are justifiable. However, when applied voluntarily, risks associated with failures to comply with specified requirements need to be reflected and alternative methods to address the requirements have to be developed. In a research context, applying rigorous assurance case may pose its own challenges, as identified technical or human engineering discrepancies may form justifiable research results on their own (e.g., by generating new knowledge about the feasibility of certain technologies in a novel application domain). Notwithstanding, systematic organisation of evaluation information significantly supports future research aims by directing and scoping subsequent activities in potential follow-up projects.

Assurance case could be applied partially, covering only a certain subsystem(s), or it could be conducted in phases. In EURMARS, gathering evaluation information also assisted iterative systems development in parallel as developers remedied or enhanced discovered discrepancies presenting a revised version of the system in the next test or demonstration events of the project. Isolating the assurance case from iterative systems development might bring more methodological clarity to the process for example by reducing repetitive or redundant evidence collection or by lowering the need to adjust selected evidence collection methods. However, information generated within the evaluation process is valuable from many perspectives and useful for different purposes. Technical development often sustains to the end of a research project with final demonstration(s) executed in the last months. Thus, completing the assurance case may become a significant challenge, as for example the processing of technical benchmarking data, generated in vast quantities by various sub-systems, might continue until the concluding month of the project. Moreover, during the various events, the system components were constantly configured and adjusted for example to local conditions along with suddenly or otherwise occurring disruptions or performance gaps being fixed on-site on-the-fly.

Building an Assurance Case for Assessing Novel Maritime Border Surveillance System 445

Applying assurance case into novel application domains may generate new knowledge and understanding about the domain itself and support method development alongside. However, based on reviewed literature and experiences gained in the EURMARS project, harmonising different elements of the assurance case and their associated practices could be improved. For example, the means of obtaining verifiable and validated evidence for requirements compliance within a particular field could be standardised. Otherwise, assurance cases may remain difficult to generalise across systems designed for similar users and consisting of same technologies, would they be used for border or other surveillance purposes.

CONCLUSION AND FUTURE WORK In this paper, we have described an assurance case generated for a maritime border surveillance system developed within an EU-funded civil security research project.

As the characteristics and needs of various border surveillance projects and implementations greatly differ at European level, the assurance case approach could facilitate in organising and illustrating system complexities and thus enhance understanding interconnections and outcomes of the developed system.

More broad application of the approach to the development and operation of current and future border surveillance systems requires practical guidelines and templates for implementing assurance case-based system evaluation. These should cover among others used terminology, processes involved, roles of contributing participants, common methods for evidence collection and required documentation. Hands-on instructions on adopting the approach to different project types and use environments should be included.

Finally, consolidating the approach with iterative systems development is needed, allowing the evaluation results to be efficiently attended also in technical development. Assurance case is especially suitable for continuous, multi-staged evaluation of border surveillance systems where systematic methods are needed for the consolidation and systematization of evaluation data and drawing conclusions from the evidence. The multi-staged evaluation is carried out in several stages in sequence so that cumulative evidence of the safety, reliability, usability etc. of the proposed system is achieved. The multi- staged approach is well suited to continuous iterative systems engineering process, in which system characteristics are incrementally discovered during the design process. There is a cyclic ongoing specification of system requirements and design solutions so that system requirements and design solutions are concurrently developed.

ACKNOWLEDGMENT

This work has received funding from the European Union’s Horizon Europe research and innovation programme under grant agreement No 101073985.

REFERENCES

Cobos, L.-P., Miao, T., Sowka, K., Madzudzo, G., Ruddle, A. R., El Amam, E.

(2022) Application of an Automotive Assurance Case Approach to Autonomous Marine Vessel Security. 16–18 Nov 2022, International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME). Hawkins, R., Picardi, C., Donnell, L., Ireland, M. (2023) Creating a Safety Assurance Case for a Machine Learned Satellite-Based Wildfire Detection and Alert System. Journal of Intelligent & Robotic Systems, 108: 47. ISO/IEC/IEEE 15026-2:2023. Systems and software engineering – Systems and

software assurance – Part 2: Assurance case. ISO. SFS. 20 p. Koskinen, H., Laarni, J., Norros, L., Liinasuo, M. & Savioja, P. (2021) Systems Usability Case in Stepwise Control Room Validation. Safety Science, 134. Napolano, E., Machida, F., Pietrantuono, R., Cotroneo, D. (2015) Preventing

recurrence of industrial control system accident using assurance case. 2–5 Nov 2015, IEEE International Conference on Software Reliability Engineering Workshops (ISSREW). Rushby, J., Xu, X., Rangarajan, M., Weaver, T. L. (2015) Understanding and Evaluating Assurance case. NASA/CR-2015-218802. Salmela, L., Toivonen, S., Laarni, J., Väätänen, A. & Keränen, J. (2024). Participatory

Approach for Specifying User Requirements for Maritime Border Surveillance System. In: Tareq Ahram, Waldemar Karwowski, Darko Etinger, Tea Mijaˇc (eds) Human Systems Engineering and Design (IHSED2024): Future Trends and Applications. AHFE (2024) International Conference. AHFE Open Access, vol. 158. AHFE International, USA. Sklyar, V., Kharchenko, V. (2020) Assurance case for safety and security implementation: A survey of applications. International Journal of Computing, 19: 4, 610–619. Valkonen, J., Tommila, T., Linnosmaa, J. (2016) Safety demonstration of nuclear I&C – An introduction, SAUNA Task 3.1 report. VTT Research report.

Cover of the proceedings volume Human Systems Engineering and Design (IHSED2025): Future Trends and Applications

Explore the complete digital volume

Human Systems Engineering and Design (IHSED2025): Future Trends and Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-74-8

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →