Paul M. Salmona, Michael G. Lenneb, Gemma Readb, Guy H. Walkerc, Neville A. Stantond (2021). Pathways to Failure? Using Work Domain Analysis to Predict Accidents in Complex Systems. In: Advances in Human Aspects of Transportation: Part II. AHFE Open Access, vol. 16. https://doi.org/10.54941/ahfe100688
1 University of the Sunshine Coast Accident Research, Maroochydore, QLD, Australia ↗
2 Monash University Accident Research Centre, Monash University, Clayton, Victoria, Australia ↗
3 School of the Built Environment, Heriot-Watt University, Edinburgh, UK d Civil, Maritime, Environmental Engineering and Science, Faculty of Engineering and the Environment, University of Southampton, Southampton, UK ↗
4 Civil, Maritime, Environmental Engineering and Science Faculty of Engineering and the Environment University of Southampton, Southampton, UK ↗
Forecasting accidents before they occur is the final frontier for safety science. Although this has long been recognized, the discipline of human factors has yet to produce an appropriate methodology for achieving this. This paper presents some of the findings from an exploratory study in which the abstraction hierarchy method from the work domain analysis phase of cognitive work analysis was used to predict potential accidents. Using rail level crossings as a test case, the exploratory study revealed that the abstraction hierarchy method was able to predict a range of failure pathways that could potentially lead to a collision between a road user and a train at rail level crossings. In addition, certain features of the abstraction hierarchy method were found to make it highly consistent with contemporary systems level views on accident causation, including that it provides a systems level analysis of potential accident pathways, that is does not support a focus on broken human components (since the abstraction hierarchy model is actor independent), and that the primary focus is on the relationships between components rather than the components themselves. Further testing of the approach is recommended, including sensitivity and validity testing whereby the predictions made are compared to real world events.
Forecasting accidents before they occur is the final frontier for safety science (Moray, 2008, Salmon et al, 2011, Stanton and Stammers, 2008). It is the breakthrough needed in order to make the next step in reducing risks in human-technology systems. Although there have been various attempts at developing accident prediction models (e.g. Deublein et al, 2013), most are statistical models that are unable to identify and describe how behavior across overall sociotechnical systems might combine to create failure scenarios. Despite decades of research into accident causation and prevention, we do not yet have a methodology that can be used to determine how interactions across sociotechnical systems will lead to emergent behavior that creates accidents. Using rail level crossings as a test system, the aim of this paper is to take the first steps in filling this critical capability gap.
Why rail level crossings? Worldwide, the problem of collisions between road users and trains at rail level crossings remains resistant to current countermeasures. In Australia, between 2000 and 2009, there were 695 collisions between road vehicles and trains at rail level crossings, resulting in 97 fatalities (Independent Transport Safety Regulator, 2011). Despite various initiatives, in 2011 there were 49 collisions between trains and road vehicles at rail level crossing in Australia, leading to 33 fatalities (ATSB, 2012). In addition to the high levels of trauma and personal cost, the financial costs associated with rail level crossing collisions are significant. In Australia they have been estimated to cost around $24 million a year. The problem is not exclusive to Australia, with unacceptable incident, fatality, and injury rates worldwide. Across Europe, for example, there were 604 fatalities and serious injuries at rail level crossings in 2011(European Railway Agency, 2013). Clearly rail level crossings represent a significant safety issue worldwide and one that has so far proven intractable to safety interventions. This makes them the perfect test case for this accident prediction exercise.
The problem is exacerbated by the fact that the intensity of operations in rail is increasing. The railway industry in Australia is a nationally important infrastructure asset and a key enabler of economic growth. The number of trips made by rail in Australia is increasing, and in 2009 equated to over 180 million train kilometers travelled (National Transport Commission Australia, 2011). Risk levels have improved but maintaining these levels, and improving them further, has to occur in a context of shifting system boundaries resulting from both higher operational intensity and risk exposure. The current approach to preventing rail level crossing incidents is reactive and involves waiting for the next catastrophe to occur before implementing new safety initiatives such as advanced warning devices and boom barriers. Are we willing to accept that this is ‘as good as it gets’? Clearly not, but in order to make continued progress the problem-space needs to be looked at afresh.
Part of the problem is that, despite having access to a range of valid and exhaustive systems analysis methods, we are still only able to exhaustively describe failure scenarios after they have happened (e.g. Salmon et al, 2013). This is particularly the case with rail level crossings, where pro-active risk assessment tools currently in use have been questioned (Salmon et al, 2013). The step change required in accident research is the ability to predict entire accident scenarios, not just individual human errors. This paper reports some of the initial findings from an exploratory study that aims to solve this problem. We present a study in which we explored the ability of the Work Domain Analysis (WDA) phase of the popular Cognitive Work Analysis framework (CWA; Vicente, 1999; Jenkins et al, 2008) to predict the causes of collisions between rail and road users at rail level crossings. Specifically, an abstraction hierarchy model of an ‘active’ rail level crossing (i.e. where road users are controlled by active warnings such as boom gates and flashing lights) was used to predict the range of failure scenarios that could occur where the end result is a collision between a road user and a train.
A Systems Approach to Accident Prediction There is a range of well used and tested methodologies available that can be used to predict the kinds of human errors that might lead to accidents (see Stanton et al, 2013). Indeed, various error prediction methods have been shown to achieve acceptable levels of reliability and validity (e.g. Stanton et al, 2009). The problem with these methods is that they predict what is likely the last behavior in a long and complex network of interacting and emergent behaviors occurring across various parts of the system in question. Whilst it is of course useful to examine what errors a human operator might make in a given system, accident prevention efforts are far better served by looking at the interactions that occur before the human operator makes the error. In short, it is the entire accident scenario, including interacting factors and emergent behaviors that is important for understanding how to prevent accidents.
Accidents are a systems phenomenon; they are emergent properties of complex sociotechnical systems. Effectively predicting accident scenarios therefore requires a systems approach. The systems approach argues that safety, and indeed accidents, are emergent properties arising from non-linear interactions between multiple components distributed across complex sociotechnical systems (e.g. Leveson, 2004). It has a long legacy in safety science, from the foundational work of Heinrich (1931) through to the evolution of a number of more recent accident causation models and analysis methods (e.g. Leveson, 2004; Rasmussen, 1997; Reason, 1990). As it stands currently, the most up-to-date methods and approaches rely on a form of systems thinking closely linked to current debates in the science of complexity.
Complexity is formally described in a number of ways (e.g. Walker et al., 2009), central to which is the property of emergence (e.g. Waldrop, 1992). Emergence is behavior that arises from dynamic human-technical systems which rely on a continuous throughput of information, resources, actions and behaviors. In everyday terms, emergence is the ‘more than the sum of its parts’ effect of synergy and the ability to yield disproportionately favorable outcomes from small, targeted interventions. It is also the property that gives rise to system errors and other pathologies (e.g. Perrow, 1999; Reason, 1990). Decisions and actions at all levels of a complex system interact with one another to shape system performance: safety and accidents are thus shaped by the decisions of all actors, not just the front line workers in isolation, and accidents are caused by multiple contributing factors, not just one bad decision or action. In either case, emergence describes behavior that is not readily deducible from its low level properties and cannot always be detected via reductionist, non-systemic approaches.
Existing error prediction methodologies can be thought of as reductionist (although they are not entirely reductionist as they do focus on man-machine interactions). Reductionist approaches to safety science, those which rely on taking the system apart in order to understand the components, then reassembling the components back into the complete system (on the tacit assumption that the whole cannot be greater or less than the sum of its parts) do not allow us to detect the emergent properties associated with the types of strategic risk issues we wish to make progress on (see Walker et al., 2009). Systems approaches do. One means by which they can enable forecasting and prediction is to consider the causal texture of the systems environment, and the system’s movement through that environment. Rasmussen (1997) describes a set of ‘boundaries’. There is a boundary of economic failure: these are the financial constraints on a system that influence behavior towards greater cost efficiencies. Then there is a boundary of unacceptable workload: these are the pressures experienced by people and equipment in the system as they try to meet economic and financial objectives. The boundary of economic failure creates a pressure towards greater efficiency, which works in opposition to a similar pressure against excessive workload. Because transport systems involve human as well as technical elements, and because humans are able to adapt situations to suit their own needs and preferences, these pressures inevitably introduce variations in behavior which are not explicitly designed and can lead to increasingly emergent system behaviors, both good and bad (Qureshi, 2007; Clegg, 2000). Over time this adaptive behavior can cause the system to cross safety boundaries and accidents to happen (Qureshi, 2007; Rasmussen, 1997). The key, then, is to detect in advance a) where those boundaries are and b) where the system is travelling in relation to them.
The systems approach has become popular in part because of various systems analysis methodologies that can, to some extent at least, do this (e.g. Rasmussen, 1997). These methods, for example Accimap (Rasmussen, 1997), are becoming increasingly popular for accident investigation purposes in safety critical domains such as aviation, rail, maritime, and process control. This arises from recognition among practitioners and industry stakeholders that new approaches are needed to tackle issues that are proving resistant to previous approaches. The major limitation of current systems-based accident analysis methodologies is that, so far, they cannot be used in a pro-active manner: organizations are effectively waiting for loss events to occur before they can work on prevention strategies. This is not acceptable. The lack of data resulting from improved safety trends combined with greater operational intensity and risk exposure means that, if anything, loss events are more likely to be large-scale and unexpected, meaning that ‘learning from disasters’ is becoming increasingly dubious from an ethical perspective. The need for systems-based prediction approaches is discussed extensively in the literature (e.g. Moray, 2008, Salmon et al, 2011, Stanton and Stammers, 2008) but as yet a credible approach has yet to emerge.
Cognitive Work Analysis (CWA): A Framework for Predicting Vulnerability at Rail Level Crossings CWA (Jenkins et al, 2008; Vicente, 1999) is a systems analysis and design framework that is used to identify the constraints imposed on activities and then to design new systems that better support the activities of interest. To date CWA has been used in a range of systems analysis and design activities in various safety critical domains. Although the framework is becoming popular in safety applications, applications examining accident causation are limited in number.
The first CWA phase, Work Domain Analysis (WDA), is used to provide an event and actor independent description of the system under analysis, in this case rail level crossing systems. The aim of the WDA phase is to describe the purposes of the system and the constraints imposed on the actions of any actor working within that system (Vicente, 1999). The system under analysis is described at the following five conceptual levels using the abstraction hierarchy method:
1. Functional purpose – The overall purposes of the system and the external constraints imposed on its operation;
2. Values and priority measures – The criteria that organizations use for measuring progress towards the functional purposes;
3. Generalized functions – The general functions of the system that are necessary for achieving the functional purposes;
4. Physical functions – The functional capabilities and limitations of the physical objects within the system that enable the generalized functions; and
5. Physical objects – The physical objects within the system that are used to undertake the generalized functions.
The output is a detailed description of the system under analysis in terms of the constraints influencing behavior and the physical objects (and their affordances) and functions that enable the system to achieve its functional purpose. Importantly, the abstraction hierarchy model that is used for the WDA phases uses means-ends relationships to link nodes across the five levels of abstraction. Every node in the abstraction hierarchy should be the end that is achieved by all of the linked nodes below it, and also the means that (either on its own or in combination with other nodes) can be used to achieve all of the linked nodes above it. This representation of the relationships between nodes can be thought of as one way of representing interactions between system components (e.g. object x and y are used together to achieve process z, and process z happens with process a to achieve function b) This way of describing complex systems appears to be suited to identifying the emergent behaviors that might lead to accidents. As all of the functions required to achieve the functional purposes of the system are described, it is possible to identify which functions can potentially create adverse outcomes. This can be through either the function not being achieved or through emergent behavior arising from achievement of the functions. For example, in the rail level crossing context, if the function ‘Maintain road and rail user separation’ is not achieved, then the outcome is a collision between a road user and the train. It is then possible to examine the other functions (or combinations of other functions) that might have contributed to the failure to maintain separation. For example, should the function ‘Alert (road user) to presence of train’ not be achieved, then a failure to maintain separation between the road users and train is the likely end result. Moving down the abstraction hierarchy, it is then possible to explore the affordances and physical objects that can potentially play a role in the function not being achieved. This enables both the individual behaviors and combinations of behaviors that could lead to the failure to alert the road user to the presence of a train.
An additional feature of the abstraction hierarchy is that it is actor and event independent and so does not focus on specific human actors when identifying accident pathways (e.g. road users, pedestrians, train drivers). This is a significant departure from existing human error prediction and accident analysis methodologies, which tend to focus first on human actors as the source of error and accidents. The focus on interactions between components rather than the behavior of specific human components is a feature of the WDA approach that gives it a high degree of consistency with contemporary accident causation models (e.g. Rasmussen, 1997). Dekker (2011), for example, argues that the focus on broken human components is one of the major flaws in accident and safety-related research.
By examining failed functions and the related means-ends links, all potential pathways to failure can be explored. This is represented in Figure 1, which shows selected failure pathways leading to the function ‘Alert (road user) to presence of train’ not being achieved.
Figure 1. Work domain analysis extract showing accident pathways linked to failed ‘alert to presence of train’ function.
In Figure 1, any combination of the highlighted nodes at the physical object and object-related processes level failing or being absent at a particular rail level crossing environment could lead to the function ‘alert (road user) to presence of train’ not being achieved. For example, at a particular rail level crossing if there are no active advanced warning assemblies or boom gates, and the road user did not perceive the flashing lights or hear the bells, then the object-related processes ‘visual warning of approaching train’ and ‘audible warning of approaching train’ have failed, which in turn means the function ‘alert to presence of a train’ will not be achieved. This is precisely the scenario that occurred in the Kerang rail level crossing tragedy of 2007 in which the driver of a loaded semi-trailer truck continued toward a rail level crossing apparently unaware that a passenger train was also approaching. The resulting collision killed 11 train passengers and injured a further 15 people, including the truck driver (see Salmon et al, 2013).
METHOD
Work Domain Analysis Development An abstraction hierarchy was developed for a generic ‘active’ rail level crossing. Active rail level crossings are those that have ‘active’ warning devices which provide a warning of approaching trains, such as boom gates, flashing lights, and bells. The purpose was to construct a WDA model of active rail level crossings generally, rather than for one specific rail level crossing system. The abstraction hierarchy was developed based on data collected from a range of activities, including on-road studies of driver behavior at rail level crossings (e.g. Young et al, 2014), interviews with road users regarding their behavior at rail level crossings, interviews with train drivers, surveys of road user behavior at rail level crossings (e.g. Beanland et al, 2013) and a workshop involving a range of rail level crossing subject matter experts (e.g. designers, rail safety practitioners, regulators, road safety practitioners, engineers). A summary of the abstraction hierarchy is presented in Figure 2.
Prediction of Failure Scenarios One analyst with significant experience in accident analysis, error prediction, and applying CWA predicted the failure scenarios. This involved examining functions at the purpose-related functions level of the abstraction hierarchy to identify those that, if not achieved, could create a collision between a road user and train. The next step involved examining functions or combinations of functions that could potentially create the identified ‘collision causing failed function’. Next, the means-ends links were used to trace all object-related processes and physical objects related to these functions. In addition, new links down from the purpose-related functions level to the physical objects level were explored. Once relevant object-related processes and physical objects were identified, a simple failure mode taxonomy was applied to these objects and processes to determine all of the ways in which they might not work or be achieved. Each failure pathway was then rated in terms of probability of occurring (low, medium and high) and criticality (low, medium and high). This is based on the approach used by error prediction methods such as the Systematic Human Error Reduction and Prediction Approach (SHERPA, Embrey, 1986) and enables the most likely and critical scenarios to be focused on during accident prevention efforts. A representation of a simple failure pathway is presented in Figure 3.
RESULTS
Due to space constraints it is not possible to show the entire rail level crossing abstraction hierarchy. A summary is therefore presented in Figure 2. Figure 2 shows in full the functional purpose, values and priority measures, and generalized functions levels from the abstraction hierarchy, along with categories of the nodes found at the physical objects and physical functionality levels.
Figure 2. Summary of active rail level crossing abstraction hierarchy.
As shown in Figure 2, the primary function that is linked to a collision between a road user and a train is the function ‘maintain road and rail user separation’. Should this function not be achieved, then a collision between road user and train is the outcome. Of more interest are the functions and combinations of functions that can create the failure to maintain road and rail user separation. For example, if the function ‘alert to presence of a train’ is not achieved, then maintenance of separation may not be achieved. In turn, other functions can combine to create the failure to alert road users to the presence of a train. For example, a inadequate maintenance (maintain infrastructure function) combined with a lack of performance monitoring might lead to the warning devices at the physical object level not functioning properly, and in turn a failure to provide a warning at the next level up. This failure to provide a warning then creates the failed ‘alert to presence of train’ function.
This example failure pathway is presented in Figure 3.
Figure 3. Example failure pathway causing a collision between a road user and train.
Extracts of the critical failure pathways that could potentially lead to a collision between a road user and train is presented in Table 1.
Table 1. Example failure pathways
A range of failure scenarios were identified. These will be presented in full during the conference presentation. The key findings from the exploratory study were that: a. The abstraction hierarchy is able to identify a range of failure pathways that will lead to a collision between a road user and train; and
b. The failure pathways identified include instances where the ‘maintain road and rail user separation’ function is not achieved. The findings show that this function might not be achieved due to failed functions in isolation (e.g. alert to presence of train) or a combination of failed functions (e.g. maintain infrastructure, performance monitoring, alert to presence of a train). The failed functions in turn can be caused by individual physical objects (e.g. flashing lights failed) or combinations of physical objects.
DISCUSSION
This paper presents a summary of some of the findings derived from an exploratory study in which the first phase of CWA, WDA, was used to predict the range of failure pathways leading to a collision between a road user and a train at an active rail level crossing. The findings demonstrate that the approach was able to predict a range of failure scenarios, providing initial evidence that the WDA phase of CWA is capable of predicting accidents in a manner that is consistent with the systems level view on accident causation (e.g. Rasmussen, 1997).
The exploratory study revealed some notable features of this approach when used for accident prediction. First, provided the abstraction hierarchy is exhaustive, the majority of failure pathways present within a particular system can be identified before they happen. This is possible because all of the system components should be described at the physical objects level, and in turn, all of the affordances and functions should be described at the two levels above within the abstraction hierarchy. In addition, new failure pathways that might emerge following the introduction of new technologies or systems can also be identified by adding the new objects to the abstraction hierarchy and running the analysis again. Second, the abstraction hierarchy enables a systems level analysis of the potential pathways to failure. This is attractive because it ensures a level of comprehensiveness, but also because it is very much in line with contemporary models of accident causation which argue that accidents are a systems phenomenon and that the interactions between components are of interest rather than the components themselves. Accidents will be caused by a range of factors across the overall system; the abstraction hierarchy enables the overall system to be examined. Third, the fact that the abstraction hierarchy is actor independent ensures that there is no focus on the human operators as the broken component or major cause of the accidents identified. Rather, the way in which the system shapes human operator behavior is examined (i.e. in this case being unaware of an approaching train). The analysis is focused on objects, affordances and functions within the system and how they might interact to shape human operator behavior.
It is worth noting that the study described was highly exploratory. Further testing of the approach as an accident prediction tool is required. In particular, a validation study whereby the abstraction hierarchy is used to predict accidents in a particular system following which the predictions made are compared to actual accidents over a period of time in that system is recommended. In addition, using the other CWA phases (e.g. control task analysis, strategies analysis) to provide a deeper level of analysis is another pertinent line of inquiry.
In addition to predicting accidents, there is potential for the abstraction hierarchy to be used as an accident analysis template. This involves examining each node in the abstraction hierarchy during the accident analysis and investigation process in order to identify nodes and means and links that played a role in the accident. Specifically, the means ends links related to functions that were not achieved could be traced down the abstraction hierarchy in order to identify what object-related processes or physical objects played a part.
ACKNOWLEDGEMENTS
Paul Salmon’s contribution to this research and paper is funded through the Australian National Health and Medical Research Council post-doctoral training fellowship scheme.
REFERENCES
ATSB. (2012). Australian Rail Safety Occurrence Data 1 July 2002 to 30 June 2012. Report number RR-2012-010.
Australian Transport Safety Bureau, Canberra, ACT. Beanland, V., Lenné, M. G., Salmon, P. M., Stanton, N. A. (2013). A Self-Report Study Of Factors Influencing Decision-Making At Rail Level Crossings: Comparing Car Drivers, Motorcyclists, Cyclists And Pedestrians. In Proceedings of the Australasian Road Safety Research, Policing and Education Conference 2013, Brisbane, Australia Clegg, C. W. (2000). Sociotechnical principles for system design. Applied Ergonomics, 31, 463-477. Dekker, S. (2011). Drift into failure: from hunting broken components to understanding complex systems. Ashgate,
Aldershot, UK. Deublein, M., Schubert, M., Adey, B. T., Köhler, J., & Faber, M. H. (2013). Prediction of road accidents: A Bayesian hierarchical approach. Accident Analysis & Prevention, 51, 274-291 Embrey, D.E., (1986). SHERPA: a systematic human error reduction and prediction approach. Paper presented at
the International Meeting on Advances in Nuclear Power Systems, Knoxville, Tennessee. European Railway Agency. (2013). Intermediate report on the development of railway safety in the European Union. European Railway Agency Safety Unit, 15 May 2013. Heinrich., H. W. (1931). Industrial accident prevention: A scientific approach, McGraw-Hill, New York. Independent Transport Safety Regulator. (2011). Level Crossing Accidents in Australia.ITSR, Sydney. Jenkins, D. P., Stanton, N. A., Salmon, P. M., & Walker, G. H. (2008). Cognitive work analysis: coping with
complexity. Ashgate, Aldershot, UK. Leveson, N. G. (2004). A new accident model for engineering safer systems. Safety Science, 42:4, 237—270. Moray, N. (2008). The Good, the Bad, and the Future: On the Archaeology of Ergonomics. Human Factors, 50(3), 411-417. National Transport Commission Australia. (2011). Rail Safety National Law Draft Regulatory Impact Statement,
September 2011 Perrow, C. (1999). Normal Accidents: Living With High-Risk Technologies. Princeton, New Jersey: Princeton University Press. Qureshi, Z. H. (2007). A review of accident modelling approaches for complex critical sociotechnical systems.
Conferences in Research and Practice in Information Technology Series; Vol. 336 Rasmussen, J. (1997). Risk management in a dynamic society: A modelling problem. Safety Science, 27:2/3, 183-
213. Reason, J. (1990). Human Error. New York, NY, Cambridge University Press Salmon, P. M., Read, G., Stanton, N. A, Lenné, M. G. (2013). The Crash at Kerang: Investigating systemic and psychological factors leading to unintentional non-compliance at rail level crossings. Accident Analysis and Prevention. 50, 1278-1288 Salmon, P. M., Stanton, N. A., Lenné, M. G., Jenkins, D. P., Rafferty, L., & Walker, G. H. (2011). Human factors
methods and accident analysis: practical guidance and case study applications. Ashgate, Aldershot, UK. Stanton, N. A. & Stammers, R. B. (2008) Bartlett and the future of ergonomics. Ergonomics 51 (1), 1 – 13. Stanton, N. A., Salmon, P. M., Rafferty, L. & Walker, G. H. (2013). Human factors methods: A practical guide for engineering and design. 2nd Edition, Ashgate, Aldershot, UK. Stanton, N. A., Salmon, P. M., Harris, D., Demagalski, J., Marshall, A., Young, M. S., Dekker, S. W. A. &
Waldmann, T. (2009). Predicting pilot error: testing a new method and a multi-methods and analysts approach. Applied Ergonomics, 40:3, pp. 464-471. Vicente, K.J. (1999). Cognitive work analysis: Towards safe, productive, and healthy computer-based work.
Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Waldrop, M. M. (1992). Complexity: the emerging science at the edge of order and chaos. New York: Simon & Schuster. Walker, G. H, Stanton, N. A., Salmon, P. M. & Jenkins, D. P. (2009). Command and control: the sociotechnical
perspective. Farnham, UK: Ashgate. Young, K. L., Lenné, M. G., Beanland, V., Salmon, P. M., Stanton, N. A. (2014). Drivers’ visual scanning and head check behavior on approach to urban rail level crossings. Applied Human Factors and Ergonomics conference, 19th-23rd July, Krakow, Poland.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Human Aspects of Transportation: Part II
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2098-5
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional