Lucas Paletta, Verena Wagner, K. Wolfgang Kallus, Helmut Schrom-Feiertag (2020). Human Factors Modeling fromWearable Sensed Data forEvacuation based Simulation Scenarios. In: Advances in Applied Digital Human Modeling. AHFE Open Access, vol. 11. https://doi.org/10.54941/ahfe100413
The design and the evaluation of evacuation systems are crucial to guarantee successful responses after an incident. Recent results are presented that target to significantly improve evacuation simulation by parameterizing the human agents’ behavior with information on human factors about stress, perception and decision making. In particular, the single person’s behavior in its specific situational context is investigated in the frame of its embodied decision making. For this purpose, users were equipped with wearable sensors that capture information about the environment, the psychophysiological status of the user, and its viewing (eye tracking glasses) and motion behavior. The studies take place during regularly performed evacuation exercises of large business buildings. From the correlation between the multisensory perceptual and psychophysiological data on the one hand, and the automatically sensed and interpreted situational context on the other hand, we will extract a rule base with a set of logical “pre-condition – action” pairs that will parameterize the crowd simulation model.
Modelling and evaluation of evacuation systems are pivotal to guarantee efficient reactions in the real system in response to a serious incident. The work described in this paper targets to significantly improve evacuation simulation by introducing parameters of human factors, such as stress, perception and human decision making, into the modeling of human agents’ behavior. In particular the single person’s behavior in its specific situational context is investigated. The human agent is outlined not only as an abstract information seeking and processing agent but is modelled in the frame of its embodied decision making. The embodiment is characterized in various aspects and fundamentally impacts the agent’s information processing behavior. Perception induces orientation from which specific information sources are selected. Stress as measured from the body worn sensors, i.e., the analysis from the psychophysiological sensing, impacts attention and from this the information processing as well. In order to consider the embodiment aspect, users were equipped with wearable sensors that capture information about the environment, the psychophysiological status of the user, and its viewing (eye tracking glasses) and motion behavior. The studies took place during regularly performed evacuation exercises of large business buildings. From the correlation between the multisensory perceptual and psychophysiological reactions on the one hand, and the automatically sensed and interpreted situational context on the other hand, we will extract a rule base with a set of logical “pre-condition – action” pairs that will parameterize the crowd simulation model.
In a first stage of the project the wearable multisensory interface was applied to an employee and a visitor of a large company that performed an evacuation exercise. On the basis of the fixation annotated eye tracking videos, the analysis of fixations and saccades as well as the psychophysiological data, events can be classified and associated with typical situational context frames. In particular, the results from the continuously measured gaze patterns of the employee during the evacuation demonstrated that (i) the probability density of gaze on appearances of signage was much smaller than the density on person appearances, (ii) social cues caused the user to change from waiting to acting. Post-event interviews show that the collaborator preferred to follow the guidance of other persons while the visitor followed the guidance from signage.
In summary, the study shows that the data from the wearable multisensory interface provide substantial features to model human factors in evacuation scenarios, this methodology promises better performance of the evacuation simulations.
EVACUATION SCENARIO AND PSYCHOPHYSIOLOGICAL STUDY Evacuation Scenario
Objective of the first human factors study was to evaluate the mechanisms in an evacuation study of a large business building and its industrial environment. The evacuation exercise was applied in a large, eight stories high building, and was performed in the frame of a pre-defined evacuation plan. For the appropriate management of the evacuation, business employees were selected by the evacuation management team and prepared as ‘evacuation supporters’ with the mission to coordinate the navigation of the test persons in the evacuation exercise. Two evacuation supporters have been advised in each story, with one supporter waiting in the intermediate platform of the staircase, and the second had to do an inspection round through all office rooms to collect all employees and motivate them to progress to the staircase. The purpose to run the study, for the business company, was to make employees familiar with the process in an evacuation exercise, detect deficits in the management and the behavior, and possibly find motivation to update the evacuation concept.
During the evacuation exercise, two employees were equipped with wearable sensors and interviewed before and after the evacuation exercise using a standardized questionnaire (N = 2). The participants were accompanied by two experienced observers who monitored and kept notes during the complete experiment. The employee was equipped in addition with eye tracking glasses and a navigation system (Figure 1). After the end of the experiment, a post- interview was performed with the two test persons, at the final destination.
Sample, Material and Study Design Two participants were tested during an evacuation exercise: One employee of the company where the exercise took place and one visitor who has never been in the company building before. The evacuation exercise consisted of five different parts: (1) Normal working situation before the evacuation (‘office’), (2, 3, 4) onset of fire alarm followed by leaving the office, gathering in the stairwell, leaving the stairwell and the building (‘alarm, evacuation’), and (5) gathering at the meeting point outside the building (‘gathering’). A multidimensional approach (Figure 2) was used to combine subjective assessments of the actual psychological state (BSKE; Janke, Debus, Kallus, Hüppe and Schmidt-Atzert, 1989) before and after the evacuation exercise (part 1 and 5), psychophysiological reactions of the cardiovascular and the electrodermal activity to measure activation/arousal and emotional tone, eye tracking (fixations and saccades), and results of behavioral analysis before, during, and after the evacuation exercise (part 1 to 5) as well as results of post-interviews which were executed after the evacuation exercise at the meeting point (part 5). Due to the limited availability of eye tracking glasses only the employee was equipped with them. Therefore, the results that combine psychophysiological reactions and eye tracking are only shown for the employee.
The physiological reactions of the cardiovascular and the electrodermal activity were recorded with the portable VARIOPORT system of Becker Meditec. The psychophysiological measures in the statistical analyses were for ECG: heart rate (HR) in beat per minute (bpm) and heart rate variability (HRV) calculated as mean square of successive differences (RMSSD), and for EDA: Skin conductance level (SCL) and count of non-specific skin conductance responses (NS.SCR) of SCR.
Figure 1. Participant of the user study with wearable sensor interface for human factors analysis.
Figure available in the original PDF paper.
Figure 2. Schematic sketch of the processing of information in the evacuation study and analysis of human factors thereafter.
Figure available in the original PDF paper.
Results
The results of the subjective assessments of the actual psychological state measured before and after the evacuation exercise, showed that after effects of the evacuation exercise are reflected in the actual psychological state only in a lesser extent. The onset of the fire alarm was accompanied by various reactions of the psychophysiological system of the employee (see Figure 3 and Figure 4): An increase of HR, SCL and NS.SCR, which reflect an increase of mental and emotional stress. While waiting in the stairwell and at the meeting place, a decrease of HR, SCL and NS.SCR and an increase of HRV are observable. The psychophysiological reactions of the employee suggest a decrease of mental and emotional stress in these two situations. Therefore, resting points during the evacuation exercise like gathering in the stairwell can be used as "recovery islands" or "safety islands" within an evacuation scenario which could help evacuees to calm down to prevent and avoid panic behavior. Thereby, it is important that these points are clearly signed as points that are “safe”. The results of the analyses of the post-interviews show that going outdoors when leaving the building and arriving at the meeting place is experienced as safe by the participants. Generally, fresh air has a positive effect on the well-being of the participants. While the employee, who was familiar with the building, oriented himself to other evacuees, the visitor used the emergency exit signs for path finding. More research is needed to investigate if this behavior is individual for the two participants of this initial study or can be interpreted as general behavior of evacuees.
Figure 3. Heart rate (HR) – employee of the large office building.Figure 4. Skin conductance level (SCL) - employee of the large office building.
FRAMEWORK FOR MOBILE EYE TRACKING AND HUMAN FACTORS MONITORING For the processing and analysis of visual information we decided to apply eye tracking glasses (ETG) in order to measure the attention process directly during the evacuation task. The mass marketed SMI™ eye-tracking glasses - a non-invasive video based binocular eye tracker with automatic parallax compensation - measures the gaze pointer for both eyes with 30 Hz. The gaze pointer accuracy of 0.5°–1.0° and a tracking range of 80°/60° horizontal/vertical assure a precise localization of the human’s gaze in the HD 1280x960 scene video with 24 fps. An accurate three point calibration (less than 0.5° validation error) was performed and the gaze positions within the HD scene video frames were used for further processing.
In marketing and usability engineering, the tracking of human eye movements, i.e. eye-tracking, has been the central technology for capturing visual attention and motivation (e.g. Buswell 1920). For long times the application of eye- tracking technology was limited to stationary settings due to technical restrictions. Only within the last couple of years miniaturized mobile eye-tracking systems have become available and been successfully applied in different areas of research (e.g. Land and Lee, 1994; Land and Furneaux, 1997; Land et al. 1999; Pelz and Canosa, 2001; Hayhoe et al., 2003; Hayhoe and Ballard, 2005) with the major advantage to evaluate attention in the field where the task of interest is performed.
Mobile eye tracking with automated annotation of the eye tracking video by means of computer vision methodology has recently been introduced for the purpose of identifying visual semantics and relate them to the viewing behavior (Fritz and Paletta, 2010). For a calibrated reference to the environment, (Munn et al. 2008; Voßkühler et al., 2009; Pirri et al., 2011) introduced monocular eye-tracking and triangulation of 2D gaze positions of subsequent key frames of the eye-tracking video. Paletta et al. (2013) presented a straight forward solution of mapping distributions of point-of-regard very precisely onto heat maps within a model of indoors environments, a method that will be used in future specified studies on measuring the perception of evacuation signage in detail.
For the segmentation of the eye tracking video record, the experiment was first investigated in terms of cognitive load and alertness (Holmqvist et al., 2011; Figure 5, Figure 6). From the gathered data and by plotting mean trends overlaid to the original data, we deduce the fact that the evacuation task involved the test person into low cognitive load and medium level of alertness.
For the purpose of evaluating the attention of test persons on the basis of information processing on signage along the way, the location of relevant evacuation signs (Figure 8a) and persons (Figure 8b) was extracted by means of computer vision methodology.
For the purpose of monitoring all data channels from the multisensory stream of, the FACTS Monitor (Figure 7) was developed. This framework enables to view in real-time or via post-processing the synchronized data channels, raw sensor signals as well as meta-information resulting from complex processing.
Figure 8a, left, clearly visualizes by means of the position of the POR that the overt attention has not been focused on the signage above (Figure 8a, right). Then from all ETG video frames, where a signage had been detected, the relative distance between the signage and the POR was extracted and overlaid this information in a single image which is depicted in Figure 8c, left. The color coding reveals - in terms of a heat map - the distribution of PORs in the vicinity of the signage which has been positioned above for the purpose of getting a measure on how far the human attention is focused in the frame of signage presence. It can be clearly depicted that signage does not focus attention of the evacuation person.
Figure 8b, left, demonstrates the POR being located on a person, in the hallway of the office building, where persons gathered to wait for an official evacuation guide. (Figure 8b, right) shows the detection of a person by the annotation of a square overlaid on the position of the person, in fact, its head/shoulder based outline in the video frame. We repeated the process described in the paragraph above, this time for the purpose of a POR based heat map in relative distance to person detections. The resulting heat map clearly visualizes that persons are definitely in the focus of attention, with a slight offset that might be due to calibration purposes.
Figure 5. Mean dwell time as a measure of cognitive load.
Figure available in the original PDF paper.
Figure 6. Number of saccades as a measure of alertness.Figure 7. FACTS Monitor for real-time observation of multiple, multimodal data streams.
(a)
(b)
(c)
Figure 8. Analysis of eye tracking videos: (a) detected signs, (b) detected person, (c) heat maps relative to scene objects.
Figure available in the original PDF paper.
SIMULATION MODELL AND HUMAN FACTORS In the case of disastrous accidents (fire, terrorist attacks, etc.) in large public and residential buildings, human factors play an important role in the effective outcome of evacuations (Pu and Zlatanova, 2005). The design and the evaluation of evacuation systems are crucial to guarantee successful responses after an incident. Mathematical models for the simulation of crowd behavior have been investigated as quantitative tool for demonstrating evacuation performance in the case of an emergency (Gwynne, Galea, Owen, Lawrence and Filippidis, 1999; Helbing, Farkas, Molnar and Vicsek, 2001; Sagun, Bouchlaghem and Anumba, 2011; Schadschneider et al., 2009).
Recently, microscopic models have been used for evacuation simulations (Zheng, Zhong and Liu, 2009) where some consider pedestrians as homogeneous and some as heterogeneous individuals (groups) differentiated by characteristics like gender, age and different psychological parameters. Commercially available software for the simulation of pedestrian evacuation (e.g., PedGo, TraffGo, EXODUS) includes several parameterizations, however, with rather limited insight into modeling issues. Attempts to include human factors e.g. by including visibility constraints into the model (Brunnhuber, Schrom-Feiertag, Luksch, Matyus and Hesina, 2012; Veeraswamy, Galea and Lawrence, 2009) or models which enables information sharing during evacuation were made (Okaya, Takahashi and Southern, 2012). In human behavior models (PMFserv; Pelechano and Badler, 2006) are integrated into a crowd simulation that incorporates high level wayfinding to explore unknown environments and also implements roles and communication to realistically spread information about the environment in the crowd.
The combination with human behavior models offers mature models for physiology, stress, perception and emotion and can handle the dysfunctional behavior that emerges in people during disasters, such as trance-like disbelief, milling, grouping and docile “sheep-like” following. Such advanced simulation models can exploit their full potential for emergency planning only if they are linked to empirical investigations of emergency situations that would provide the appropriate parameters to describe social behavior.
The three main factors which influence an emergency evacuation are the building, the environment and the human factors. The most complex and important ones are the human factors and they are the most difficult factors to be described. The presented work targets to significantly improve evacuation simulation by parameterizing the human agents’ behavior with information on human factors, such as stress, perception and decision making. In particular, the single person’s behavior in its specific situational context is investigated in the frame of its embodied decision making. For this purpose, users were equipped with a wearable sensor equipment that captures information about the environment (localization sensors), the psychophysiological status of the user (e.g. ECG, EDA), and its viewing (eye tracking glasses) and motion behavior.
From the correlation between the multisensory perceptual and psychophysiological data on the one hand, and the automatically sensed and interpreted situational context on the other hand, we will extract a rule base with a set of logical “pre-condition – action” pairs that will parameterize the crowd simulation model. Based on this rule base a behavioral model as described in (Kuligowski and Gwynne, 2010) will be developed were realistic behavior is a result of multiple factors rather than single specific binary if-then conditions. Therefore threshold values for influential factors have to be provided out of the experiments.
This behavioral model will be integrated in an agent based simulation model to allow an egress analysis that is able to reflect the frequently observed human social behaviors, through simulating the cognitive processes of individual agents and interactions among neighboring agents in the simulation environment. This will enable an evacuation simulation with a high degree of reality and a reliable evaluation of alternative evacuation scenarios.
CONCLUSIONS
First results that target at a significant improvement of the evacuation simulation were presented in this paper. The parameterization of the human agents’ behavior with information on human factors about stress, perception and decision making has been studied thoroughly in an evacuation scenario at a large office building. The combination of visual information from eye tracking and the analysis of information from psychophysiological measures together establish indicators for stress events and reduced information processing by the participant. These events have been identified in the data on the basis of continuous measurements during the evacuation experiment.
The data from the concrete field trial demonstrate that visual orientation as well as social cues are relevant, furthermore, that social gaze is capable to initiate social interactions with consequences on the overall evacuation results, and that orientation is highly focused on the ground during evacuation. We conclude from these cues that it is worth to continue with more focused studies on determining concrete parameters that are intended to be provided to the interface of a cognitive simulation model.
ACKNOWLEDGMENTS
This work has been partially funded by the Austrian Research Promotion Agency by grant n°836270 (EVES) and by grant n°832045 (FACTS).
REFERENCES
Brunnhuber, M., Schrom-Feiertag, H., Luksch, C., Matyus, T., & Hesina, G. (2012). Bridging the gap between visual exploration and agent-based pedestrian simulation in a virtual environment. In Proceedings of the 18th ACM symposium on Virtual reality software and technology (pp. 9–16). New York, NY, USA: ACM. doi:10.1145/2407336.2407339. Fritz, G. and Paletta, L. (2010). Semantic Analysis of Human Visual Attention in Mobile Eye Tracking Applications, Proc. IEEE
International Conference on Image Processing, ICIP 2010, Hongkong, China, September 26-29, 2010, pp. 4565 - 4568. Gwynne, S., Galea, E. R., Owen, M., Lawrence, P. J., & Filippidis, L. (1999). A review of the methodologies used in the computer simulation of evacuation from the built environment. Building and Environment, 34(6), pp. 741–749. doi:10.1016/S0360-1323(98)00057-2 Hayhoe M & Ballard D. (2005) Eye movements in natural behaviour. TRENDS in Cognitive Sciences, Vol. 9, April 2005. Hayhoe, M. et al. (2003) Visual memory and motor planning in anatural task. Journal of Vision. 3, pp. 49–63. Helbing, D., Farkas, I., Molnar, P., & Vicsek, T. (2001). Simulation of Pedestrian Crowds in Normal and Evacuation Situations,
pp. 21–58. Holmqvist, K., Nyström, M., Andersson, R., Dewhusrt, R., Jarodzka, H., and van de Weijler, J. (2011). Eye Tracking – A Comprehensive Guide to Methods and Measures, Oxford University Press, 2011. Janke, W., Debus., G., Kallus, K. W., Hüppe, M., Schmidt-Atzert, L. (1989), “BSKE (EWL)-ak“, Unveröffentlichtes Manuskript.
Universität Würzburg, Germany. Kuligowski, E. D., & Gwynne, S. M. V. (2010). The Need for Behavioral Theory in Evacuation Modeling. In W. W. F. Klingsch, C. Rogsch, A. Schadschneider, & M. Schreckenberg (Eds.), Pedestrian and Evacuation Dynamics 2008 (pp. 721 2008amSpringer Berlin Heidelberg. Land, M. and Furneaux, S. (1997) The knowledge base of the oculomotor system. Philos. Trans. R. Soc. Lond. B Biol. Sci. pp.
352, pp.1231–1239 Land, M.F. and Lee, D.N. (1994). Where we look when we steer. Nature 369, pp. 742–744. Land, M.F. et al. (1999) Eye movements and the roles of vision in activities of daily living: making a cup of tea. Perception, Vol.
28, pp. 1311–1328 Munn, S. M., and Pelz J. B. (2008). 3D POR, position and head orientation from a portable monocular video-based eye tracker.
Proc. ETRA, pp. 181-188, 2008. Okaya, M., Takahashi, T., & Southern, M. (2012). Effect of Guidance on Evacuation Behavior Simulations Using Agent Communication. Paletta, L., Santner, K., Fritz, G., Mayer, H., Schrammel, J. (2013). 3D Attention: Measurement of Visual Saliency Using Eye
Tracking Glasses, Proc. ACM SIGCHI Conference on Human Factors in Computing Systems, CHI’13 extended abstracts, p. 199-204, Paris, France. Pelechano, N., & Badler, N. I. (2006). Modeling Crowd and Trained Leader Behavior During Building Evacuation. IEEE Computer Graphics and Applications, 26(6), 80–86. doi:10.1109/MCG.2006.133 Pelz, J.B. and Canosa, R. (2001) Oculomotor behaviour and perceptual strategies in complex tasks. Vision Res. 41, 3587–3596. Pirri, F., Pizzoli, M., Rudi, A.: A general method for the POR estimation in 3D space. Proc. CVPR, 2011. Sagun, A., Bouchlaghem, D., & Anumba, C. J. (2011). Computer simulations vs. building guidance to enhance evacuation
performance of buildings during emergency events. Simulation Modelling Practice and Theory, 19(3), 1007–1019. doi:10.1016/j.simpat.2010.12.001 Schadschneider, A., Klingsch, W., Klüpfel, H., Kretz, T., Rogsch, C., & Seyfried, A. (2009). Evacuation Dynamics: Empirical Results, Modeling and Applications. In R. A. Meyers (Ed.), Encyclopedia of Complexity and System Science. Berlin Heidelberg New York: Springer. Veeraswamy, A., Galea, E.R., & Lawrence, P. (2009). Implementation of Cognitive Mapping, Spatial Representation and
Wayfinding Behaviours of People within Evacuation Modelling Tools. In Proceedings of the 4th International Symposium on Human Behaviour in Fire, Robinson College, Cambridge, UK, 13-15 July 2009, (pp. 501–512). Voßkühler A., Nordmeier V. and Herholz S. (2009). Gaze3D - Measuring gaze movements during experimentation of real physical experiments. Proc. ECEM, 2009. Zheng, X., Zhong, T., & Liu, M. (2009). Modeling crowd evacuation of a building based on seven methodological approaches.
Building and Environment, 44(3), 437 – 445. doi:10.1016/j.buildenv.2008.04.002
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Advances in Applied Digital Human Modeling
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-4951-2094-7
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional