Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Using Cardiac and Electrodermal Activity as Cognitive Markers for Interruptions and Distraction in a Surveillance Simulation

1 Université Laval, Québec, QC, G1V 0A6, Canada ↗
2 ENAC, Université de Toulouse, France ↗
Publication year
Digital Object Identifier10.54941/ahfe1004733

Abstract

Security surveillance is frequently used to increase public safety. Characteristics of the surveillance rooms, however, pose many cognitive challenges pertaining to distraction and interruptions, which may affect surveillance performance. Affective computing could represent a potential solution. It involves the recognition and the interpretation of human states using, for instance, different psychophysiological measures. As a first step toward this goal, the present study aimed at assessing whether cardiac and electrodermal activity, could be used as potential markers of interruptions and distraction during a surveillance simulation. A total of 126 participants went through a simulation involving four 8-min scenarios using a high-fidelity urban security surveillance microworld. Task interruption in the form of a realistic secondary task to perform and distraction in the form of background noise representative of a busy operational centre were also implemented into the simulation. Different features of the electrocardiographic (ECG) signal varied with the presence of distraction, but also as a function of time on task. Electrodermal (EDA) features mainly varied as a function of time. These results suggest that distraction and time on task specifically impacted cognitive functioning, potentially increasing sympathetic activity through cognitive workload, and that EDA and ECG measures may represent relevant markers to use from an affective computing perspective to particularly pinpoint periods of distraction and hypovigilance. Implications for the development of user-adaptive systems are discussed.

INTRODUCTION

In Western countries, security surveillance represents one of the strategies for ensuring security and integrity of population and infrastructure. It often relies on human operators that monitor a large set of closed-circuit tele- vision (CCTV) camera feeds (Kruegle, 2004). Such a task is characterized by an important number of cognitive challenges. Hodgetts et al. (2017)

© 2024. Published by AHFE Open Access. All rights reserved. 1

described how CCTV surveillance, by nature, imposes important atten- tional demands upon human operators. For instance, high camera-to-person ratios are typically observed (e.g., up to 50 screens to monitor concurrently; Troscianko et al., 2004). This can pose problems pertaining to cognitive over- load (Keval and Sasse, 2010). Vigilance is also highly challenged in this work domain. Operators must actively remain open to or search for any specific suspicious activities over long periods of time (work shifts that can reach up to 12 hours), but in an environment typically characterized by very low activity (Keval and Sasse, 2010). Over and above the observa- tion of cognitive overload and vigilance decrement, external events can also affect the workflow of surveillance operators. Typically, operators working in CCTV surveillance centres must not only monitor the different cameras, but also contribute to other reporting tasks, which involve surrounding discus- sions, phone calls, and even field patrolling. As such, surveillance operation centres can be characterized by noise and commotion caused by the coordina- tion between colleagues, interruptions by supervisors, and different types of alarms. Such a context is prone to distraction and task interruption. The cur- rent study was interested in examining how such situations could be detected using a set of psychophysiological markers.

Auditory distraction has been studied in laboratory and field environ- ments. It can reduce performance on the focal task. For example, incongru- ent/unexpected sounds can provoke the deviation effect, which ensues from an automatic attention diversion produced toward the “deviant” sound, at the expense of the focal task (e.g., Hughes et al., 2007). Constantly chang- ing sound (e.g., noise or background conversations) rather produces the changing-state effect (Hughes et al., 2007; Marsh et al., 2009). This phe- nomenon is generally thought to stem from a conflict of processing between the irrelevant sound and the content of the focal task, the first one being involuntary yet ineluctable, and the second one being deliberate. Within surveillance centres, many auditory alarms, phone calls or conversations between colleagues can be heard, which may induce attention diversion and interference with the main monitoring task. Task interruption also represents an important bulk of the literature in human factors, having been studied in many applied contexts (Darmoul et al., 2015). Interruptions are frequent in surveillance centres, following for instance a request from an external author- ity (Hodgetts et al., 2014) or ensuing from dynamic team communication within the surveillance room (Tremblay et al., 2012). Interruptions diverse attention from an ongoing task toward a secondary, different task. Once the secondary task is over, efforts must be deployed to regain awareness of the situation as well as to reconfigure one’s task goals to resume the interrupted activity. This process not only induces task flow hindrance and situation awareness reduction, but may also impose important workload demands and performance costs for post-interruption actions (Altmann and Trafton, 2002; Hodgetts et al., 2015; St. John and Smallman, 2008).

Different strategies can be deployed to mitigate the negative impacts of auditory distraction and interruption among security surveillance operators. One approach is by ensuring that hired personnel possesses proper abilities to protect themselves again the impediment caused by distracting sound and

Using Cardiac and Electrodermal Activity as Cognitive Markers 3

interruptions (cf. Marois et al., 2021). For instance, individuals with higher working memory capacity may be more resistant to distraction (e.g., Hughes et al., 2013) and prone to more efficient post-interruption performance (e.g., Labonté and Vachon, 2021). Although pre-hiring assessments of surveillance- relevant cognitive abilities would be advisable, organizations cannot neces- sarily implement such a strategy, especially due to the important attrition rate and turnover reported among surveillance centres (Piza and Moton, 2023; Shukla et al., 2020). Forewarnings could also represent a good asset for reducing the impacts of distraction and task interruption. Studies showed that foreknowledge of an upcoming auditory distraction can, in some situa- tions, reduce its negative impact on the focal task (e.g., Hughes et al., 2013). Similar conclusions have also been proposed for interruptions (e.g., Labonté et al., 2019). Realistically, however, surveillance centres may not necessarily possess the ability to anticipate imminent distraction and interruptions given the unpredictability of certain events. Affective computing represents an inter- esting alternative. This technique allows recognizing a variety of human states using different sorts of data, including physiological responses (Picard, 2003). For developing proper affective computing technologies, one must first iden- tify appropriate markers of the states wished to be automatically detected by the system. Auditory distraction, for example, can be observed via different types of signals collected from sensing devices such as electroencephalogra- phy, eye tracking, and functional magnetic resonance imaging (Marois and Vachon, 2024). One of the main challenges is to find measures that are suf- ficiently specific to the state to be identified, but also measures noninvasive enough to be easily collected throughout different sets of contexts, in applied settings. In that regard, peripheral measures may be properly adapted to this. Work by Benaroch (1993) and Thayer and Lane (2009) contributed to pinpointing potentially relevant markers for such conditions. Through the neurovisceral integration model, they outlined constant interactions between electrocardiac (ECG) activity and the brain. These interactions can be both top- down and bottom-up, with physiological states impacting cognitive states and conversely (e.g., Valenza et al., 2019). New markers based on sig- nal entropy have been developed to capture the complexity of information exchanged between heart and brain (Costa et al., 2005; Richman and Moor- man, 2000). More specifically, signal entropy has proved to be an effective marker of anxiety (Dimitriev et al., 2016) and attention (Young and Ben- ton, 2015). Similar conclusions can also be reached for electrodermal (EDA) activity. It has been widely used to study phenomena such as stress (e.g., Visnovcova et al., 2016) and cognitive load (e.g., Buchwald et al., 2019). These markers could represent interesting measures for identifying periods of distraction and interruptions among surveillance operators.

The present study aimed at assessing whether cardiac and electrodermal activity, could serve as markers of interruptions and distraction during a surveillance simulation. ECG and EDA signal from 126 participants were collected for four 8- minute high-fidelity urban security surveillance simu- lations involving CCTV cameras. During the surveillance task, participants were additionally either interrupted by questions related to the surveillance

task, distracted by background conversations, or both. Their effect, as well as that of time on task, were evaluated on different ECG and EDA features.

METHOD

Participants Data from 126 participants (56 females, 70 males, Mage = 24.69 years) were recorded using a Biopac. All reported normal or corrected-to-normal vision and hearing and no diagnosed neurological disorder.

Material and Design We made use of the Cognitive Solutions to Security Surveillance (CSSS) microworld to simulate a surveillance environment (Vachon et al., 2016).

Participants were asked to monitor eight CCTV camera feeds concurrently on an interface that could only display six simultaneously. The task involved navigating through the eight cameras by displaying them as desired and to detect and report any suspicious event or event that was prioritized a priori (e.g., a missing person). The cameras displayed simulated feeds representing a crowded Quebec City music festival. The surveillance interface, including the eight cameras, an incident report tab, and a map of the area to monitor were presented on two LCD monitors and on a display wall of knowledge (see Figure 1).

Figure 1: CSSS microworld used for the surveillance simulation.
Original PDF Figure 1:

After having provided informed consent, participants completed a brief sociodemographic questionnaire and were connected to the Biopac system for collecting ECG and EDA data. The Biopac system collected ECG and EDA data at a 100-Hz sampling frequency. Participants performed four counterbalanced 8-min scenarios in which the nature and the timing of the incidents to detect varied. Details on the incidents for each scenario can be found in Marois et al. (2021). Participants were asked to report inci- dents using the surveillance interface, selecting the camera of the incident, and categorizing it according to a predefined list. Each participant also

Using Cardiac and Electrodermal Activity as Cognitive Markers 5

went through four conditions, independently counterbalanced from the sce- nario order. Half of the scenarios were characterized by auditory distraction. Auditory distraction was implemented via a background audio track that comprised typical conversations between control-room operators, as well as background noises (e.g., walking, doors opening/closing, keyboard typing, mouse clicking, eating, and drinking). Half of the scenarios also contained interruptions. The interruptions took the form of questions, presented audi- bly via the headset worn by participants. In each 8-minute scenario, the participant was presented with two interruptions. The distraction and inter- ruption manipulations led to a 2 × 2 experimental design with the following conditions: a) control; b) distraction only; c) interruptions only; and d) distraction + interruptions.

Data Processing and Analysis We focus on the psychophysiological measures collected in the experiment (for the performance measures on the surveillance task, see Marois et al., 2021). Eighteen participants were first removed from the analysis because of missing data. For the ECG analysis, another set of 22 participants was removed because of poor signal quality. ECG data was low-pass filtered at a cutoff frequency of 30 Hz before computing the RR interval time series. Fol- lowing the recommendations of the Task Force (1996), RRmean, RMSSD, PNN50 (i.e. the proportion of successive RR interval differences of duration greater than 50 ms) were computed in the time domain. After resampling the RR interval series to 4 Hz, a Fourier transform was applied to obtain the frequency markers, using windows of 0.04 Hz – 0.15 Hz for the sympathetic activity (Low Frequencies [LF]) and 0.15 Hz – 0.40 Hz for the parasympa- thetic activity (High Frequencies [HF]). The power in both frequency bands (LF, HF) and the total power were computed as the area under the curve (AUC) for the power spectral density. The normalized powers LFnu and HFnu, and the LF/HF ratio were also computed. Finally, signal complexity was estimated with the RCMSE method (Wu et al., 2014). The AUC for the multiscale entropy, calculated with 4 scales, was used as a marker of cardiac entropy (entropy index). For the EDA analysis, only 39 participants could be analyzed due to important missing values. The EDA signal was down sam- pled to 2 Hz and Z-scored. Tonic and phasic components of the signal were extracted using the cvxEDA method (Greco et al., 2016) and an index of sympathetic tone, referred to as TvSymp (Posada-Quintero et al., 2016) was computed using the VFCDM method (Wang et al., 2006). The mean values for the phasic and tonic components, as well as the mean AUC of the TvSymp measure, were chosen as physiological features.

Features were extracted for the total duration of the scenarios (i.e. 8 min). Normal distribution of the datasets was assessed using Shapiro-Wilk tests. To explore the effects of distraction, interruptions and time on physiological data, either repeated-measures analysis of variance (ANOVA) with four lev- els (blocks 1 to 4, or distraction/interruption conditions) or Friedman tests were used depending on the normality of the data. To further explore the spe- cific effects of interruptions and distractions, two-way ANOVAs were used to assess potential differences, and if no interaction effect was found, either

Wilcoxon signed-rank tests or paired-samples t-tests were performed to assess the differences between conditions.

RESULTS

ECG Analysis No difference was found according to the presence of interruptions in sce- narios (i.e. average of Interruptions only and Interruptions + Distraction vs. average of Control and Distraction only) for all ECG features. Yet, signifi- cant increases were found for the scenarios containing auditory distraction on the RMSSD (Z = −3.95, p < 0.001), PNN50 (Z = −2.67, p = 0.008), LF (Z = −3.11, p = 0.002), HF (Z = −5.53, p < 0.001) and total power (Z = −4.72, p < 0.001). The entropy index rather reduced with the presence of distraction (t = 3.65, p < 0.001).

Analysis of the impact of time also raised several differences across the four measurement blocks. At least one significant difference could be found across the measurement blocks for the mean RR, F(3, 255) = 22.61, p < 0.001, RMSSD, χ2(3) = 8.19, p = 0.04, LF power, χ2(3) = 30.15, p < 0.001, normalized LF power, F(3, 255) = 10.84, p < 0.001, normalized HF power, F(3, 255) = 10.84, p < 0.001, LH/HF ratio, χ2(3) = 25.02, p < 0.001, and total power, χ2(3) = 20.41, p < 0.001. As depicted in Table 1, generally, more time on task induced increases in mean RR, LF power, normalized LF power, LF/HF ratio and total power. Normalized HF power rather decreased as time unfolded.

Table 1. Means (SDs) and multiple comparisons for the impact of time on the ECG features.

Table available in the original PDF paper.

ECG feature Blocks Differences*

1 2 3 4

RRmean 0.968 (0.142)

0.989 (0.141)

0.995 (0.133)

1.00 (0.132) -

1-2, 1-3, 1-4, 2-4 RMSSD 0.049 (0.028)

0.049 (0.027)

0.050 (.027)

0.052 (0.034) -

PNN50 0.288 (0.210)

0.297 (0.209)

0.301 (0.205)

0.299 (0.202)

LF 3.36×10-4

3.57×10-4

3.77×10-4

3.80×10-4 1-3, 1-4, 2-4

(1.41×10-4)

(1.49×10-4)

(1.52×10-4) (1.56×10-4)

HF 3.33×10-4

3.35×10-4

3.37×10-4

3.54×10-4 -

(1.81×10-4)

(1.78×10-4)

(1.79×10-4)

(2.46×10-4)

LFnu 0.514 (0.085)

0.529 (0.087)

0.541 (0.922)

0.537 (0.088)

1-2, 1-3, 1-4, 2-3 HFnu 0.486 (0.085)

0.471 (0.087)

0.459 (0.092)

0.463 (0.088) 1-3, 1-4

1-2, 1-3, 1-4, 2-3 LF/HF 1.126 (0.400)

1.195 (0.415

1.271 (0.474)

1.243 (0.452)

Total power 6.69×10-4

6.92×10-4

7.13×10-4

7.33×10-4 1-3, 1-4

(2.99×10-4)

(3.00×10-4)

(2.97×10-4)

(3.71×10-4) -

Entropy index 5.593 (0.778)

5.757 (0.689)

5.676 (0.712)

5.564 (0.601)

*p < 0.05, with Bonferroni corrections.

Using Cardiac and Electrodermal Activity as Cognitive Markers 7

EDA Analysis No significant difference was found according to the presence of interrup- tions and of auditory distraction for all three EDA features. All tests failed to reach significance, but a trend was observed in the two-way ANOVA, with values of meanPhasic being lower in the conditions with distraction, F(1,151) = 3.54, p = 0.062. The effect of time yielded some differences as shown in Table 2. Friedman tests confirmed the presence of at least one signif- icant difference across blocks for the mean tonic EDA values, χ2(3) = 21.83, p < 0.001, the mean phasic values, χ2(3) = 17.37, p < 0.001, and mean TvSymp values, χ2(3) = 27.86, p < 0.001. Globally, the mean tonic and pha- sic values decreased from Block 1 to Block 4 (ps < 0.043). The mean TvSymp values of Blocks, 2, 3 and 4 were also significantly lower than those observed during Block 1 (ps < 0.001).

Table 2. Means (SDs) and multiple comparisons for the impact of time on the EDA features.

Table available in the original PDF paper.

EDA feature Blocks Differences*

1 2 3 4

1-4, 2-4, 3-4

meanTonic 0.033 (0.815)

0.090 (0.477)

-0.225 (0.423)

-0.678 (0.576) 1-3, 2-3

meanPhasic 0.281 (0.140)

0.255 (0.169)

0.206 (0.140)

0.225 (0.165) 1-2, 1-3, 1-4

meanTvSymp 0.990 (0.119)

0.905 (0.143)

0.884 (0.153)

0.862 (0.160)

*p < 0.05, with Bonferroni corrections.

DISCUSSION

The goal of this study was to assess whether ECG and EDA signals could index interruptions and distraction within a high-fidelity security surveillance simulation. Interruptions did not generate significant variations in both ECG and EDA signals. Yet, some ECG features, namely RMSSD, PNN50, LF, HF, total power and entropy index, were sensitive to the presence of auditory distraction. The EDA features extracted remained impervious to distraction. Further analysis raised differences that emerged as a function of time, that is across the different blocks of measurement. Increases in ECG and decre- ments in EDA activities were observed. Overall, these results support that ECG could represent a useful tool to denote instances of distraction among surveillance operators, but also that both ECG and EDA measures could serve useful to index variations in vigilance levels.

Past research showed that entropy modulations of heart-brain interactions could index variations in levels of anxiety (Dmitriev et al., 2016; Young and Benton, 2015). In our study, temporal, frequential and nonlinear fea- tures were affected by auditory distraction. Such effect could represent an increase in cognitive workload, which is in line with the dual-mechanism perspective of auditory distraction, suggesting that changing sound induces

interference by process, and thus requires efforts to inhibit automatic pro- cessing of the distracting sound (Huges et al., 2007; Marsh et al., 2014). As for the decrease in cardiac entropy, this could be construed as evidence of the stress (Blons et al., 2019) experienced when distracting sound is presented. As for the variations observed with time on task, it has been shown that prolonged cognitive tasks induce cognitive fatigue, which can be observed with frequency markers of HRV (Melo et al., 2017; Zhang and Yu, 2010). Again, this was observed on a set of temporal and frequential ECG features, but also on measures of EDA. While EDA is also reflexive of sympathetic activity, it is known to be highly influenced by arousal and stress (Ali et al., 2023). As such, the decrease in EDA activity across the measurement blocks may represent arousal decrement caused by habituation to the task.

These results are consistent with previous literature that outlined how the activity of the autonomous nervous system can be influenced by attention- related events. Marois and Vachon (2024) identified the main physiological markers used to index auditory distraction. The markers selected, however, were mainly related to direct brain measures such as electroencephalography (e.g., event-related potentials or specific power bands) and magnetic imag- ing, and to pupillometry. Our results support the addition of ECG-related features to that list for potentially indexing auditory distraction. The fact that both ECG and EDA features were sensitive to time on task also suggests that these represented variations in vigilance as time unfolded. This is in line with many physiological models that have been developed for assessing vig- ilance (cf. Marois et al., 2023). The absence of an effect of interruptions was, however, unexpected. As described earlier, interruptions of surveillance and monitoring activities entail two changes in task goal: one that is related to the new task, and another one pertaining to the interrupted task, once it is resumed (Altmann and Trafton, 2002; Hodgetts et al., 2015; St. John and Smallman, 2008). In this situation, cognitive effort is required to adjust the task set as well as to gain or regain awareness of the situation. There- fore, effects should have been observed on the EDA and ECG signals. These changes in workload are, however, transient and specific to the interruption. Consequently, it might be the case that the time windows used for evaluating the impact of interruptions on the different signals (i.e. 8 min) may have been too large. In such a case, the sole effect of the interruption might have been drowned among the variability caused by the task itself. Further analysis will focus on assessing more specific windows to better investigate the impact of these interruptions, and to detect specific skin conductance responses in addition to the frequency-domain analysis.

Future work will also involve improving physiological data management. A high of number of missing values was found due to noise in the data or simply hardware malfunctions. One of the goals of this study was to assess the usefulness of adopting an affective computing perspective, using phys- iological data collected in real time, in a real-life surveillance simulation. There is thus a necessity to use robust sensors and to acquire a maximum of data despite the lack of experimental control—balanced by high ecolog- ical validity—that is characteristic of high-fidelity simulations. Overall, our study represents a first step toward adopting such an approach, but work is

Using Cardiac and Electrodermal Activity as Cognitive Markers 9

still needed to ensure that the markers collected can truly pinpoint instances of distraction, interruptions, and even hypovigilance.

ACKNOWLEDGMENT

This work was supported by grants from the National Sciences and Engineer- ing Research Council of Canada (NSERC), the Ministère de l’Économie et de l’Innovation du Québec (MEI), and Prompt Québec awarded to Sébastien Tremblay. We are also grateful to all the students involved in data collection and to the contribution of Thales Canada, Emergensys Solutions, and Graph Synergie.

REFERENCES

Ali, N., Tschenett, H. and Nater, U. M. (2023) “Biomarkers of stress and disease”, In H. S. Friedman and C. H. Markey (Eds.), Encyclopedia of Mental Health (pp. 221–231). Elsevier Inc. https://doi.org/10.1016/B978-0-323-91497- 0.00231–9 Altmann, E. M. and Trafton, J. G. (2002) “Memory for goals: An activation based model”, Cognitive Science, 26, 39–83. https://doi.org/10.1207/s15516709cog2601_2 Benarroch, E. E. (1993) “The central autonomic network: Functional organiza-

tion, dysfunction, and perspective”, Mayo Clinic Proceedings, 68, 988–1001. https://doi.org/10.1016/S0025-6196(12)62272-1 Blons, E., Arsac, L. M., Gilfriche, P., McLeod, H., Lespinet-Najib, V., Grivel, E.

and Deschodt-Arsac, V. (2019) “Alterations in heart-brain interactions under mild stress during a cognitive task are reflected in entropy of heart rate dynamics”, Scientific Reports, 9, 18190. https://doi.org/10.1038/s41598-019-54547-7 Buchwald, M., Kupi´nski, S., Bykowski, A., Marcinkowska, J., Ratajczyk, D. and Jukiewicz, M. (2019) “Electrodermal activity as a measure of cognitive load: a methodological approach”, Signal Processing, 175–179, https://doi.org/10.23919/SPA.2019.8936745. Costa, M., Goldberger, A. L. and Peng, C.-K. (2005) “Multiscale entropy analysis of biological signals”, Physical Reviews: Part E, 71, 021906. https://doi.org/10.1103/PhysRevE.71.021906 Darmoul, S., Ahmad, A., Ghaleb, M. and Alkahtani, M. (2015) “Interruption management in human multitasking environments”, IFAC-Papers Online, 48, 1179–1185. https://doi.org/10.1016/j.ifacol.2015.06.244 Dimitriev, D. A., Saperova, E. V. and Dimitriev, A. D. (2016) “State anxiety and non-

linear dynamics of heart rate variability in students”, PLOS ONE, 11, e0146131. https://doi.org/10.1371/journal.pone.0146131 Greco, A., Valenza, G., Lanata, A., Scilingo, E. P. and Citi, L. (2016) “cvxEDA: A convex optimization approach to electrodermal activity pro- cessing”, IEEE Transactions on Biomedical Engineering, 63, 797–804. https://doi.org/10.1109/TBME.2015.2474131 Hodgetts, H. M., Tremblay, S., Vallières, B. R. and Vachon, F. (2015) “Decision support and vulnerability to interruption in a dynamic multitasking envi- ronment”, International Journal of Human-Computer Studies, 79, 106–117. https://doi.org/10.1016/j.ijhcs.2015.01.009

Hodgetts, H. M., Vachon, F., Chamberland, C. and Tremblay, S. (2017) “See no evil: Cognitive challenges of security surveillance and monitor- ing”, Journal of Applied Research in Memory and Cognition, 6, 230–243. https://doi.org/10.1016/j.jarmac.2017.05.001 Hodgetts, H. M., Vachon, F. and Tremblay, S. (2014) “Background sound impairs interruption recovery in dynamic task situations: Procedural conflict?”, Applied Cognitive Psychology, 28, 10–21. https://doi.org/10.1002/acp.2952 Hughes, R. W., Hurlstone, M., Marsh, J. E., Vachon, F. and Jones, D. M. (2013) “Cog-

nitive control of auditory distraction: Impact of task difficulty, foreknowledge, and working memory capacity supports duplex-mechanism account”, Journal of Experimental Psychology: Human Perception and Performance, 39, 539–553. https://doi.org/10.1037/a0029064 Hughes, R., Vachon, F. and Jones, D. M. (2007) “Disruption of short-term memory by changing and deviant sounds: Support for a duplex-mechanism account of auditory distraction”, Journal of Experimental Psychology: Learning, Memory, and Cognition, 33, 1050–1061. https://doi.org/10.1037/0278-7393.33.6.1050 Keval, H. and Sasse, M. A. (2010) ““Not the usual suspects”: A study of

factors reducing the effectiveness of CCTV”, Security Journal, 23, 134–154. https://doi.org/10.1057/sj.2008.2 Kruegle, H. (2004) “CCTV surveillance”, In L. J. Fennelly (Ed.), Effective physical security (pp. 209–217). Elsevier Inc. Labonté, K., Tremblay, S. and Vachon, F. (2019) “Forewarning interruptions in

dynamic settings: Can prevention bolster recovery?”, Journal of Experimental Psychology: Applied, 25, 674–694. https://doi.org/10.1037/xap0000219 Labonté, K. and Vachon, F. (2021) “Resuming a dynamic task following increasingly long interruptions: The role of working memory and reconstruction”, Frontiers in Psychology, 12, 659451. https://doi.org/10.3389/fpsyg. 2021.659451 Marois, A., Hodgetts, H. M., Chamberland, C., Williot, A. and Tremblay, S. (2021)

“Who can best find Waldo? Exploring individual differences that bolster perfor- mance in a security surveillance microworld”, Applied Cognitive Psychology, 35, 1044–1057. https://doi.org/10.1002/acp.3837 Marois, A., Kopf, M., Fortin, M., Huot-Lavoie, M., Martel, A., Boyd, J. G., Gagnon, J.-F. and Archambault, P. M. (2023) “Psychophysiological models of hypovigilance detection: A scoping review”, Psychophysiology, 60, e14370. https://doi.org/10.1111/psyp.14370 Marois, A. and Vachon, F. (2024) “Psychophysiological markers of auditory dis-

traction: A scoping review”, in press in Auditory Perception & Cognition. https://doi.org/10.1080/25742442.2023.2274270 Marsh, J. E., Hughes, R. and Jones, D. (2009) “Interference by process, not content, determines semantic auditory distraction”, Cognition, 110, 23–38. https://doi.org/10.1016/j.cognition.2008.08.003 Melo, H. M., Nascimento, L. M. and Takase, E. (2017) “Mental fatigue and heart

rate variability (HRV): The time-on-tas effect”, Psychological Neuroscience, 10, 428–436. https://doi.org/10.1037/pne0000110 Picard, R. W. (2003) “Affective computing: challenges”, International Jour- nal of Human- Computer Studies, 59, 55-64. https://doi.org/10.1016/S1071- 5819(03)00052-1 Piza, E. L. and Moton, L. N. (2023) “Proactive monitoring and operator discretion: A systematic social observation of CCTV control room operations”, Journal of Criminal Justice, 86, 102071. https://doi.org/10.1016/j.jcrimjus.2023.102071

Using Cardiac and Electrodermal Activity as Cognitive Markers 11

Posada-Quintero, H. F., Florian, J. P., Orjuela-Cañón, Á. D. and Chon, K. H. (2016) “Highly sensitive index of sympathetic activity based on time-frequency spectral analysis of electrodermal activity”, American Journal of Physiology, 311, R582- R591. https://doi.org/10.1152/ajpregu.00180.2016 Richman, J. S. and Moorman, J. R. (2000) “Physiological time-series analysis using

approximate entropy and sample entropy”, American Journal of Physiology, 278, H2039-H2049. https://doi.org/10.1152/ajpheart.2000.278.6. H2039 Shulka, R., Lawrence, D. S., Peterson, B. E. and McClure, D. (2020) Optimizing public surveillance systems for crime control and prevention: A guide for law enforcement and their municipal partners. Washington, DC: Urban Institute. St. John, M. and Smallman, H. S. (2008) “Staying up to speed: Four design principles for maintaining and recovering situation awareness”, Journal of Cognitive Engineering and Decision Making, 2, 118–139. https://doi.org/10.1518/155534308X284408 Task Force of The European Society of Cardiology and The North American Society

of Pacing and Electrophysiology. (1996) “Heart rate variability. Standards of mea- surement, physiological interpretation, and clinical use”, European Heart Journal, 17, 354–381. Thayer, J. F. and Lane, R. D. (2009) “Claude Bernard and the heart– brain connection: Further elaboration of a model of neurovisceral integration”, Neuroscience and Biobehavioral Reviews, 33, 81–88. https://doi.org/10.1016/j.neubiorev.2008.08.004 Tremblay, S., Vachon, F., Lafond, D. and Kramer, C. (2012) “Dealing with task interruptions in complex dynamic environments: Are two heads better than one?”, Human Factors, 54, 70–83. https://doi.org/10.1177/0018720811424896 Troscianko, T., Holmes, A., Stillman, J., Mirmehdi, M., Wright, D. and Wilson, A.

(2004) “What happens next? The predictability of natural behaviour viewed through CCTV cameras”, Perception, 33, 87–101. https://doi.org/10.1068/p3402 Vachon, F., Vallières, B. R., Suss, J., Thériault, J.-D. and Tremblay, S. (2016) “The CSSS microworld: A gateway to understanding and improving CCTV security surveillance”, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 60, 265–269. https://doi.org/10.1177/1541931213601061 Valenza, G., Sclocco, R., Duggento, A., Passamonti, L., Napadow, V., Barbieri, R.,

and Toschi, N. (2019) “The central autonomic network at rest: Uncovering func- tional MRI correlates of time-varying autonomic outflow”, NeuroImage, 197, 383–390. https://doi.org/10.1016/j.neuroimage.2019.04.075 Visnovcova, Z., Mestanik, M., Gala, M., Mestanikova, A. and Tonhajze- rova, I. (2016) “The complexity of electrodermal activity is altered in mental cognitive stressors”, Computers in Biology Medicine, 79, 123–129. https://doi.org/10.1016/j.compbiomed.2016.10.014 Wang, H., Siu, K., Ju, K., and Chon, K. H. (2006) “A high resolution approach to estimating time-frequency spectra and their amplitudes”, Annals of Biomedical Engineering, 34, 326–338. https://doi.org/10.1007/s10439-005-9035-y Zhang, C. and Yu, X. (2010) “Estimating mental fatigue based on electroen-

cephalogram and heart rate variability”, Polish Journal of Medical Physics and Engineering, 16, 67–84. https://doi.org/10.2478/v10013-010-0007-7

Cover of the proceedings volume Neuroergonomics and Cognitive Engineering

Explore the complete digital volume

Neuroergonomics and Cognitive Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-02-1

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →