Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Evaluating the restorative impact of nature through multimodal mobile sensing of neural, physiological, and behavioral activity in ambulatory settings

1 School of Biomedical Engineering, Science and Health Systems, Drexel University, Philadelphia, PA, 19104, USA ↗
2 Lebow College of Business, Drexel University, Philadelphia, PA, 19104, USA ↗
Publication year
Digital Object Identifier10.54941/ahfe1003015

Abstract

One of the fundamental principles of neuroergonomics is that human cognition is profoundly shaped by the environment in which it operates. In the modern world, this environment can often be highly artificial, noisy, barren, and intentionally distracting. On the other hand, natural environments compare favorably as they may offer not only an appreciation of beauty but a rich array of sensory and contextual information which can be undemanding to the observer. Attention Restoration Theory (ART) proposes that exposure to natural environments can provide various benefits to stress, health, and cognition. Understanding how the brain responds to natural environment presentation poses a crucial hurdle to using traditional neuroimaging techniques as many approaches necessitate highly controlled and resultingly, low-fidelity stimuli presentation to mimic the environmental effects of nature. Functional near-infrared spectroscopy (fNIRS), a non-invasive brain monitoring technology that relies on optical techniques to detect changes in cortical hemodynamic responses to human perceptual, cognitive, and motor functioning, is an ideal candidate tool for understanding the brain in natural environments. In this paper, we will describe an experimental setup that involves the integration of mobile fNIRS systems with simultaneous wrist-based optical heart rate monitoring (OHRM) and electrodermal activity (EDA) recordings that can record the cognitive and physiological responses of individuals to natural settings.

© 2023. Published by AHFE Open Access. All rights reserved. 152

Evaluating the Restorative Impact of Nature Through Multimodal Mobile Sensing 153

INTRODUCTION

Humans living in the modern world occupy vastly different spaces than evo- lution may have prepared them for. Presently, 55% of the world’s population lives in urban areas and this number is expected to increase to 68% by the year 2050 (World Urbanization Prospects, 2019). The rise of highly urba- nized spaces has been accompanied broadly by the declining availability of green spaces and an overall reduction of natural settings in everyday life (Beyer et al., 2018, p. 29). Living within urban spaces involves significant exposure to noise, pollution, information overload, and other factors which may require individuals to alter their attentional processing to co-exist with the environment (Linnell et al., 2013). In addition to burdens imposed by urban settings, work in modern office settings has been associated with incre- ased burnout, reduced productivity, and a variety of stress-induced illnesses such as depression and anxiety (An et al., 2016). As the costs of sustained stress and attentional demands mount, researchers have explored the benefits of using nature exposure sessions as an antidote (Bratman et al., 2021).

A notable framework attempting to describe both the negative impacts of attention-grabbing environments as well as the restorative quality of nature derives from the neuroscience of attention. Attention Restoration Theory (ART) (Kaplan, 1995) proposes that directed attention can be modeled as a muscle, wherein the sustained attentional demands of urban environments or prolonged focus on work-related tasks results in significant mental fatigue which requires rest to recover from. Here, effortful attention is considered to be the purview of the dorsal attention network and fatigue may induce deficits similar to frontal cortex impairment including difficulty in planning, socializing, and impulse control (Foster et al., 1994; Holtzer et al., 2010; Kaplan and Berman, 2010). On the other hand, natural stimuli are considered to be optimally restorative because they are able to be “softly fascinating”and with which the brain can effortlessly attend while the ability to direct effort- ful attention is restored. The automatic ventral attention network takes over and is able to engage loosely with these complex, coherent, but undemanding stimuli and promote attentional recovery (Williams et al., 2018).

Another prominent framework advocates psycho-physiological stress as the primary means by which urban environments impact individual physio- logy and well-being (Ulrich et al., 1991). Ulrich suggests that mental fatigue from prolonged directed attention is also accompanied by an accumulation of physiological and mental stressors which negatively affect the individual. In this framework, unthreatening natural stimuli offer a psychological benefit rooted in an evolutionary and aesthetic preference for nature (Meidenbauer et al., 2019). Accordingly, spending time in visually pleasant environments can help reduce stress and improve mood by promoting parasympathetic acti- vity. Together these two frameworks propose that natural environments are offer significant advantages as a means of restoring an individual’s cogni- tive wellbeing in contrast with urban environments (Kjellgren and Buhrkall, 2010). Although the cognitive benefits of nature exposure have been studied under a variety of circumstances (Stevenson et al., 2018), neuroimaging

studies of nature exposure under real-world contexts are few and far betw- een with the vast majority focusing on virtual or static natural images as a means of nature immersion. Environmental settings in lab research are often purposefully reduced to eliminate experimental variability, these con- trols can also overlook the negative impacts of barebone environments (Ayaz and Dehais, 2019; Tognoli, 1973). Nature immersion draws on a of number auxiliary non-visual stimuli which are difficult to recreate within the labo- ratory environment (Mollazadeh and Zhu, 2021). In order to understand how our mind responds to nature, we cannot substitute the experience for a cardboard cutout.

NEUROERGONOMIC ASSESSMENT OF RESTORATIVE ENVIRONMENTS Neuroergonomics is a research field that aims to study the brain at work and in everyday life (Ayaz and Dehais, 2021; Parasuraman, 2003). This approach asserts that an understanding of human brain function is critical to under- standing how individuals operate within society and human-systems. Key to this philosophy is the idea that the environment profoundly shapes the nature of cognition and accordingly, cognitive processes should be studied “in the wild”, under realistic contexts which closely match the systems being studied (Curtin and Ayaz, 2018; Dehais et al., 2020). This form of study has been made feasible in part by the proliferation of mobile neuroimaging technolo- gies which has in turn freed researchers from static laboratory experiments. These advances have allowed researchers to study the brain under a wide vari- ety of circumstances including individuals walking outdoors (McKendrick et al., 2016), attending class (Barreto et al., 2021), performing surgery (She- wokis et al., 2017), and even during flight piloting aircraft (Gateau et al., 2018). The ability to study the brain in context is especially important in stu- dies of environmental reception where environmental context is the defining feature.

In addition, neuroergonomics highlights the need for integration of wea- rable neuroimaging with non-invasive physiological monitoring to provide multi-modal neural and peripheral physiological activity (Curtin and Ayaz, 2018). Early neuroimaging works often have taken the view that periphe- ral physiological responses to stimulation represent purely nuisance signals which should be filtered to better understand the specific cortical response without assessing the relationships between physiology and the brain. How- ever, physiological response offer unique insights into the underlying brain states and greatly assist in state-related classification (Liu et al., 2017; Mark et al., 2018). Parsing the restorative properties of nature exposure may expli- citly require both an understanding of the cortical response from the active dorsal attention network as well as peripheral measures of physiological stress. In this work, we outline a comparative protocol for studying the environmental effects of nature using mobile neuroimaging and non-invasive physiological monitoring.

fNIRS and Nature Assessment In order to assess cortical function, we will employee functional near-infrared spectroscopy (fNIRS), a non-invasive neuroimaging technique which has

Evaluating the Restorative Impact of Nature Through Multimodal Mobile Sensing 155

emerged over the last two decade (Ayaz et al., 2013). fNIRS uses near infra- red light to monitor changes in oxygenated and deoxygenated hemoglobin at the outer cortex of the brain (Hoshi et al., 2003). By employing wearable light sources and detectors, photons are emitted over the scalp that pass by layers of tissue and detectors then collect the fraction of them that return. Because most tissue is transparent to light between 700–900 nm and because absorption is minimal within this optical window, fNIRS systems use multiple wavelengths within this near infrared range. fNIRS is able to measure optical density fluctuations caused by metabolic changes in neural activity through a mechanism called neurovascular coupling (Cauli, 2010) and can measure the hemodynamic response in a similar fashion to functional magnetic resonance imaging (fMRI). However, fNIRS sensors are wearable, portable, low-cost and possess a higher temporal resolution then fMRI. fNIRS allows subje- cts to freely move in natural postures while monitoring cortical regions such as the prefrontal cortex or motor cortex in a way that is compatible with research surrounding cognitive and motor tasks (Ayaz et al., 2022).

EDA and Nature Assessment Evaluation of psychophysiological interaction using conductance change of the skin is a classic measure of stress and stimuli reception. Early research in the study of nature presentation is predominately based on stimuli-induced changes in conductance (Perrins et al., 2021; Ulrich et al., 1991). Electro- dermal activity (EDA) is a non-invasive, portable sensor that measures the changes in electrical properties of the skin. The electrical property changes occur due to the electrolytes inside sweat when sweat secretion occurs by the eccrine sweat glands (Sharma et al., 2016). Eccrine sweat glands play a key role in regulating thermoregulation and are activated by sympathetic activity of the autonomic nervous system (ANS). ANS activity is responsible for fight-or-flight response, which reflects bodily arousal and is associated with emotional expressions and behaviors in humans (Sequeira et al., 2009). Hence, EDA is considered as an indicator of emotional arousal and proces- ses (Caruelle et al., 2019; Shi et al., 2007) and has been utilized in diverse applications in ambulatory settings (Topoglu et al., 2020).

HRV and Nature Assessment In addition, peripheral physiological activity will be assessed via wrist-based integrated optical heart-rate monitoring (OHRM). OHRM employs a similar principle to fNIRS but uses different wavelengths and focuses on capturing pulsatile variations in blood concentrations associated with the heartbeat. Increases in heart rate and changes in heart-rate variability (HRV) are often taken as peripheral measures of exertion and engagement (ChuDuc et al., 2013). Increased HRV is associated with increased cognitive flexibility and decreased autonomic activation, whereas decreased HRV is associated with increased autonomic activity and inhibition of the prefrontal cortex (Thayer et al., 2012). These measures can also be used to help control for changes in functional measures observed in fNIRS which may result from periphe- ral changes in addition to local cortical activation. The addition of HRV to

prefrontal measures from fNIRS and autonomic activity via EDA will help tie together the relationships between cognitive burden and physiological responses to stress to which nature exposure is expected to mediate (Scott et al., 2021).

Protocol In order to study the role of different environments on brain and body measu- res, participants will be asked to perform an environmental mindful presence task in several locations located throughout a local botanical garden (Lon- gwood Gardens, Kennett Square, PA). Participants will freely walk between multiple environmental locations, each representing different forms of natu- ral beauty and built-environment settings (Office, Italian Garden, Indoor Garden, Outdoor Meadow, Outdoor Forest) (Fig. 1). In each environment, the participant will be asked to freely observe the environment during which the participant’s prefrontal cortex will be monitored using a mobile fNIRS system (fNIR Devices 1100). In addition, participant HRV and EDA respon- ses will be monitored using a mobile actigraph (Empatica E4). Following the 5 minute exposure tasks, participants will be surveyed on their perce- ptual responses to the environment including the Perceived Restorative Scale (PRS) (Pasini et al., 2014) as well as scales related to the experience of the environment and aesthetic qualities.

Participants will be guided through the different environments according to a counter-balanced order. Data will be analyzed to determine the evoked and sustained responses to environmental immersion with attention paid to variations in resting HRV, intracortical prefrontal connectivity from fNIRS, and interactions between cortical responses and responses from peripheral measures of physiology (EDA, HRV). In addition, participant neural signals

Figure 1: Nature Immersion Task under different environments and example peripheral physiological response.
Original PDF Figure 1:

Evaluating the Restorative Impact of Nature Through Multimodal Mobile Sensing 157

will be compared using qualitative measures from survey to establish the relationship between presented environmental properties, received aesthe- tic qualities, and self-evaluation of the restorative properties via the SRS. Together, this study will offer a real-world insight on the neural and peri- pheral physiological effects of nature immersion as well as a window into the mechanisms behind its asserted restorative potential.

CONCLUSION

While the demanding nature of the modern world can be taxing for the indi- viduals who inhabit it, exposure to natural environments has been suggested to induce a number of beneficial effects including reductions in stress as well as inducing improvements in mood. While competing theories exist as to the mechanisms by which these effects may operate, there is convergent agree- ment that the mind and body work in tandem. This protocol proposes one means of investigating the inter-relationship of psychological, neural, and peripheral responses to nature immersion with an eye towards understanding how these may be beneficial to society at large.

ACKNOWLEDGMENT

The authors would like to acknowledge Longwood Gardens and Drexel Solu- tions Institute for facilitating the study as well as Dr. Kim Lewis Meidenbauer for their helpful advice on experiential scale selection.

REFERENCES

An, M., Colarelli, S. M., O’Brien, K., Boyajian, M. E., 2016. Why We Need More Nature at Work: Effects of Natural Elements and Sunlight on Employee Mental Health and Work Attitudes. PLOS ONE 11, e0155614. Ayaz, H., Baker, W. B., Blaney, G., Boas, D. A., Bortfeld, H., Brady, K., Brake, J.,

Brigadoi, S., Buckley, E. M., Carp, S. A., Cooper, R. J., Cowdrick, K. R., Culver, J. P., Dan, I., Dehghani, H., Devor, A., Durduran, T., Eggebrecht, A. T., Ember- son, L. L., Fang, Q., Fantini, S., Franceschini, M. A., Fischer, J. B., Gervain, J., Hirsch, J., Hong, K.-S., Horstmeyer, R., Kainerstorfer, J. M., Ko, T. S., Licht, D. J., Liebert, A., Luke, R., Lynch, J. M., Mesquida, J., Mesquita, R. C., Naseer, N., Novi, S. L., Orihuela-Espina, F., O’Sullivan, T. D., Peterka, D. S., Pifferi, A., Pollonini, L., Sassaroli, A., Sato, J. R., Scholkmann, F., Spinelli, L., Srinivasan, V. J., St. Lawrence, K., Tachtsidis, I., Tong, Y., Torricelli, A., Urner, T., Wabnitz, H., Wolf, M., Wolf, U., Xu, S., Yang, C., Yodh, A. G., Yücel, M. A., Zhou, W., 2022. Optical imaging andspectroscopy for the study of the human brain: status report. Neurophotonics 9. Ayaz, H., Dehais, F., 2019. Neuroergonomics: The Brain at Work and Everyday Life.

Elsevier Academic Press. Ayaz, H., Dehais, F., 2021. NEUROERGONOMICS. In: Salvendy, G., Karwowski, W. (Eds.), HANDBOOK OF HUMAN FACTORS AND ERGONOMICS. Wiley, pp. 816–841. Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P. A., McKendrick, R., Parasuraman,

R., 2013. Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci. 7, 871.

Barreto, C., Bruneri, G. de A., Brockington, G., Ayaz, H., Sato, J. R., 2021. A New Statistical Approach for fNIRS Hyperscanning to Predict Brain Activity of Preschoolers’ Using Teacher’s. Front. Hum. Neurosci.15. Beyer, K. M. M., Szabo, A., Hoormann, K., Stolley, M., 2018. Time spentoutdoors,

activity levels, and chronic disease among American adults. J. Behav. Med. 41, 494–503. Bratman, G. N., Olvera-Alvarez, H. A., Gross, J. J., 2021. The affective benefits of nature exposure. Soc. Personal. Psychol. Compass 15. Caruelle, D., Gustafsson, A., Shams, P., Lervik-Olsen, L., 2019. The use of electroder-

mal activity (EDA) measurement to understand consumer emotions – A literature review and a call for action. J. Bus. Res. 104, 146–160. Cauli, B., 2010. Revisiting the role of neurons in neurovascular coupling. Front.

Neuroenergetics 2, 1–8. ChuDuc, H., NguyenPhan, K., NguyenViet, D., 2013. A Review of Heart Rate Variability and its Applications. APCBEE Procedia 7, 80–85. Curtin, A., Ayaz, H., 2018. The Age of Neuroergonomics: Towards Ubiqui-

tous and Continuous Measurement of Brain Function with fNIRS: Theage of neuroergonomics and fNIRS. Jpn. Psychol. Res. 60, 374–386. Dehais, F., Karwowski, W., Ayaz, H., 2020. Brain at Work and in Everyday Life as the Next Frontier: Grand Field Challenges for Neuroergonomics. Front. Neuroergonomics 1. Foster, J. K., Eskes, G. A., Stuss, D. T., 1994. The cognitive neuropsychology of

attention: A frontal lobe perspective. Cogn. Neuropsychol. 11, 133–147. Gateau, T., Ayaz, H., Dehais, F., 2018. In silico vs. Over the Clouds: On-the- Fly Mental State Estimation of Aircraft Pilots, Using a Functional Near Infrared Spectroscopy Based Passive-BCI. Front. Hum. Neurosci. 12. Holtzer, R., Shuman, M., Mahoney, J. R., Lipton, R., Verghese, J., 2010. Cogni-

tive Fatigue Defined in the Context of Attention Networks. Aging Neuropsychol. Cogn. 18, 108–128. Hoshi, Y., Tsou, B. B. H., Billock, V. A. V., Tanosaki, M., Iguchi, Y., Shimada, M., Shinba, T., Yamada, Y., Oda, I., 2003. Spatiotemporal characteristics of hemody- namic changes in the human lateral prefrontal cortex during working memory tasks. Neuroimage 20, 1493–1504. Kaplan, S., 1995. The restorative benefits of nature: Toward an integrative framew-

ork. J. Environ. Psychol. 15, 169–182. Kaplan, S., Berman, M. G., 2010. Directed Attention as a Common Resource for Executive Functioning and Self-Regulation. Perspect. Psychol. Sci.5, 43–57. Kjellgren, A., Buhrkall, H., 2010. A comparison of the restorative effect of a natural

environment with that of a simulated natural environment. J. Environ. Psychol. 30, 464–472. Linnell, K. J., Caparos, S., de Fockert, J. W., Davidoff, J., 2013. Urbanization decreases attentional engagement. J. Exp. Psychol. Hum. Percept. Perform. 39, 1232–1247. Liu, Y., Ayaz, H., Shewokis, P. A., 2017. Multi subject “Learning” forMental Wor-

kload Classification Using Concurrent EEG, fNIRS, and Physiological Measures. Front. Hum. Neurosci. 11. Mark, J., Thomas, N., Kraft, A., Casebeer, W. D., Ziegler, M., Ayaz, H., 2018. Neurofeedback for personalized adaptive training. Adv. Intell. Syst. Comput. 586, 83–94. McKendrick, R., Parasuraman, R., Murtza, R., Formwalt, A., Baccus, W., Paczynski,

M., Ayaz, H., 2016. Into the Wild: Neuroergonomic Differentiation of Hand-Held and Augmented Reality Wearable Displays during Outdoor Navigation with Functional Near Infrared Spectroscopy. Front. Hum. Neurosci. 10, 216.

Evaluating the Restorative Impact of Nature Through Multimodal Mobile Sensing 159

Meidenbauer, K. L., Stenfors, C. U. D., Young, J., Layden, E. A., Schertz, K. E., Kardan, O., Decety, J., Berman, M. G., 2019. The gradual development of the preference for natural environments. J. Environ. Psychol. 65, 101328. Mollazadeh, M., Zhu, Y., 2021. Application of Virtual Environments forBiophilic

Design: A Critical Review. Buildings 11, 148. Parasuraman, R., 2003. Neuroergonomics: Research and practice. Theor. Issues Ergon. Sci. 4, 5–20. Pasini, M., Berto, R., Brondino, M., Hall, R., Ortner, C., 2014. How to Measure the

Restorative Quality of Environments: The PRS-11. Procedia -Soc. Behav. Sci. 159, 293–297. Perrins, S. P., Varanasi, U., Seto, E., Bratman, G. N., 2021. Nature atwork: The effects of day-to-day nature contact on workers’ stress and psychological well- being. Urban For. Urban Green. 66, 127404. Scott, E. E., LoTemplio, S. B., McDonnell, A. S., McNay, G. D., Greenberg, K.,

McKinney, T., Uchino, B. N., Strayer, D. L., 2021. The autonomic nervous system in its natural environment: Immersion in nature is associated with changes in heart rate and heart rate variability. Psychophysiology 58. Sequeira, H., Hot, P., Silvert, L., Delplanque, S., 2009. Electrical autonomic correlates of emotion. Int. J. Psychophysiol. 71, 50–56. Sharma, Mahima, Kacker, S., Sharma, Mohit, 2016. A Brief Introduction and Review

on Galvanic Skin Response. Int. J. Med. Res. Prof. 2. Shewokis, P. A., Shariff, F. U., Liu, Y., Ayaz, H., Castellanos, A., Lind, D. S., 2017.

Acquisition, retention and transfer of simulated laparoscopic tasks using fNIR and a contextual interference paradigm. Am. J. Surg. 213, 336–345. Shi, Y., Ruiz, N., Taib, R., Choi, E., Chen, F., 2007. Galvanic skin response (GSR) as an index of cognitive load. In: CHI ‘07 Extended Abstracts on Human Factors in Computing Systems. Presented at the CHI07: CHI Conference on Human Factors in Computing Systems, ACM, San Jose CAUSA, pp. 2651–2656. Stevenson, M. P., Schilhab, T., Bentsen, P., 2018. Attention Restoration Theory II:

a systematic review to clarify attention processes affected by exposure to natural environments. J. Toxicol. Environ. Health Part B21, 227–268. Thayer, J. F., Åhs, F., Fredrikson, M., Sollers, J. J., Wager, T. D., 2012. A metaanalysis of heart rate variability and neuroimaging studies: Implications for heart rate variability as a marker of stress and health. Neurosci. Biobehav. Rev. 36, 747–756. Tognoli, J., 1973. The Effect of Windowless Rooms and Unembellished Surroundings

On Attitudes and Retention. Environ. Behav. 5, 191–201. Topoglu, Y., Watson, J., Suri, R., Ayaz, H., 2020. Electrodermal Activity in Ambulatory Settings: A Narrative Review of Literature. pp. 91–102. Ulrich, R. S., Simons, R. F., Losito, B. D., Fiorito, E., Miles, M. A., Zelson, M., 1991.

Stress recovery during exposure to natural and urban environments. J. Environ. Psychol. 11, 201–230. Williams, K. J. H., Lee, K. E., Hartig, T., Sargent, L. D., Williams, N. S. G., Johnson, K. A., 2018. Conceptualising creativity benefits of nature experience: Attention restoration and mind wandering as complementary processes. J. Environ. Psychol. 59, 36–45. World Urbanization Prospects: The 2018 Revision, 2019., (ST/ESA/SER. A/420).

United Nations, Department of Economic and Social Affairs, Population Division, New York.

Cover of the proceedings volume Neuroergonomics and Cognitive Engineering

Explore the complete digital volume

Neuroergonomics and Cognitive Engineering

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-78-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →