Rafael De Pinho Andre, Almir Fonseca, Lucas Westfal (2024). NurseAid Monitor: A Non-Invasive Monitor to Assess Respiratory Rate and Pattern of Bedridden Patients. In: Health Informatics and Biomedical Engineering Applications. AHFE Open Access, vol. 142, pp. 97–106. https://doi.org/10.54941/ahfe1005075
The clinical management of bedridden patients necessitates meticulous attention to their respiratory health, as their constrained mobility significantly increases the risk of respiratory complications. Considering the critical link between respiratory function and recovery outcomes, this research underscores the importance of monitoring respiratory frequency and patterns as an essential aspect of care for these individuals. Diligent observation of respiratory parameters enables healthcare providers to identify early signs of deterioration in respiratory health, allowing for timely intervention and, consequently, a reduction in the incidence of serious complications. We propose a platform based on non-invasive contactless Infrared thermography that analyzes respiratory frequency and patterns. With the help of volunteers, we conducted an experiment to collect data for statistical treatment and modeling. Our results, discussed in this work, substantiate the data collection approach and the selected methodology.
Bedridden patients, due to their limited mobility and prolonged periods of immobility, are at a heightened risk for developing complications that can significantly impede their recovery process, with respiratory complications being among the most critical to address. The immobilized state of these patients not only predisposes them to pneumonia and atelectasis but also contributes to a decline in the overall functionality of the respiratory system. Hence, the clinical care of bedridden patients necessitates a comprehensive approach, with proper observation and maintenance of respiratory health. It posits that through early detection of alterations in respiratory param- eters, healthcare professionals can intervene promptly, thus mitigating the risk of severe respiratory complications and facilitating a smoother recovery trajectory. Henceforth, proactive monitoring of respiratory frequency and patterns is not just a preventative measure, but a vital practice that supports the overall recovery process of bedridden patients and has pivotal role in a comprehensive clinical care.
In the preceding half-decade, the realm of Mobile Health (mHealth) has witnessed a remarkable ascension, emerging as a pivotal force in the transformation of healthcare delivery across the globe. This period has been characterized by the rapid development and adoption of mobile technolo- gies designed to enhance the accessibility, efficiency, and personalization of healthcare services. This technological evolution has not only facilitated real- time monitoring and management of chronic conditions but also significantly improved the capacity for early disease detection and intervention. The inte- gration of advanced analytics, artificial intelligence, and telemedicine into mHealth platforms has further propelled the sector, enabling personalized healthcare recommendations and remote patient-physician interactions. Con- sequently, the last five years have indubitably solidified the role of mHealth as an indispensable component of modern healthcare systems, heralding a new era of healthcare that is both universally accessible and fundamentally aligned with the needs of the digital age. Usually, mHealth platforms are based on two main concepts: medical information collection and medical information display.
Concerning medical information collection, we propose a monitoring plat- form that employs non-invasive contactless Infrared thermography to collect respiratory data and analyse a patient’s breathing frequency and pattern. To protect the patients’ privacy and autonomy and allow for deployment on diverse environments, such as home care, we employed low-resolution, low- cost thermal sensors to collect patients real-time information. Our results show that the platform and the modeling solution is reliable for long-term respiratory frequency monitoring of regular patients, although a refinement is required for intensive care units and operating rooms due to the strict regulatory requirements of those environments.
Concerning medical information display, we are developing a second iteration of the platform presented in this work to provide a data visual- ization layer to help displaying and understanding the patient’s information - considering that visual management in high-pressure medical environments optimizes the workflow of nurses and caregivers, enhances patient recovery outcomes and reduces the chances of oversight or errors (Arora et al., 2022; Kuge et al., 2021; Enshaeifar et al., 2020). This new version of the platform will address the above-mentioned required refinement of the modeling for critical applications.
Section Contactless Data Acquisition discusses methods of respiratory fre- quency assessment used in the literature. The design of the platform, sensor deployment, replication information and data collection are shown on section Building the Platform and Data Collection. Section Data Processing shows the statistical treatment of the data and the mathematical modeling. Con- clusion and future work are discussed on section Conclusion and Future Work.
CONTACTLESS DATA ACQUISITION This section presents a discussion of contactless data acquisition methods for respiratory frequency estimation. We conducted a review of the related works
NurseAid Monitor: A Non-Invasive Monitor to Assess Respiratory Rate 99
on relevant databases such as, although not limited to, ACM Digital Library, IEEE Xplore, NIH, PubMed and Springer Link.
Our review suggests that methods for contactless data acquisition compatible with our approach are:
Analysis of the difference between frames of a video;
Monitoring the temperature of the air around the nostrils or mouth; and,
Monitoring the movement of the thoracic cage with distance sensors.
Analysis of the Difference Between Frames of a Video Digital images are sets of pixels organized spatially, where each pixel is usu- ally represented by integer numbers that define its color, such as RGB cameras that identify colors as three (Red, Green, Blue) integer components. IR ther- mal cameras usually represent the temperature of each pixel with a single integer. The variation of these values carries information about respiratory frequency, which can be extracted by (i) Monitoring the variation in pixel intensity over time with digital cameras or (ii) Monitoring the luminous flux between pixels over time with digital cameras.
Each pixel is the result of the exposure of an optical sensor to radiation emitted by a part of the captured object’s surface, and therefore will be an average of the temperatures of that region. Even minor movements on the observed surface cause variation in the temperature average of nearby pixels, which can be used for the estimation of respiratory frequency. Works, such as (Massaroni et al., 2019), use this method to monitor the variation in pixel intensity over time.
The method of monitoring the luminous flux between pixels over time, less common in the literature, is based on the capture of multiple images over time and the analysis of their variation to obtain information about the captured object’s surface. Works, such as (Queiroz et al., 2020), have employed this method successfully.
Monitoring the Temperature of the Air Around the Nostrils or Mouth Infrared thermography technologies are designed to be sensitive to wave- lengths larger than that of visible light, exceeding the capacity of human vision; in particular, they capture the light that is radiated by the heat of warmed bodies and allow for the perception of brightness differences in the nostril area during breathing. The research conducted in (Murthy and Pavlidis, 2006) exemplifies this methodology.
Monitoring the Movement of the Thoracic Cage With Distance Sensors During breathing, the variation in lung volume causes the thoracic cage to expand, and there is a range of methods aimed at measuring the depth of this expansion. The research conducted in (Massaroni et al., 2021) categorizes these methods as active and passive.
In active methods, specialized sensors emit signals that travel through the air and are reflected by the captured object’s surface. The main approaches are the use of RGB-D cameras - that capture a combination of a RGB image and its corresponding depth image, better known as depth map - and infrared ToF (Time-of-Flight) distance sensors based on LiDAR (Light Detection and Ranging). The research conducted in (Soleimani et al., 2015) uses a ToF dis- tance sensor to successfully measure the chest movement of 40 people for comparison with spirometry data.
In passive methods, triangulation is employed to analyze images of the captured object’s surface from different perspectives, to map the surface and its variations. Studies, such as (Bernal et al., 2014), use this approach.
The IR thermal cameras used in the study are classified as passive devices. We opted for low-resolution (32x24 pixels) sensors, not suitable for methods that use triangulation to measure chest expansion. We chose the analy- sis of the difference between frames of a video to estimate the respira- tory frequency, as discussed on section Building the Platform and Data Collection.
BUILDING THE PLATFORM AND DATA COLLECTION On this Section, we describe the stages followed to prototype the platform and implement the infrastructure for data collection.
Building the Platform We employed the MLX90640 infrared thermal array sensor from Melexis, notable for its compact form factor and cost-effectiveness, featuring a 32x24 pixel resolution. This sensor is distinguished by its ease of inte- gration, facilitated by an I2C-compatible digital interface, and its capacity for high-precision, non-contact temperature measurements. The MLX90640 offers a broad field of view (FoV) of 110
x75
and a programmable refresh rate ranging from 0.5Hz to 64Hz, ensuring versatility in various applications.
The MLX90640, encapsulated within the M5 Thermal Unit by M5 Stack, was interfaced with the M5 Core Capsule – an ESP32-S3-based micro- controller from the same manufacturer. This controller is equipped with a 2.4GHz 802.11b/g/n WiFi transceiver and dual-mode Bluetooth® (classic and low-energy), enabling efficient data acquisition and transmission. Data, timestamped using NTP servers, was relayed to a remote real-time database. Communication between these components was achieved using the I2C pro- tocol via the Groove cables, connecting essential pins (VCC, GND, SDA, SCL) provided by M5 Stack.
The importance of precisely capturing subtle movements in amplitude curves, aimed at monitoring the body’s breathing during inhalation and exhalation, has guided our approach to configure our device to operate at the highest feasible sampling rate, considering the capabilities for com- pression, real-time transmission, and database storage, achieving a rate of 16Hz. However, due to the particularity of each sensor frame being com- posed of two sub-frames called ‘pages,’ the effective sampling rate adopted
NurseAid Monitor: A Non-Invasive Monitor to Assess Respiratory Rate 101
was 8Hz. Although this rate may seem modest for capturing the respira- tory phenomenon with the desired precision, it is crucial to emphasize that this sampling rate is sufficient to reconstruct the continuous signal from the samples and detect variations of up to 240 breaths per minute according, to the Nyquist-Shannon Sampling Theorem, thus exceeding the monitoring requirements of an individual during sleep and validating the applicability of our approach (Bai et al., 2023).
Data Collection Using an individual lying on their back as a reference, we positioned the sen- sor on the right side of the bed, at the height of the lower part of the sternum - the xiphoid process - and parallel to the sternum itself. The image orientation was set to landscape, providing coverage from the waist to the head and fully encompassing the thoracic wall and the abdomen. We collected data in 3 dis- tinct positions: supine (lying on the back, with the belly facing up), left lateral decubitus (ventral side facing the sensor), and right lateral decubitus (back facing the sensor). Considering the significant variability in the respiratory rate over an individual’s lifetime (Bai et al., 2023), we used the adult breath- ing rate during sleep as a reference, which can vary from 12 to 20 breaths per minute (0.2 to 0.33Hz). Given this rate, we simulated breathing frequen- cies of 0.1, 0.2, 0.3, 0.4, and 0.5Hz, covering a wide range of frequencies to improve the model’s robustness. The data collection window was limited to 60 seconds, with intervals between each collection, in order to avoid the onset of hypoventilation or hyperventilation symptoms. A metronome was used to ensure the accuracy and consistency of the simulated frequency, generating samples with constant respiratory frequencies.
Noise Treatment and ROI Definition Given the low-resolution and the accuracy range of IR the thermal sensor, which varies between ±0.5
C and ±4
C depending on the frame area, we only monitored temperature variations at the edges of the individual’s body. This approach focus on the observation of the movements of the thoracic wall during breathing (De Groote et al., 1997) - the expansion and contraction of the rib cage in the lateral and ventral directions. Thus, it became necessary to establish a Region of Interest (ROI) to limit the observed area, thereby avoiding the processing of areas that do not reflect respiratory movements.
The definition of a ROI based on a rectangular boundary box proved inef- fective, as it does not conform well to the curves of the chest and abdomen, nor is it robust to user movements, such as those that may occur during an episode of obstructive apnea. Moreover, adopting fixed thresholds could lead to inconsistent results, as minute variations occur regularly in ambient tem- perature. Therefore, we developed a dynamic mask to focus exclusively on the edges of the individual’s body in the images. The algorithm used is based on Otsu’s method (Otsu, 1979) to binarize the image, differentiating between the foreground (the individual’s body) and the background (the environment).
To refine the resulting edge, and mitigate noise in areas of low accuracy and regions that provide little information about the individual’s respira- tory rate, several additional steps were implemented. The mask’s tolerance is determined by the dilation kernel, which can be adjusted to accommo- date movements of different amplitudes or different sensor placements. The suppression of peripheral zones is crucial for noise reduction, as these areas are identified in the IR thermal sensor’s datasheet as prone to significant distortions.
In addition to discarding the pixels of the regions prone to noise and strate- gically positioning the sensor to improve the collected data quality, the time series used for Fourier analysis consists of the average pixel luminosity in the ROI. Assuming that the noise of the ROI pixels are IID (Independently and Identically Distributed) with a mean of zero, the Law of Large Numbers ensures that the data is robust against sensor fluctuations.
DATA PROCESSING AND FORECASTING On this Section, we describe the statistical treatment of the data and the mathematical modeling.
After ROI definition and the noise treatment process discussed on section Building the Platform and Data Collection, the resulting data is a uni- dimensional time series. This data, consisting of uniformly frequent breaths, is subsequently processed using Discrete Fourier Transform (DFT) for fre- quency space analysis and signal analysis through band-pass filtering (BPF). To ensure stability for data analysis in the frequency space we employed ADF tests with a generic critical value of 5%.
The data, when analyzed in the frequency domain via DFT, convey infor- mation regarding the intensity and frequency of motion of each element captured by the thermal camera within the ROI. Subsequently, we fore- cast using the frequency associated with the highest Power Spectral Density (PSD), while adhering to the BPF, as the respiratory frequency is antici- pated to be the primary feature within the dataset. This methodology not only enhances interpretability but also provides a natural means of evalu- ating its significance: it is expected that the frequencies of the time series exhibit a single dominant peak, with secondary peaks being considerably less pronounced.
Effectiveness of the Method and Significance of the Data Figures 1 and 2 illustrate the data extracted from the movement of the tho- racic cage viewed in the frequency domain. The first figure compares the quality of the data acquired in each position in terms of noise and promi- nence of the most significant frequency, while the second plot demonstrates the stability of the method by applying it to data across a wide range of respiratory rates.
NurseAid Monitor: A Non-Invasive Monitor to Assess Respiratory Rate 103
Figure 1: Comparing significance of data collected with the individual in different positions, with selected BPF as reference.
It is evident that collecting data with the camera aligned parallel to the ster- num is preferable, exhibiting a prominent peak intensity and minimal noise. Collecting data with the camera directed towards the ventral side – corre- sponding to the left lateral decubitus – has been demonstrated as feasible, as despite having significantly higher noise compared to the former, the region delimited by the BPF region remains stable. However, data collected with the camera positioned towards the back – corresponding to the right lateral decubitus – has shown an absence of useful respiratory information.
Figure 2: Forecasting the respiratory frequency of the data collected, with selected BPF as reference.
The transformation of supine data using DFT has successfully extracted the desired features of the respiratory process across all collected respiratory frequencies, thereby attesting the robustness of the method and confirming the suitability of the chosen ROI.
CONCLUSION AND FUTURE WORK According to (Young et al., 2002), it is estimated that in the United States, the annual costs associated with obstructive sleep apnea, including direct medical expenses, loss of productivity, and costs related to accidents, are in the range of billions of dollars. It is estimated that 80% of obstructive sleep apnea cases are undiagnosed. Obstructive sleep apnea is associated with an increased risk
NurseAid Monitor: A Non-Invasive Monitor to Assess Respiratory Rate 105
of hypertension, stroke, and cardiovascular diseases, conditions that further increase the real annual cost imposed by sleep apnea. Whereas in Brazil, where this work was developed, a study (Tufik et al., 2010) indicated that the prevalence of obstructive sleep apnea is approximately 40% in men and 26% in women, affecting more than 80% of obese men and 52% of women in this condition.
This ongoing work introduced an IR thermal imaging process for esti- mating respiratory frequency. However, we aim to help detect and diagnose sleep apnea using contactless data collection that (i) protects user privacy and (ii) has low cost to enable mass adoption by the Public Health Sys- tem. Future initiatives will focus on aligning the platform’s technological capabilities with its clinical integration, leveraging on a recently established partnership with Pedro Ernesto and Gaffrée e Guinle College Hospitals for pilot studies. Through the integration of this solution with dashboard and dynamic reporting methodologies, we plan on assisting healthcare profes- sionals in making informed decisions about patients current and historical respiratory patterns to enhance quality of life and help mitigate the risk of respiratory complications.
REFERENCES
Arora, Teena, Balasubramanian, Venki and Mai, Shenhan (2022). Prior- itization of Clinical Alarms using Semantic Features of Vital Signs in Remote Patient Monitoring, in: Proceedings of the 2022 Australasian Com- puter Science Week, pp. 242–245. Association for Computing Machinery. https://doi.org/10.1145/3511616.3513124 Bai, Zhen-Min, Sun, Yi-Ting, Liang, Wen-Ming, Truskauskait˙e, Inga, Yan, Miao-E, Li, Chun-Ri, Xiao, Jing, Aihemaiti, Maiwulamu, Yuan, Lei and Rukš˙enas, Osvaldas (2023). Respiratory Movements at Different Ages, Volume 59, No. 1024. https://www.mdpi.com/1648-9144/59/6/1024 Bernal, Edgar A., Mestha, Lalit K. and Shilla, Eribaweimon (2014). Non contact
monitoring of respiratory function via depth sensing, in: IEEE-EMBS Interna- tional Conference on Biomedical and Health Informatics (BHI), pp. 101–104. https://doi.org/10.1109/BHI.2014.6864314 De Groote, A., Wantier, M., Cheron, G., Estenne, M., and Paiva, M. (1997).
Chest wall motion during tidal breathing, Volume 83, No. 5, pp. 1531–1537. https://doi.org/10.1152/jappl.1997.83.5.1531 Enshaeifar, Shirin, Barnaghi, Payam, Skillman, Severin, Sharp, David, Nilforooshan, Ramin and Rostill, Helen (2020). A Digital Platform for Remote Healthcare Monitoring, in: Companion Proceedings of the Web Conference 2020, pp. 203–206. Association for Computing Machinery. https://doi.org/10.1145/3366424.3383541 Kuge, Jeremias, Grundgeiger, Tobias, Schlosser, Paul, Sanderson, Penelope and
Happel, Oliver (2021). Design and Evaluation of a Head-Worn Display Appli- cation for Multi-Patient Monitoring, in: Proceedings of the 2021 ACM Design- ing Interactive Systems Conference, pp. 879–890. Association for Computing Machinery. https://doi.org/10.1145/3461778.3462011 Massaroni, Carlo, Nicolò, Andrea, Sacchetti, Massimo and Schena, Emiliano (2021).
Contactless Methods For Measuring Respiratory Rate: A Review, Volume 21, No. 11, pp. 12821–12839. https://doi.org/10.1109/JSEN.2020.3023486
Massaroni, Carlo, Schena, Emiliano, Silvestri, Sergio and Maji, Soumyajyoti (2019). Comparison of two methods for estimating respiratory waveforms from videos without contact, in: 2019 IEEE International Sympo- sium on Medical Measurements and Applications (MeMeA), pp. 1–6. https://doi.org/10.1109/MeMeA.2019.8802167 Murthy, R. and Pavlidis, I. (2006). Noncontact measurement of breathing function,
Volume 25, No. 3, pp. 57–67. https://doi.org/10.1109/MEMB.2006.1636352 Otsu, Nobuyuki (1979). A Threshold Selection Method from Gray-Level Histograms, Volume 9, No. 1, pp. 62–66. https://doi.org/10.1109/TSMC.1979.4310076 Queiroz, Leonardo, Oliveira, Helder, Yanushkevich, Svetlana and Ferber, Reed (2020). Video-Based Breathing Rate Monitoring in Sleeping Subjects, in: 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 2458–2464. https://doi.org/10.1109/SMC42975.2020.9283398 Soleimani, Vahid, Mirmehdi, Majid, Damen, Dima, Hannuna, Sion, Camplani, Mas-
simo, Viner, Jason and Dodd, James (2015). Remote pulmonary function testing using a depth sensor, in: 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS), pp. 1–4. https://doi.org/10.1109/BioCAS.2015.7348445 Tufik, S., Santos-Silva, R., Taddei, J. A., and Bittencourt, L. R. (2010). Obstructive sleep apnea syndrome in the Sao Paulo Epidemiologic Sleep Study, Volume 11, No. 5, pp. 441. https://doi.org/10.1016/j.sleep.2009.10.005 Young, T., Peppard, P. E., and Gottlieb, D. J. (2002). Epidemiology of obstruc-
tive sleep apnea: a population health perspective, in: American journal of respiratory and critical care medicine, Volume 165, No. 9, pp. 1217–1239. https://doi.org/10.1164/rccm.2109080
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Health Informatics and Biomedical Engineering Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-964867-18-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional