Amandeep Singh, Siby Samuel, Jagmeet Singh, Yash Kumar Dhabi (2023). Internet of Things (IoT) based Drowsiness Detection and Intervention System. In: Human Interaction and Emerging Technologies (IHIET-AI 2023): Artificial Intelligence and Future Applications. AHFE Open Access, vol. 70, pp. 296–304. https://doi.org/10.54941/ahfe1002955
This study aimed to develop a non-intrusive smart monitoring system that could identify and prevent drowsy driving, reducing the risk of accidents. The study developed a system that uses video processing to measure the Euclidean distance of the eye and an eye aspect ratio (EAR) in order to detect drowsiness. The system employed face recognition to accurately identify the driver's eye aspect ratio. An Internet of Things (IoT) module used for remote assessment of the driver's drowsiness response in real-time. If the driver is in a drowsy state, the system sends an alert/warning to the driver and relevant authorities. In addition, if a crash occurs, the system sends a warning message with the location of the collision. The system was tested on 20 participants, achieving an overall eye detection accuracy of 99.98% (with glasses), 99.89% (without glasses), and a drowsiness detection accuracy of 98.05% (with glasses) and 99.05% (without glasses). This system has the potential to be implemented in a variety of driving applications, where expensive technologies are often difficult to adopt.
In recent years, technology has improved road safety with tools like “Drowsy Driver Detection Systems” (DDDS) created by major vehicle manufacturers. These systems detect a driver’s drowsiness to prevent crashes caused by exh- austion, difficult road conditions, and bad weather (Vural et al., 2007; Fisher, 2015). The World Health Organization (WHO) and the National Highway Traffic Safety Administration report that over 1.35 million people die from vehicle crashes worldwide, and inexperienced driving is the most influen- tial factor in serious road crashes. In general, road crashes are caused by inexperienced driving, and driver drowsiness is recognized as a significant
Internet of Things (IoT) Based Drowsiness Detection and Intervention System 297
factor in fatal mishaps. When drivers fall asleep, they lose vehicle control. The evaluation of driver drowsiness can be performed using multiple meth- ods, including vehicle-based, psychological, and behavioral metrics. These metrics are analyzed using various analytical algorithms to determine drow- siness. An effective way to detect driver drowsiness is through the use of a Machine Learning (ML) algorithm known as Convolutional Neural Network (CNN), which analyzes a driver’s facial features captured by a camera to dia- gnose drowsiness (Jabbar et al., 2020). This approach is accurate and works well in different lighting conditions and when drivers wear eyeglasses or spe- ctacles. Another study used Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) to analyze drivers’ behavior using digital sen- sors (Saleh et al., 2017). Using a multi-layered CNN approach, this real-time detection technology can prevent crashes by categorizing driver actions and identifying drowsiness with a 92% accuracy rate. A low-invasive drowsiness recognition system that uses a field programmable gate array (Navaneethan et al., 2019). This system utilizes an infrared light source embedded in the vehicle to identify eye pupils, which allows the retinas to be detected up to 90% of the time. The system analyzes the driver’s eyes over multiple fra- mes to track tiredness and prevent major accidents. Real-time pupil tracking is implemented using the Cyclone II FPGA. A system that uses the Wavelet Network (WN) Algorithm is also available to detect drowsiness (Babaeian, 2019). This system monitors the eyes using classification algorithms, such as the WN Classifier. It also uses physiological factors such as heart rate and ECG, which are analyzed using a wavelet transformation and regression technique. A wavelet network is then used to organize the heart rate data and identify a pattern for detecting drowsiness. The Vehicle State Algorithm (VSA) is another method to create a non-invasive system for detecting driver drowsiness (Mutya et al., 2019). This method uses a neuro-fuzzy technique and a support vector machine, which gathers data from the vehicle control system.
Additionally, a steering wheel algorithm has been proposed as a solution to drowsiness (Budak et al., 2019). This algorithm uses image-based steering and the Convolutional Neural Network (CNN) algorithm to track drow- siness accurately. Researchers have used various algorithms, such as ECG, EEG, and EMG, to detect drowsiness (Hayawi and Waleed, 2019; Song et al., 2019; Artanto et al., 2017). For instance, an EEG-based system was develo- ped using the AlexNet, VGGNet, and WN algorithms, which use sensors to detect drowsiness and alert the driver. Another study used Heart Rate Variabi- lity signals with an EOG technique to develop a tiredness alarm system, which achieved high accuracy using a K-nearest neighbor classifier. The EMG and human-machine interface were also used to analyze muscle skin movement and assess drowsiness. Another method used is eye-blinking and collision sensors to alert authorized users through a vibration sensor and heart rate detector (Kinage and Patil, 2019; Jang and Ahn, 2020). A smart system com- bining sensors, a camera, and an Arduino board was developed to detect drowsiness with IoT message delivery and ML technologies (Samanta et al., 2020). A new approach includes regular monitoring of eye blinks and face
Figure 1: Flow chart of the smart drowsiness detection system.
detection, with additional sensors and audio software to enhance alerting (Samuel et al., 2020).
Drowsiness detection systems currently available are expensive, limiting their widespread use. To address this issue, this study developed a cost- effective and reliable solution to decrease accidents caused by drowsy driving. An IoT module with ESP 32 and OV2640 camera modules was incorporated into the system to send an email notification to the relevant department in case of an accident. This IoT device shares the GPS location with emergency services and the vehicle owner to assist in rescuing the driver.
METHODOLOGY
This paper presents a smart drowsiness detection system that utilizes adva- nced technology to detect and prevent drowsy driving. The system uses a combination of image processing, face recognition, and eye blink rate dete- ction to monitor a driver’s drowsiness behavior in real time. The system is designed to provide warning alert in case of drowsiness or crash through various alerts such as a ringing phone, sending an auto-generated email, acti- vating collision sensors, and enabling GPS tracking. The general methodology of the system is shown in the accompanying flow chart as shown in Figure 1, which depicts the various steps involved in the process, from vehicle start to alert generation.
A smart system can transmit data to the cloud and server to detect drow- siness. This requires a microcontroller compatible with Wi-Fi, a force sensor,
Internet of Things (IoT) Based Drowsiness Detection and Intervention System 299
an impact switch, and a camera module for image processing. The study uses the ESP32 series module with a Tensilica Xtensa LX6 CPU, antenna switches for Wi-Fi processing, a power management module, filters, and a low-noise receive amplifier. The module is compact and energy-efficient, making it suitable for various IoT applications. The ESP32-D0WDQ6 chip can be connected to this module to ensure scalable and adaptable functi- ons. The module requires 3.3v DC for operation, and a DC-DC converter is needed to convert 12v DC to 24v DC. Detailed technical specifications are provided below:
Main Processor: Tensilica Xtensa 32-bit LX6 microprocessor;
Wireless Connectivity Wi-Fi: (802.11n @ 2.4 GHz up to 150 Mbit/s);
Bluetooth: v4.2 BR/EDR and Bluetooth Low Energy (BLE);
ROM: 448 KB For booting and core functions;
SRAM: 520 KB For data and instruction;
RTC Slow SRAM: 8 KB For co-processor accessing during the deep sleep mode;
RTC Fast SRAM: 8 KB For data storage and main CPU during RTC Boot from the deep-sleep mode;
eFuse: 1 K-bit of which 256 bits are used for the system (MAC address and chip configuration), and the remaining 768 bits are reserved for customer applications, including Flash-Encryption and Chip-ID;
Embedded Flash: 0 MB or 2 MB (depending on variation);
External Flash and SRAM: ESP32 without embedded flash supports up to 4×16 mebibytes of external QSPI flash and SRAM with hardware encryption based on AES to protect developer’s programs and data;
Peripheral Input/Output: Rich peripheral interface with DMA that inclu- des capacitive touch, ADCs (analog-to-digital converter), DACs (digital- to-analog converter), I2C (Inter-Integrated Circuit), UART (universal asynchronous receiver/transmitter), CAN 2.0 (Controller Area Network), SPI (Serial Peripheral Interface), I2S (Integrated Inter-IC Sound), RMII (Reduced Media-Independent Interface), PWM (pulse width modulation), and more;
Security: IEEE 802.11 standard security features all supported, inclu- ding WFA, WPA/WPA2, and WAPISecure boot Flash encryption1024-bit OTP, up to 768-bit for customers Cryptographic hardware acceleration: AES, SHA-2, RSA, elliptic curve cryptography (ECC), random number generator (RNG).
We have integrated the OV2640 camera module with the ESP 32 device through a PCB board, as shown in Figure 2. The camera records the move- ments of the driver’s face in real time, and our focus is on analyzing the drowsiness while measuring collision severity. Using the ESP 32 and OV2640 modules together, we can record facial data and eye blinking, which helps us accurately calculate the Eye Aspect Ratio (EAR). The controller’s proces- sing system ensures that the modules are processed safely. This study has developed an Android application and a smart monitoring system by integra- ting smart IoT modules such as GPS sensors and wireless networking. These
Figure 2: Representation of smart drowsiness detection system.Figure 3: Flowchart of the Drowsiness Detection Process Using Face Detection and Eye Blinking Analysis.
modules are connected to the ESP 32 device, which controls and notifies a drowsy driver. This smart monitoring system helps manage various challen- ges in real-time monitoring, such as controlling complex sensing and provides a flexible platform for controlling connectivity. This system is a reliable way to record the driver’s drowsiness and alert the driver with a warning message to avoid driving-related risks.
The drowsiness detection process involves steps, as illustrated in Figure 3. The developed system can monitor the driver’s drowsiness in real time conti- nuously. The proposed approach employs the OV2640 camera for recording video footage of the driver’s face. The recorded video is processed into frames and analyzed using a combination of face features and predictor algorithms to detect drowsiness indicators such as eye movements and blinking. The resul- ting Eye Aspect Ratio (EAR) values are then used to determine the driver’s drowsiness level and activate safety measures if necessary.”
Internet of Things (IoT) Based Drowsiness Detection and Intervention System 301
Figure 4: Six coordinates of the human eye.Figure 5: Mean eye aspect ratio (a) drowsy condition and (b) non-drowsy condition.
RESULTS AND DISCUSSION
One important aspect of the drowsy driver detection system is the Eye Aspect Ratio (EAR) calculation to determine if a driver is becoming drowsy. The EAR is calculated using the Euclidean distance between pairs of n-space points in Cartesian coordinates representing the location of various lan- dmarks on the driver’s face. Specifically, the distance between three points on the eyelid segment (eye P2, eye P6), (eye P3, eye P5), and (eye P1, eye P4) is calculated using the distance package of the scipy library in Python. These distances are then used to calculate the EAR value for each eye using the coor- dinate values of all eye features from all video frames. Figure 4 illustrates six coordinates of the eye.
The features P0 and P3 correspond to the left and right corners of the eye, respectively. The features P1 and P2 correspond to the upper corners of the eye. Similarly, P5 and P4 correspond to the lower corners of the eye, providing the complete set of coordinates for eye tracking. Figure 4 represents the coordinate of one participant out of a total of 20 participants (12 male and 6 female). The mean eye-aspect ratio response of the 20 participants is represented in Figure 5. This Figure illustrates the drowsy condition (a) and non-drowsy condition (b) for the computed eye aspect ratio.
Figure 6: Representation of the email alert.Table 1. Analysis of 1000 frames with and without glasses.
Table available in the original PDF paper.
Face condition ED Email Alarm EED EMD EAD With glasses 989 986 985 1 17 22 Without glasses 998 995 996 5 12 7
*FC: Face condition; ED: Eye detection; EED: Error in eye detection; EMD: Error in email detection; and EAD: Error in alarm detection
When the system detects drowsiness in the driver, it triggers a repeated alarm system, a phone call, and an email notification to the relevant parties responsible for the driver’s safety. The system uses an alarming module to ensure increased alertness, which is more effective than a standard speaker. If the module fails, the system sends additional warnings through a phone call or email to the concerned person. This system is illustrated in Figure 6. The notification process is an important safety feature designed to alert the driver and relevant authorized person promptly.
Driver fatigue or unintentional driving can lead to a car accident, which is a significant concern. Therefore, it is important to use sensors to determine the severity of the situation and locate the crash site quickly for immediate medical attention. A web page is created using a local server to display details about the crash, and it can manage multiple incidents simultaneously. The web page also contains a Google map link that displays the location of the crash in green and the destination in red, as shown in Figure 7. This helps quickly locate and respond to the emergency. The study tested a drowsiness detection system on 20 individuals under various settings, including using glasses as an alternate view. The system was tested for drowsiness and eye movement accuracy and noticed eye blinking and drowsiness on 1000 frames, as shown in Table 1.
The results showed that face detection was more precise than the cascading approach, though it took longer to load in night vision. The technique offered a successful and efficient drowsy detection approach based on face movement and another interface for crash detection. The alarming system warned of
Internet of Things (IoT) Based Drowsiness Detection and Intervention System 303
Figure 7: Locating and Alerting Emergency Services with GPS.
drowsiness through an auto-generated phone call and email. The system’s accuracy in detecting drowsiness was 98.05% with and without glasses and 99.89% and 99.98% for the complete analysis of the examined results with and without glasses, respectively.
CONCLUSION AND FUTURE IMPLICATIONS This study developed a low-cost system to detect driver drowsiness based on eye aspect ratio. The system can alert the driver through email and phone calls if the eye-aspect ratio crosses the recommended threshold limit. It also sends GPS location in case of drowsiness or collision. The system effecti- vely detects drowsiness regardless of whether the driver wears glasses or not. However, the developed system has been tested using a limited number of par- ticipants. Future research should investigate the system’s effectiveness with a larger number of participants and under varying driving conditions. Additio- nally, the system’s effectiveness may differ for people of different ages, gender, or experience. Future research should explore the feasibility of widespread dissemination of drowsiness detection and intervention systems.
ACKNOWLEDGMENT
The authors would like to acknowledge AysRox Solutions Pvt Limited for their technical assistance in conducting this research.
REFERENCES
Artanto, D., Sulistyanto, M.P., Pranowo, I.D. and Pramesta, E.E., 2017, November. Drowsiness detection system based on eye-closure using a low-cost EMG and ESP8266. In 2017 2nd International conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE) (pp. 235–238). IEEE.
Babaeian, M., Amal Francis, K., Dajani, K. and Mozumdar, M., 2019. Real-time driver drowsiness detection using wavelet transform and ensemble logistic regres- sion. International Journal of Intelligent Transportation Systems Research, 17, pp. 212–222. Budak, U., Bajaj, V., Akbulut, Y., Atila, O. and Sengur, A., 2019. An effective
hybrid model for EEG-based drowsiness detection. IEEE sensors journal, 19(17), pp. 7624–7631. Fisher, D.L., 2015. Cognitive distraction in the wild: Next steps—Addressing a notso-humdrum conundrum. Human factors, 57(8), pp. 1331–1333. Hayawi, A.A. and Waleed, J., 2019, August. Driver’s drowsiness monitoring and
alarming auto-system based on EOG signals. In 2019 2nd International Confe- rence on Engineering Technology and its Applications (IICETA) (pp. 214–218). IEEE. Jabbar, R., Shinoy, M., Kharbeche, M., Al-Khalifa, K., Krichen, M. and Barkaoui, K., 2020, February. Driver drowsiness detection model using convolutional neural networks techniques for android application. In 2020 IEEE International Con- ference on Informatics, IoT, and Enabling Technologies (ICIoT) (pp. 237–242). IEEE. Jang, S.W. and Ahn, B., 2020. Implementation of detection system for drowsy driving
prevention using image recognition and IoT. Sustainability, 12(7), p. 3037. Kinage, V. and Patil, P., 2019, December. Iot based intelligent system for vehicle accident prevention and detection at real time. In 2019 Third International conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC) (pp. 409–413). IEEE. Mutya, K., Shah, J., McDonald, A.D. and Jefferson, J., 2019, November. What are
steering pictures are worth? using image-based steering features to detect dro- wsiness on rural roads. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 63, No. 1, pp. 2041–2045). Sage CA: Los Angeles, CA: SAGE Publications. Navaneethan, S., Nandhagopal, N. and Nivedita, V., 2019. An FPGA-based realtime human eye pupil detection system using E2V smart camera. Journal of Computational and Theoretical Nanoscience, 16(2), pp. 649–654. Saleh, K., Hossny, M. and Nahavandi, S., 2017, October. Driving behavior classifica-
tion based on sensor data fusion using LSTM recurrent neural networks. In 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) (pp. 1–6). IEEE. Samanta, D., Shivamurthaiah, M., Kumar, P., Umesh, D. and Siddalingappa, K.,
2020. Wireless sensor networks model for monitoring system based on iot. Solid State Technology, 63(5), pp. 1102–1106. Samuel, S., Yamani, Y. and Fisher, D.L., 2020. Understanding drivers’ latent hazard anticipation in partially automated vehicle systems. International journal of human factors and ergonomics, 7(3), pp. 282–296. Vural, E., Cetin, M., Ercil, A., Littlewort, G., Bartlett, M. and Movellan, J., 2007.
Drowsy driver detection through facial movement analysis. In Human–Computer Interaction: IEEE International Workshop, HCI 2007 Rio de Janeiro, Brazil, October 20, 2007 Proceedings 4 (pp. 6–18). Springer Berlin Heidelberg.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Human Interaction and Emerging Technologies (IHIET-AI 2023): Artificial Intelligence and Future Applications
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-46-9
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional