Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Early detection of risk for cognitive decline using mobile apps and eye tracking-based biomarkers

1 Joanneum Research Forschungsgesellschaft mbH, Graz, Austria ↗
2 Soziale Dienste Süd-Weststeiermark, Graz, Austria ↗
Publication year
Digital Object Identifier10.54941/ahfe1006092

Abstract

Early detection of Mild Cognitive Impairment (MCI), a precursor to Alzheimer’s disease, is essential for timely interventions. However, traditional cognitive assessments are often inaccessible and unsuitable for continuous monitoring. This study presents a mobile, gaze-based assessment system using eye-tracking as a digital biomarker for cognitive decline. Fourteen older adults with MCI used gamified apps over four months, including an emotionally weighted object-tracking task (PAIRS; Paletta et al., 2020a), an antisaccade task (Mobile Instrumental Recovery of Attention; MIRA; Paletta et al., 2020b), and the psychomotor vigilance task (PVT; Dinges & Powell, 1985). Eye movement features such as blink rate and reaction time significantly correlated with scores of Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) scores. A Support Vector Regression model estimated cognitive scores supporting the potential of mobile eye-tracking for home-based cognitive monitoring and early dementia risk detection.

INTRODUCTION

Dementia, particularly Alzheimer’s disease, is a growing global health concern, with MCI often representing a precursor stage (Petersen et al., 2014). Early detection of MCI is crucial, as it provides a valuable opportunity to initiate lifestyle changes or therapeutic interventions that may slow cognitive decline (Livingston et al., 2020). However, conventional diagnostic procedures—typically involving neuropsychological assessments—are resource-intensive, infrequent, and often inaccessible outside clinical settings (Albert et al., 2011). This gap highlights the need for scalable, unobtrusive, and user-friendly tools that support regular cognitive monitoring in real-world environments.

Digital health technologies, particularly those utilizing mobile devices and embedded sensors, are increasingly recognized as promising solutions for cognitive screening (Kourtis et al., 2019). Among them, eye-tracking has emerged as a sensitive modality for detecting early cognitive impairment

© 2025. Published by AHFE Open Access. All rights reserved. 81

due to its strong association with attention, working memory, and executive function (Anderson & MacAskill, 2013; Molitor et al., 2015; Crawford et al., 2005). Importantly, gaze-based measures are non-invasive, require minimal user input, and can be embedded in engaging digital tasks suitable for home use.

Figure 1: Mobile cognitive assessment at home using tablet-based embedded eye tracking and app score analytics.
Original PDF Figure 1:

In this study, we developed a suite of gamified mobile applications incorporating eye-tracking to assess cognitive performance in older adults with MCI. The tasks included: (1) an emotionally weighted object-tracking task (PAIRS), designed to probe attention-emotion interactions relevant to aging and cognitive decline (Paletta et al., 2020a); (2) an antisaccade task (MIRA; Paletta et al., 2020b; Figure 1), which has shown sensitivity to executive dysfunction in MCI (Kaufman et al., 2012); and (3) the psychomotor vigilance task (PVT), a well-established measure of sustained attention (Dinges & Powell, 1985). These tasks were administered weekly over a four-month period, enabling ecologically valid assessment of cognitive state in home settings.

Participants completed the tasks using mobile devices equipped with camera-based eye-tracking software. Significant correlations were found between MoCA total scores and gaze- as well as interaction-derived features, such as, blink rate, and mean reaction time, supporting the validity of these features as digital biomarkers. Moreover, a machine learning model using Support Vector Regression estimated MoCA scores from eye movement features with a mean absolute error of 2.487 points—comparable to inter-rater variability in traditional assessments (Nasreddine et al., 2005; König et al., 2015).

These findings demonstrate the feasibility of mobile gaze-based interfaces for continuous cognitive monitoring. We experienced a high adherence rate to MIRA that further support the usability and acceptability of this approach. By combining principles from human factors, digital health, and cognitive

Early Detection of Risk for Cognitive Decline using Mobile Apps 83

neuroscience, this work advances the development of scalable, playful, and adaptive tools for dementia prevention and early intervention.

Early detection of dementia risk by screening cognitive decline is fundamental for timely adaptation of lifestyle or interventions. These results of digital biomarker development indicate successful steps towards frequent use of mobile gaze-based cognitive assessment. These playful assessment apps offer a promising potential for future long-term monitoring in dementia prevention, early detection as well as in numerous dementia care services.

RELATED WORK

Eye-tracking has long been used in controlled laboratory environments to assess cognitive processes, including attention, memory, and executive function. These studies have established robust associations between eye movement patterns and cognitive decline, particularly in populations at risk for dementia (Molitor et al., 2015). For example, reduced saccade inhibition and altered fixation durations have been observed in individuals with MCI and Alzheimer’s disease during tasks requiring executive control and visual attention (Zhou & Chen, 2021; Kaufman et al., 2012). Traditionally, such assessments required specialized equipment and lab-based protocols, limiting scalability and accessibility. However, recent technological advancements have enabled the deployment of eye-tracking through webcams and front- facing cameras in mobile devices, making cognitive screening more feasible in home settings. Webcam-based eye-tracking has demonstrated acceptable validity for assessing oculomotor behavior, enabling broader population studies outside research laboratories (Semmelmann & Weigelt, 2018).

Studies have begun leveraging mobile applications that integrate gaze estimation to conduct cognitive assessments remotely. For instance, Xu et al. (2021) demonstrated the feasibility of remote cognitive testing using smartphone-based eye-tracking to detect attentional decline in older adults. Similarly, Wolf et al. (2022) described different attempts to collect eye movement features for predictive, early cognitive impairment, supporting the utility of gaze-based digital biomarkers. The transition from lab to mobile platforms allows not only for ecologically valid assessment in everyday environments but also for increased frequency of monitoring, which is essential for tracking subtle changes over time in at-risk populations. This shift supports the integration of eye-tracking into daily routines and long- term dementia prevention strategies, combining accessibility with clinically meaningful insights.

TABLET-BASED MULTIMODAL INTERVENTION Tablet-based training interventions have gained attention as accessible and scalable tools to support cognitive functioning in individuals at risk for dementia. These digital applications offer engaging, adaptive exercises aimed at enhancing memory, attention, and executive functions (Lampit et al., 2014). Studies have shown that app-based cognitive training can lead to modest improvements in cognitive performance and may help delay cognitive

decline (Zuschnegg et al., 2023; Gates et al., 2019). Additionally, the portability and user-friendly nature of tablets make them suitable for older adults, including those with MCI, supporting continued independent living and self-guided training.

The research prototype of the Multimodal App (MMA; Paletta et al., 2021) was further developed into a professional, playful, multimodal training software BRAINMEE designed for cognitive and sensorimotor activation (Pszeida et al., 2023). It includes exercises targeting attention, visuospatial executive function, and motor skills, along with lifestyle-related tasks. The BRAINMEE app offers structured cognitive training with 44 thematic topics, each containing 36–47 exercises across 16 types and four difficulty levels, based on Mini-Mental State Examination scores (MMSE; Folstein et al., 1975) from <18 to 30. In total, over 6,000 exercises are available in five languages for the purpose of personalized cognitive support.

A recommender system was developed based on users’ success rates in completing BRAINMEE exercises, based on the digital results that were obtained from the gameplay. It analyzed training performance to suggest either easier or more challenging difficulty levels, dependent on established low or high performance of the user, respctively. Recommendations were shown on the tablet as ‘system suggestions’ but were not applied automatically. Users retained full control and could decide whether to adjust their training intensity.

DIGITAL BIOMARKERS FROM MOBILE APPLICATIONS Three apps for MCA (Figure 2) apps were applied during the intervention:

  • PAIRS (Paletta et al., 2020a) asks users to regard several emotionally (positive, neutral, negative) weighted image pairs. Then the images are rotated so that they appear face-down, like playing cards, and these cards are moved around to trigger a multi-object tracking task. Finally, it turns into a pairs game to select the image pairs in a minimum number of trials.
  • MIRA (Paletta et al., 2020b) is applied for assessing inhibitory control through a serious game that asks users to operate with eye movements (i) to follow and activate (prosaccades) or (ii) disregard and deactivate (antisaccades) positively and negatively acting avatars, respectively.
  • PVT (Dinges & Powell, 19856): The Psychomotor Vigilance Task (PVT) is a simple reaction time test that measures sustained attention and alertness. Participants respond to visual stimuli that appear at random intervals, with lapses and slower reaction times indicating fatigue or reduced vigilance. It is widely used in sleep and cognitive performance research, estimating fatigue effects via a gamified cognitive test.

These apps were used in the main study alongside corresponding questionnaires to identify digital correlates between test scores and digital events, such as, sensor-based data and game outcome and interaction scores. During the interaction with the MCA apps, eye-tracking with 30 Hz was extracted from a Tablet-embedded camera using the SeeSo software from the

Early Detection of Risk for Cognitive Decline using Mobile Apps 85 South Korean company VisualCamp1. Raw data were periodically uploaded via the internet to the research server of Joanneum Research with a password- secured interface.

Figure 2: Mobile cognitive assessment (MCA) apps used at home with tablet-based embedded eye tracking and app score analytics (a) MIRA, (b) PAIRS, (c) Horizontal gaze coordinate over time (MIRA) in response to stimulus signals (red filled box: correct antisaccade; red empty box: prosaccade, i.e., an erroneous antisaccade). (d) Application of the psychomotor vigilance task (PVT).
Original PDF Figure 2:

EXPERIMENTAL RESULTS Study Description

Firstly, a usability study was conducted; its findings were crucial for adapting interactions with the BRAINMEE as well as with MCA apps, particularly, for handling eye-tracking. The pilot study, lasting four months, focused on implementing the BRAINMEE app intervention and assessing cognitive performance using the MCA apps continuously.

The study was applied in Western Styria in Austria. it was divided into two phases and began with 20 participants (18 female, 2 male); during the course, the number of active participants was reduced to 14. Of the original 20 participants, data from 14 individuals (12 female, 2 male) were included in the final comparison across time points. Participants had a mean age of 77.64 years (SD = 6.61) and a cognitive score of M = 24.14 (SD = 3.14) on the MoCA, indicating mild cognitive impairment.

Psychological assessments were primarily conducted using the MoCA (Nasreddine et al., 2005), MMSE (Folstein et al., 1975), Geriatric Depression

1https://visual.camp/seeso-sdk/ (accessed May 29, 2025)

Scale (GDS; Yesavage et al., 1982), and the Trail Making Test A and B (TMT-A, TMT-B; Reitan et al., 1958) to evaluate cognitive flexibility.

Phase 1 (duration: two months between points in time T1 and T2) involved introducing and familiarizing participants with the technologies, without using the recommender system. Topics in the MCA app were selected either based on weekly suggestions from care assistants of Soziale Dienste Südweststeiermark or freely chosen by participants from a seasonally adapted exercise pool.

Within Phase 2 (duration: two months between points in time T2 and T3), the recommender was activated and in the topic menu, the most suitable exercises for each participant’s cognitive or mental state were highlighted on the Tablet screen.

Results of the Tablet-Based Intervention The dementia screening using the MoCA revealed a statistically significant, overall improvement in cognitive deficits, as determined by a paired-samples t-test (p = .012*, medium effect size with d = 0.68): a significant change was observed between T1 and T3 with an increase in the MoCA total scores (ranging from 0 to 30 points) from M = 24.14 (SD = 3.14) to M = 26.21 (SD = 2.54). This corresponds to a total increase of M = 2.07 (SD = 2.94) points on the MoCA scale over the full 4-month intervention period. The improvement was particularly measurable between the time points T2 and T3, during the use of the recommender technology (p =.0007***; strong effect size with d = 1.09), whereas the first phase (from T1 to T2) without the use of the recommender technology showed no significant improvement (p = .3624).

Furthermore, statistically significant correlations between recommended difficulty level and the difficulty level that were subsequently selected by users (ρ = .407; p =. 004**) as well as the corresponding success rate (ρ = .304; p = .036*) were identified.

Digital Biomarkers From Eye-Tracking and Interaction Correlations. In a first step, correlations between the results of the MoCA total score and eye tracking features (mean value per eye movement feature and per participant) as well as digital game scores were computed. The results demonstrate statistically significant Spearman correlations between MoCA scores and several derived eye movement features, such as, (1) fixation duration (MIRA; ρ =.419; p = .017*), (2) sEBR (PAIRS; ρ = –.554; p = .001**), as well as with (3) mean reaction time (PVT; ρ = –.508; p = .005**), for details see Figure 3.

Machine Learning. A machine learning model using Support Vector Regression (Cortes & Vapnik, 1995) was applied on the study data (n = 14) and resulted in estimating MoCA scores from eye movement features (fixation duration in MIRA) with a mean absolute error of 2.487 points —comparable to inter-rater variability in traditional assessments (Nasreddine et al., 2005; König et al., 2015). The model validation with R-squared (R2 = 0.312) may be accepted for the case of exploratory

Early Detection of Risk for Cognitive Decline using Mobile Apps 87

human behavior analysis with noise in the mobile eye tracking based sensor (Duff, 2012). Factors such as test-retest variability, practice effects, and individual differences contribute to this phenomenon. For instance, repeated assessments may be influenced by variables associated with the test itself, the testing situation, and individual patient factors, leading to increased measurement error and reduced reliability. This variability can result in lower R2 values, indicating that a smaller proportion of variance in the outcome is explained by the model.

Figure 3: Mapping between MoCA total score and eye tracking feature as well as reaction time: (a) Spontaneous eye blink rate (sEBR; mean) encountered during interaction with the PAIRS app. (b) Reaction time (mean in ms) during interaction with the PVT app.
Original PDF Figure 3:

Adherence and Motivation Adherence. The duration of each of the MCA apps (MIRA, PAIRS, PVT) was approximately 3 minutes. Participants were instructed to apply each of the individual apps at least once a week. Within the period of four weeks (phase 1), the apps were actually applied, as follows, MIRA: M = 9.83 (SD = 14.52) times, PAIRS: M = 6.72 (SD = 6.03) times, PVT: M = 3.53 (SD = 4.67) times which responds to a weekly schedule of using MIRA: M = 2.46 (SD= 3.63) times, PAIRS M = 1.68 (SD = 1.51) times, PVT M = 0.88 (SD = 1.17) times. From this data we may deduce that more playful tests (MIRA, PAIRS) cause a higher degree of adherence than not gamified tests (PVT).

Motivation. The aim of the motivation analysis was to quantitatively demonstrate the connection between individual motivation and engagement in the study. In the future, this would make it possible to implement an adaptive recommender based on automatically collected digital data,

which could respond to users’ current motivation and offer training content accordingly.

The risk choice model, developed by Atkinson (1957), is part of expectation-value theories and explains task selection and motivation strength. It considers two personal factors—hope of success and fear of failure—and two task factors—goal attractiveness (incentive) and subjective probability of success. The model assumes that in pure performance tasks, the incentive is tied to task difficulty: pride increases with harder tasks upon success, while shame increases with easier tasks upon failure. The resulting motivation (RM) is calculated as: RM = (Me × We × (1–We)) + (Mm × (1–We) × –We)

where Me and Mm are success and failure motives, and We is the probability of success. The model predicts that success-motivated individuals (Me > Mm) are most motivated by tasks of medium difficulty (We = 0.5), while failure- motivated individuals (Me < Mm) tend to avoid challenge. An analysis based on questionnaires and digital success rates showed that RM correlated positively with the number of exercises completed (ρ = .453), indicating that positive motivation strongly supports training adherence and should be actively fostered.

CONCLUSION AND FUTURE WORK This study demonstrates the feasibility and effectiveness of mobile, gaze- based applications for early detection of cognitive decline in older adults with MCI. Significant correlations between eye-tracking features and MoCA scores, along with machine learning estimation, highlight the potential of digital biomarkers for remote cognitive screening.

The playful nature of the apps supported user adherence and engagement, especially when combined with a recommender system. Future work will focus on scaling the system for broader clinical and home use, integrating adaptive personalization based on real-time motivation, and validating the approach across diverse populations and longitudinal studies. Enhancing algorithmic accuracy and incorporating additional physiological data may further improve diagnostic precision and intervention effectiveness in dementia prevention strategies.

ACKNOWLEDGMENT

This work was supported by the Eurostars-2 joint programme with co- funding from the European Union Horizon 2020 research and innovation programme and the Austrian Research Promotion Agency – FFG under project number E!115629 – MARA. The BRAINMEE app was developed by digital Life GmbH.

Early Detection of Risk for Cognitive Decline using Mobile Apps 89

REFERENCES

Albert, M. S., DeKosky, S. T., Dickson, D., Dubois, B., Feldman, H. H., Fox, N. C.,... & Phelps, C. H. (2011). The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging and Alzheimer’s Association workgroup. Alzheimer’s & Dementia, 7(3), 270–279. Anderson, T. J., & MacAskill, M. R. (2013). Eye movements in patients with

neurodegenerative disorders. Nature Reviews Neurology, 9(2), 74–85. Atkinson, J. W. (1957). Motivational determinants of risk-taking behavior. Psychological Review, 64(6, Pt.1), 359–372. https://doi.org/10.1037/h0043445 Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3). Crawford, T. J., Higham, S., Renvoize, T., Patel, J., Dale, M., Suriya, A., & Tetley, S.

(2005). Inhibitory control of saccadic eye movements and cognitive impairment in Alzheimer’s disease. Biological Psychiatry, 57(9), 1052–1060. Dinges, D. F., & Powell, J. W. (1985). Microcomputer analyses of performance on a portable, simple visual RT task during sustained operations. Behavior Research Methods, Instruments, & Computers, 17(6), 652–655. Duff, K. (2012). Evidence-based indicators of neuropsychological change in the

individual patient: relevant concepts and methods. Arch Clin Neuropsychol. 27(3): 248–61. Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini-mental state”.

A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189–198. https://doi.org/10.1016/0022- 3956(75)90026-6 Gates, N. J. et al. (2019). Computerised cognitive training for preventing dementia in people with mild cognitive impairment. Cochrane Database of Systematic Reviews, (3), CD012279. https://doi.org/10.1002/14651858. CD012279.pub2 Kaufman, L. D. et al. (2012). Executive deficits detected in mild Alzheimer’s disease

using the antisaccade task. Brain Behav. König, A. et al. (2015). Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease. Alzheimers Dement (Amst). 1(1): 112–24. Kourtis, L. C., Regele, O. B., Wright, J. M., & Jones, G. B. (2019). Digital biomarkers

for Alzheimer’s disease: The mobile/wearable devices opportunity. npj Digital Medicine, 2(1), 9. https://doi.org/10.1038/s41746-019-0074-1 Lampit, A., Hallock, H., & Valenzuela, M. (2014). Computerized cognitive training in cognitively healthy older adults: A systematic review and meta-analysis of effect modifiers. PLoS Medicine, 11(11), e1001756. Livingston, G., Huntley, J., Sommerlad, A., Ames, D., Ballard, C., Banerjee, S.,... &

Mukadam, N. (2020). Dementia prevention, intervention, and care: 2020 report of the Lancet Commission. The Lancet, 396(10248), 413–446. Molitor, R. J., Ko, P. C., & Ally, B. A. (2015). Eye movements in Alzheimer’s disease.

Journal of Alzheimer’s Disease, 44(1), 1–12. https://doi.org/10.3233/JAD-141173 Nasreddine, Z. S. et al. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4), 695–699. https://doi.org/10.1111/j.1532–5415.2005.53221.x Paletta, L., Dini, A., Pszeida, M. (2020a). Emotion Measurement from Attention Analysis on Imagery in Virtual Reality. In: Fukuda, S. (eds) Advances in Affective and Pleasurable Design. AHFE 2019. Advances in Intelligent Systems and Computing, vol 952. Springer, Cham. https://doi.org/10.1007/978-3-030-20441-9_2

Paletta, L., Pszeida, M., Dini, A., Russegger, S., Schüssler, S., Jos, A., Schuster, E., Steiner, J., Fellner, M. (2020b). MIRA – A Gaze-based Serious Game for Continuous Estimation of Alzheimer’s Mental State. Proceedings of 12th ACM Symposium on Eye Tracking Research and Applications, ETRA 2020, Stuttgart, Germany, June 2-5, 2020, ACM Press. https://doi.org/10.1145/3379157.3391989 Paletta, L., Pszeida, M., Fellner, M., Russegger, S., Dini, A., Draxler, S., Orgel, T., Jos,

A., Schuster, E., Steiner, J. (2021). Towards Decision Support with Assessment of Neuropsychological Profiles in Alzheimer’s Dementia Using Playful Tablet- Based Multimodal Activation. In: Ayaz, H., Asgher, U., Paletta, L. (eds) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2021. Lecture Notes in Networks and Systems, vol 259. Springer, Cham. https://doi.org/10.1007/ 978-3-030-80285-1_54 Petersen, R. C., Caracciolo, B., Brayne, C., Gauthier, S., Jelic, V., & Fratiglioni, L.

(2014). Mild cognitive impairment: A concept in evolution. Journal of Internal Medicine, 275(3), 214–228. https://doi.org/10.1111/joim.12190 Pszeida, M., et al. (2023). Mild Dementia Decision Support from AI-based Digital Biomarkers using Mobile Playful Exercises with High Adherence. In: Lucas Paletta, Hasan Ayaz and Umer Asgher (eds) Cognitive Computing and Internet of Things. AHFE (2023) International Conference. AHFE Open Access, Vol. 73. AHFE International, USA. http://doi.org/10.54941/ahfe1003970 Reitan, R. M. (1958). Validity of the Trail Making Test as an indicator of organic

brain damage. Perceptual and Motor Skills, 8(3), 271–276. Semmelmann, K., & Weigelt, S. (2018). Online webcam-based eye tracking in cognitive science: A first look. Behavior Research Methods, 50(2), 451–465. https://doi.org/10.3758/s13428-017-0913-7 Wolf, A., Tripanpitak, K., Umeda, S., Otake-Matsuura, M. (2023). Eye-tracking

paradigms for the assessment of mild cognitive impairment: a systematic review. Frontiers in Psychology, Vol. 14, https://doi.org/10.3389/fpsyg.2023.1197567 Xu, Y., Zhang, C., Pan, B. et al. (2024) A portable and efficient dementia screening tool using eye tracking machine learning and virtual reality. npj Digit. Med. 7, 219 (2024). Yesavage, J. A., Brink, T. L., Rose, T. L., Lum, O., Huang, V., Adey, M., & Leirer, V. O.

(1982). Development and validation of a geriatric depression screening scale: A preliminary report. Journal of Psychiatric Research, 17(1), 37–49. https://doi.org/ 10.1016/0022-3956(82)90033-4 Zhou, Y., & Chen, X. (2021). Eye-tracking biomarkers for mild cognitive impairment and Alzheimer’s disease: A systematic review and meta-analysis. Neuropsychology Review, 31(1), 31–55. https://doi.org/10.1007/s11065-020-09456-4 Zuschnegg, K., Schoberer, K., Häussl, A., Herzog, S. A., Russegger, S., Ploder, K.,

Fellner, M., Hofmarcher-Holzhacker, M. M., Roller-Wirnsberger, R., Paletta, L., Koini, M., Schüssler, S. (2023). Effectiveness of computer-based interventions for community-dwelling people with cognitive decline: a systematic review with meta-analyses. BMC Geriatr 23, 229 (2023). https://doi.org/10.1186/s12877- 023-03941-y

Cover of the proceedings volume Cognitive Computing and Internet of Things

Explore the complete digital volume

Cognitive Computing and Internet of Things

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-964867-41-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →