Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Mild Dementia Decision Support from AI-based Digital Biomarkers using Mobile Playful Exercises with High Adherence

1 Joanneum Research Forschungsgesellschaft mbH, 8010 Graz, Austria ↗
2 Medical University of Graz, Division of Neurogeriatrics, 8010 Graz, Austria ↗
3 Medical University of Graz, Institute of Social Medicine and Epidemiology, 8010 Graz, Austria ↗
4 Austrian Red Cross, 8010 Graz, Austria ↗
Publication year
Digital Object Identifier10.54941/ahfe1003970

Abstract

Early detection of cognitive decline and monitoring of cognitive functioning in mild dementia are fundamental for timely adaptation of lifestyle and intervention strategies. The development of digital dementia biomarkers through playful exercises with high adherence rate was a key objective of the national project multimodAAL (no. FFG 875345). The results of a study on computer-based cognitive and physical training (CCPT) in persons diagnosed with mild Alzheimer’s disease (PwAD) are presented.Method: Tablet-PC-based intervention was applied within 6 months in Austria, engaging PwADs living at home by means of playful multimodal training and activation (n=11; female N=8, male N=3; age M=76.6 / SD=9.2 years, MMSE score M=21.50 / SD=4.41). PwADs interacted with a prototypical version of the BRAINMEE app that included a suite of cognitive exercises (puzzle, pairs, text gap filling) based on audiovisual information. The playful training app was introduced and assisted by mobile care professionals with weekly visits, however, PwDs played alone between these visits.Result: PwADs applied training with high adherence, finalizing M=72 (M=32) digital exercises per day within the first (last) month of the study. Duration of using exercise type ‘outsider’(p=.028*) and ‘quiz’ (p=.001**), averaged about 2 week figures, each provided statistically significant correlations (Spearman) with MMSE test scores, as well as ‘spot-the-difference’ (p=.003**) with Trail Making Test A, ‘outsider’ (p=.005**) with Auditory Verbal Learning Test (AVLT), respectively. A neural network (Support Vector Machine, linear kernel, 11-fold cross validation) using duration of use of ‘quiz’, ’outsider’ and ‘hearing’ (guessing animal sounds) as input data resulted in M=2.16 absolute error in MMSE score estimation on test data.Conclusion: The work outlined within the Austrian study on digital biomarker development indicates successful steps towards daily use of cognitive assessment using highly adherent playful training. The playful training app is applied in the European project MARA (no. FFG 886427) to enable continuous estimates of MCI’s mental state over time. The app was very well accepted by both PwADs and persons with MCI. It offers with its pervasive mental assessment tool a large potential for future long-term monitoring in dementia prevention, early detection as well as in numerous dementia care services.

INTRODUCTION

Dementia is a broad category of neurocognitive disorders that cause a long- term decrease in the ability to think and remember that is great enough to affect a person’s daily functioning. Adequate, sufficient and economically fea- sible care is currently one of the greatest technological and social challenges

© 2023. Published by AHFE Open Access. All rights reserved. 12

Mild Dementia Decision Support From AI-Based Digital Biomarkers 13

(Schüssler et al., 2016). Eventually, there is no cure for dementia (Burns & Iliffe, 2009).

A key problem in developing interventions in dementia care is the lack of knowledge about the mental processes and individual dependencies betw- een functional impairments evolving over time. Neuropsychological profiles reflect the impact of the disease on distinctive neuroanatomic networks asso- ciated with complex cognitive domains. Recently, serious games were succes- sfully validated with high potential to represent biomarkers on dementia (Paletta et al., 2018). However, increased estimation accuracy and persona- lized neuropsychological profiling are still required and research in progress (Paletta et al., 2021).

Early detection of cognitive decline and monitoring of cognitive functio- ning in Mild Cognitive Impairment (MCI) as well as in mild dementia are fundamental for timely adaptation of lifestyle and intervention strategies (Livingstone et al., 2015). The earlier any risk of dementia can be dete- cted the better are the chances to delay or even prevent AD that cannot be cured today upon onset of disease: the FINGER study demonstrated that multi-component non-pharmacological interventions can decrease the rate of cognitive decline at the stage of MCI (Ngandu et al., 2015).

Digital cognitive biomarkers (DCB) are supposed to provide early warning by estimating the risk for dementia. In particular, mobile devices with apps that implicitly estimate DCBs can be applied in a nonobtrusive way at home (see Figure 1). These apps are preferably used in a context of empowerment and motivation, such as, with serious games or gamified training (McCal- lum et al., 2013). The development of digital cognitive biomarkers through playful exercises with particularly high adherence rate was a key objective of the Austrian study multimodAAL. 23 PwAD living at home participated in a randomised control test (RCT) and 11 of them completed a 6-month

Figure 1: Application of the tablet-based BRAINMEE1 app of digitAAL Life GmbH in the home environment. With the assistance of a professional caregiver, the app and its exercises for multimodal, i.e., cognitive and sensorimotor, activation are introduced. The app can be used without assistance as well, even by persons with mild dementia (credit: Joanneum Research / M. Schwarzl).
Original PDF Figure 1:

tablet-based intervention including application of neuropsychological test battery before and after the intervention, including screening-type tests, such as, the Mini-mental State Examination (MMSE; Tombaugh et al., 1992), or executive functions-type psychological tests, such as, the Trail Making Test (TMT; Tombaugh & McIntyre, 1992). The PwAD weekly received multimo- dal Computerised Cognitive Training Programs (CCTP) with a trainer and were encouraged to train as often as possible alone or with a caregiver. Digi- tally measured features, such as, the duration of using specific exercise types of the app, each provided statistically significant correlations with test scores of the MMSE as well as with the TMT. Finally, a neural network (Support Vector Machine; Schölkopf et al., 2000) was applied on the digital features and resulted in M = 2.16 absolute error in MMSE score estimation on test data.

The work outlined within the Austrian study on digital biomarker deve- lopment indicates successful steps towards daily use of cognitive assessment using highly adherent playful training.

CONTEXT AND RELATED WORK There is no cure for dementia (Burns & Iliffe, 2009), however, cognitive and behavioural interventions may be appropriate, educating and providing emotional support to the caregiver is important. The individual trajectories of dementia are often suspected to be rather specific, however, longitudinal quantitative studies about cognitive assessment in dementia are rare. Unob- trusive test instruments are highly recommended for this purpose. Physical exercise programs are beneficial by activities of daily living and potentially improve outcomes (Paletta et al., 2018). However, lack of exercise is one of the major risk factors for the dementia development (Norton et al., 2014; Forbes et al., 2015). Therefore, high adherence in the intervention is highly important, even being accompanied by community settings (Graessel et al., 2011). Playful cognitive stimulation for elderly (Lumosity, Cogmed, Cognifit) and with particular consideration of people with dementia (eMotiva, Onto D’mentia, memofit) has been largely exploited. European projects, such as, AALJP M3W, CCE, ROSETTA; GAMEUP CAREBOX and ALFA have investigated means of monitoring and estimating the status of mental pro- cesses, motivating users for cognitive and physical activities in various ways. Other solutions for early diagnosis of dementia and AD in specific include those from start-ups such as Altoida, Braincheck, Mediaire, Neotiv, Eyewise- diagnostics, Presemtecx and ClearskyMD; however, they do not include activation training.

The BRAINMEE app (Figure 2), a commercially available tablet-based training solution for the activation of elderly people with cognitive impair- ments, is based on a research prototype of Paletta et al. (2018; 2021) and presents a playful multimodal training following a multidomain concept of training - including CCTP and physical training, amongst other aspects - that was applied in the FINGER study (Ngandu et al., 2015). However, CCPT achieved only low adherence rates in the FINGER study (Coley et al., 2019).

Mild Dementia Decision Support From AI-Based Digital Biomarkers 15

Figure 2: Concept and exercise implementation of the BRAINMEE app of digitAAL Life GmbH. (a) The multidomain concept of the digital app-based intervention. (b) Sample exercise, i.e., ‘spot-the-difference’ pictures for the training of visuospatial attention and short-term memory. Credits: digitAAL Life GmbH.
Original PDF Figure 2:

The BRAINMEE app includes a diversity of gamified exercise types for men- tal activation, including cognitive but also sensorimotor exercises, and has received very positive user feedback about its usability (Zuschnegg et al., 2021).

MULTIMODAL TRAINING WITH COGNITIVE ASSESSMENT The BRAINMEE app represents a playful exercise-based concept following the framework of multimodal training. It embeds 5 principles of intervention (see Figure 2a), i.e., (i) holistic brain activation, (ii) activities of daily living, (iii) creativity and gaming, (iv) perception, (v) physical activation. There- fore, it includes a diversity of exercise types for activation including stimuli for cognitive but also sensorimotor empowerment. There are exercises for training attention (‘spot the difference’, Figure 2b), visuospatial executive (‘puzzle’), and sensorimotor training, etc. It is especially suited for people who want to live independently at home for as long as possible.

The BRAINMEE app is able to present more than 44 thematic topics, within each of these are 36–54 different exercises per topic, with 16 types of exercises (Table 1), and each with 4 grades of difficulty (level 1 for MMSE score <18; up to the level 4 for MMSE score =30), providing in total more than 6.000 exercises in 11 languages (German, Dutch, Slovenian, Croatian, English, etc.).

AI-BASED DIGITAL COGNITIVE BIOMARKERS Patients received weekly multimodal CCPT with a trainer and were encou- raged to train as often as possible alone or with a caregiver. 11 subjects had completed a 6-month app-based intervention. A neuropsychological test bat- tery was applied by means of pre- and post-study psychological tests. Typical digital exercise types in the app were ‘pairs game’, ‘spot-the-difference’, ‘ari- thmetic’ calculations, or ‘outsider’ in word groups. The patients’ game digital success score and median time required to finalise the exercise were stored to a data warehouse with data logged in real-time. Correlation coefficients

Table 1. Association of BRAINMEE-based exercises with expected related MoCA subscore 1–7 indicated by asterisk (‘*’). Statistically significant correlations betw- een exercise and sub-score are indicated by bracketed asterisks (‘[*]’; from Paletta et al., 2021).

Table available in the original PDF paper.

Exercise type MoCA 1 MoCA 2 MoCA 3 MoCA 4 MoCA 5 MoCA 6 MoCA 7 VisSpEx Naming Attention Language Abstract. Recall Orientat.

KnowledgeText * * * Puzzle * * * * BoxFinder * * * [*] GapFill * * * * Step Sequence * * * Math * * * Pairs game * * Outsider * [*] * Knowl. ImageClip * * * Hearing * * Spot-the-difference [*] * * Movement * * Number Series * * *

(CC) were extracted between patients’ MMSE and TMT scores and required exercise finalisation rates. The CCs were applied for increasing number (1-6) of weeks’ gameplay to identify the appropriate time window for, e.g., MMSE score estimation.

AI-BASED ASSESSMENT A digital analysis system was implemented to develop and evaluate digital cognitive biomarkers (Figure 3). It identifies the performance data from the individual exercises of the training app as an input vector and maps these to the individual test results of the neuropsychological test battery with a corresponding target score. This mathematical mapping requires the use of machine learning methods to adapt in a stochastic process those parameters that structurally assign the input vectors to the appropriate output value in

Figure 3: Software system for the development and evaluation of digital cognitive biomarkers.
Original PDF Figure 3:

Mild Dementia Decision Support From AI-Based Digital Biomarkers 17

each case, which is specified by the psychological score values, for exam- ple score results of instruments for dementia screening, such as, MMSE or TMT. For the machine learning process regression trees (Coppersmith et al., 1999) as well as neural networks (Schölkopf et al., 2000) were integrated into the process pipeline. Regression trees are advantageous because of their inherent property of ‘explainable AI’ (Confalonieri et al., 2021). In principle, regression trees allow a linear mapping from input to output vectors. Neural networks represent a multi-stage processing chain in which linear as well as non-linear functions are applied to the input data, thereby enabling ‘universal function approximations’.

As soon as the machine learning process is completed, the “universal function approximations” adapted in this way can be applied in the manner of “AI-supported” estimation functions to certain arbitrary input data. In the case of digital biomarkers for neuropsychological testing, current scores or representative statistical values for time periods (mean, median) can then be fed to the input and assigned to an estimated value for the psychological test. For example, the average duration (in milliseconds) for completing an exe- rcise (e.g. solving a puzzle) is fed to the input, and corresponding values for an MMSE test can be read out as output. For the processing of progression data and thus the possibility of tracking cognitive performance over longer time ranges, the optimised estimation functions can be applied to time series of performance data from the game-based exercises.

EXPERIMENTAL RESULTS RCT-Based Intervention Study

The study was conducted in the Austrian province of Styria and all participa- ting persons with Alzheimer’s dementia came from the home setting, assisted living or nursing home setting. By the end of the study, 53 persons with Alzh- eimer’s dementia were recruited. A total of 26 of the persons examined could be included in the study, whereby a final sample of 18 persons resulted after study discontinuations, who participated in a baseline study and at least one further follow-up study. The intervention group (IG) of the RCT study inclu- ded N = 11 participants with M = 76.57, SD = 9.15 years of age (8 female, 3 male). The control group (CG) included N = 11 participants with M = 74.89, SD = 6.99 years of age (8 female, 3 male). Preliminary results (Berger et al., 2022) indicated that CCPT over six months has the potential to maintain global cognitive functions in Alzheimer patients. Post-tests revealed a main- tenance of performance in the intervention group (+0.68 points in the MMSE over 6 months, p = 0.300) and a significant decrease in the performance of the control group (−1.34 points in the MMSE over 6 months; p = 0.041).

Ethical approval. All data from the patients’ devices were pseudonymised during collection. This project was conducted under the Ethical Commission of the Medical University of Graz under study protocol 31–556 ex 18/19.

Descriptive results on adherence. Pseudonymised digital data was stored in a database for further analyses. The training adherence curve in Figure 4 represents the number of exercises (e.g., of type ‘quiz’) per week over the entire study duration of 6 months. Although the data vary greatly in the

Figure 4: Curve of adherence to training. The number of exercises (M, SD) per week - type ‘quiz’ is logged for the duration of the study within the IG.
Original PDF Figure 4:

population, a clear tendency can be deduced. Initially, adherence was higher with (assuming M = 5.25 ‘quiz’ exercises per topic2) M = 14.48 (SD = 11.24) topics played in the first week, and decreased over time, with M = 7.43 (SD = 7.24) topics played in the last week.

In the FINGER study (Ngandu et al., 2015), participants were conside- red adherent if they completed at least 66% of the prescribed interventions (Coley et al., 2019) which asked to apply CCPT 3 times each week, 10–15 minutes per session. The results demonstrate that the adherence rate was low- est for cognitive training (24.7%). The FINGER-based prescription would relate to this study to require at least 66% of prescribed 3 BRAINMEE topics each week since each topic needs ≈20 minutes to interact. Under these con- ditions, the BRAINMEE-based interventions achieved an adherence rate of 91.67%, i.e., 3.71 times higher than FINGER. However, the FINGER study went over a duration of 2 years. Coley et al. (2019) reported from the FIN- GER study that poorer executive function was significantly associated with poorer adherence to intervention components. Based on this fact we hypo- thesize, considering the high adherence of the BRAINMEE app, that it may have a good impact on the performance of executive functions.

Recovery of Digital Cognitive Biomarkers Digital cognitive biomarkers (DCBs) are data-processing methods for deri- ving objective, quantifiable characteristics of human cognitive states or processes. In the context of dementia research, we are primarily interested in estimating the scores of neuropsychological screening instruments, such

2There were 4 difficulty levels: in L1 3 exercises ‘quiz’ were included in each topic; L2 included 3, L3 included 6, and L4 included 9 exercises, respectively, resulting in M = 5.25 per topic.

Mild Dementia Decision Support From AI-Based Digital Biomarkers 19

as MMSE and MoCA, from digital data of behaviour in daily life that can be easily determined. The application of DCBs may define decision support in terms of early warning systems that indicate first signs of MCI and thus enable very early interventions, possibly delaying the outbreak or the mortal impact of dementia disease.

Statistical correlation measures were applied to test the relationship betw- een the neuropsychological test results and the digital phenotypes, i.e., digital data, in order to capture the potential for developing DCBs. Figure 5a demon- strates correlations between play duration of exercise types and the MMSE score (as a cognitive screener), for the exercise types outsider (p = 0.028*; mean of 1 week, MMSE baseline measurement. Figure 5b visualises correla- tions with the estimation of other cognitive tests, not only with respect to the

Figure 5: Correlations between average play time of exercise and psychological test score, for (a) MMSE and exercise type ‘quiz’ (p=0.018*; mean of data from 1 week, MMSE baseline measure), (b) TMT-A and exercise type ’spot-the-difference (p=0,003**; mean of data from 2 weeks, TMT-A baseline measure).
Original PDF Figure 5:

MMSE cognitive screening. Correlations are shown between average playing time of different types of exercises and the scores of TMT-A (Trail Making Test, Part A) for the exercise type (p = 0.003**; mean of 2 weeks, MMSE baseline measurement). TMAT-A is particularly informative regarding exe- cutive functions, about visual search speed based on attention, scanning, and processing speed.

Machine Learning of Digital Cognitive Biomarkers Concrete digital biomarkers are represented by statistical estimators, which allow automated inference of a psychological test score based on given input data such as game scores.

In a first attempt a regression tree (Coppersmith et al., 1999) was formed on the mean values of the duration required to terminate an exercise of the following types, “listening” (multiple-choice quiz after listening to an audio file), “outsider” (multiple-choice determination of a term outside the cate- gory of all other words), and “quiz.” Psychological test scores of the MMSE from baseline (pre-study) and post-study measurement were used as supervi- sed data for machine learning, and game scores from the respective 2-week intervals were averaged. After data normalization and application of the tree- learning algorithm as well as 10-fold cross-validation, the mean absolute error in mapping was 3.57 points of the MMSE per test measurement.

In a second development step, a Support Vector Machine neural netw- ork (SVM; Schölkopf et al., 2000; Chang & Lin, 2011) based on linear kernel functions was processed with the input data from 3 game types (iden- tical to those for the regression tree, see above; using 2-week mean values). The performance characteristics of the SVM-based digital biomarker for MMSE estimation are shown in Figure 6. The residuals (median and quarti- les) are minimal specifically in the transition region between Mild Cognitive Impairment (MCI) and dementia diagnosis.

Figure 6: Performance characteristics of the digital biomarker for MMSE estimation based on a support vector machine (SVM) network that processes inputs from 3 types of games. The residuals (median and quartiles) are minimal especially in the transition region between Mild Cognitive Impairment and diagnosed mild dementia.
Original PDF Figure 6:

Mild Dementia Decision Support From AI-Based Digital Biomarkers 21

The main result of the AI-based biomarker is the mean absolute error in the precision of the estimation of a respective MMSE score - with 10-fold cross validation and based on three different game scores - of 2.16 points of the MMSE score on average.

CONCLUSION

The work indicates successful steps towards digital cognitive biomarkers in playful training apps for non-pharmacological prevention as well as interven- tion in dementia care. BRAINMEE would enable in this way continuous estimates of PwAD’s individual cognitive mental state and its course over time. Its high adherence rates promise successful monitoring over long periods of time.

ACKNOWLEDGMENT

The research leading to these results has received funding from the Austrian Federal Ministry for Climate Action, Environment, Energy, Mobility, Inno- vation and Technology (BMK) by project multimodAAL (FFG project No. 868209) of the benefit ICT of the Future Programme organized by the Austrian Research Promotion Agency (FFG).

REFERENCES

Arnett, J. A. & Labovitz, S. S. (1995). Effect of physical layout in performance of the Trail Making Test. Psychological Assessment. doi: 10.1037/1040-3590.7.2.220. Berger, M. et al. (2022), Preliminary results of computer-based multimodal training

in Alzheimer’s disease over six months – the multi modAAL study. Alzheimer’s Dement., 18: e068759 https://doi.org/10.1002/alz.068759. Burns, A., and Iliffe, S (2009). Dementia, BMJ (Clinical research ed.)338. Chang, C. C. and Lin, C. J. (2011) LIBSVM: A Library for Support Vector Machines. ACM Transactions on Intelligent Systems and Technology, 2, 27. http://dx.doi.org/10.1145/1961189.1961199. Coley, N. et al. (2019). Adherence to multi domain interventions for dementia pre-

vention: Data from the FINGER and MAPT trials. Alzheimer’s & dementia: The journal of the Alzheimer’s Association, 15(6), 729–741. https://doi.org/10.1016/ j.jalz.2019.03.005 Confalonieri, R. et al. (2021). A historical perspective of explainable Artificial Intelligence. WIREs Data Mining Knowl Discov. https://doi.org/10.1002/widm.1391. Coppersmith, D., Hong, S. J. & Hosking, J. R. Partitioning Nominal Attributes in

Decision Trees. Data Mining and Knowledge Discovery 3, 197–217 (1999). https: //doi.org/10.1023/A:1009869804967. Forbes, D., Forbes, S. C., Blake, C. M., Thiessen, E. J., & Forbes, S. (2015). Exercise programs for people with dementia. The Cochrane database of systematic reviews, https://doi.org/10.1002/14651858. CD006489.pub4. Graessel, E., Stemmer, R., Eichenseer, B., Pickel, S., Donath, C., Kornhuber, J., &

Luttenberger, K. (2011). Non-pharmacological, multi component group therapy in patients with degenerative dementia: a 12-month randomized, controlled trial. BMC medicine, 9, 129. https://doi.org/10.1186/1741-7015-9-129. Livingstone, G. et al. (2021). Dementia prevention, intervention, and care. Lancet 2017; 390: 2673–734. http://dx.doi.org/10.1016/S0140-6736(17)31363-6.

McCallum, S. & Boletsis, C. (2013). Dementia Games: A Literature Review of Dementia-Related Serious Games. In: Ma, M. et al. (eds) Serious Games Deve- lopment and Applications. LNCS vol. 8101 https://doi.org/10.1007/978-3-642- 40790-1_2 Ngandu, T. et al. (2015). A 2 year multi domain intervention of diet, exercise, cogni-

tive training, and vascular risk monitoring versus control to prevent cognitive decline in at-risk elderly people (FINGER): a randomised controlled trial. Lancet (London, England), 385(9984), 2255–2263. https://doi.org/10.1016/S0140- 6736(15)60461-5 Norton, S. et al. (2014). Potential for primary prevention of Alzheimer’s disease: an analysis of population-based data. The Lancet. Neurology, 13(8), 788–794. https://doi.org/10.1016/S1474-4422(14)70136-X. Paletta, L. et al. (2018). Playful Multimodal Training for Persons with Dementia with

Executive Function based Diagnostic Tool, Proc. Pervasive Technologies Related to Assistive Environments, Corfu, Greece, June 26-29, 2018, ACM Press. Paletta, L. et al. (2021). Towards Decision Support with Assessment of Neuropsychological Profiles in Alzheimer’s Dementia Using Playful Tablet-Based Multi modal Activation. In: Ayaz, H. et al. (eds) Adv in Neuroergonomics and Cognitive Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-80285-1_54. Schölkopf, B., Smola, A. J., Williamson, R. C., & Bartlett, P. L. (2000). New

support vector algorithms. Neural computation, 12(5), 1207–1245. https://doi.org/10.1162/089976600300015565 Schüssler S, Dassen T, Lohrmann C. (2016) Care dependency and nursing care problems in nursing home residents with and with our dementia: across-sectional study. Aging Clinical and experimental research 28(5): 973-82. Tombaugh, TN; McIntyre, NJ (1992). "The mini-mental Status Examination: A com-

prehensive Review". Journal of the American Geriatrics Society. 40(9): 922–935. doi:10.1111/j.1532–5415.1992.tb01992.x. Zuschnegg, J. et al. (2022). Usability Evaluation after a 6-month Tablet-Based Dementia Training Program by People with Alzheimer’s Disease, Relatives, and Dementia Trainers, In: Paletta, L., Ayaz H., eds., Cognitive Computing and Inter- net of Things, AHFE international open access, DOI: https://doi.org/10.54941/ah fe1001844.

Cover of the proceedings volume Cognitive Computing and Internet of Things

Explore the complete digital volume

Cognitive Computing and Internet of Things

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-49-0

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →