Skip to article
AHFE AHFE Discoveries AHFE International Open Access Journal
AHFE OPEN ACCESS OPEN ACCESS CONFERENCE PROCEEDINGS PAPER

Telerehabilitation Platforms in times of COVID-19: Usability Evaluation

1 Departamento de Informática y Ciencias de la Computación, Escuela Politécnica Nacional, Quito 170525, Ecuador ↗
2 Universidad de Las Americas ↗
3 Intelligent and Interactive Systems Laboratory, Universidad de Las Américas, Quito Ecuador; Facultad de Tecnologías de Información,Universidad Latina de Costa Rica, San José Costa Rica; Facultad de Ingeniería y Ciencias Aplicadas, Universidad de Las Américas, Quito Ecuador ↗
4 Facultad de Psicología, Pontificia Universidad Católica del Ecuador, Quito, Ecuador ↗
5 Centro de Investigación en Ciencias Humanas y de la Educación - CICHE Universidad Tecnológica Indoamérica, Quito, Ecuador ↗
6 One Health Research Group, Universidad de las Américas, Quito, Ecuador ↗
Publication year
Digital Object Identifier10.54941/ahfe1004050

Abstract

Following the pandemic caused by the SARS-CoV-2 virus, telerehabilitation (TR) has become a tool for safely providing health services. TR has positioned itself as an innovative health strategy that seeks to improve access to healthcare by applying advances in information and communication technologies (ICT). The rise of TR, an emerging field of telehealth, brings new challenges, mainly related to supporting physiatrists in planning, monitoring, and evaluating rehabilitation.Currently, it is common for patients who have shown improved health following hospitalization caused by the SARS-CoV-2 virus to be sent home. However, many of these patients may experience consequences of prolonged hospitalization and isolation, such as physical symptoms (fatigue, weakness), cognitive difficulties (delirium), and emotional complications (depression).The research conducted by some authors has demonstrated the effectiveness of TR for physical and mental health problems, improving patients' quality of life by overcoming obstacles such as distance to the institution providing care, strain associated with face-to-face rehabilitation appointments on family members, and lack of homecare for rehabilitation.Using software within TR programs supports professionals in constructing a rehabilitation program and serves as a source of learning and knowledge. When designing and developing technologies to improve TR, it is important to consider usability, a key aspect that facilitates the use of the product and relates to the ability to learn, efficiency, error reduction, and satisfaction. The correct incorporation of usability must consider principles, metrics, and patterns that facilitate effectiveness, quality, and service usefulness. Lack of usability and a user-centered approach can lead to problems such as confusion, repetitive errors, or even abandonment of the TR program.In this paper, our proposal is to evaluate three TR platforms using the concept of "usability" from the field of Human-Computer Interaction. To carry out this evaluation, we will follow the process proposed by Rautela A. and utilize the CSUQ (Computer System Usability Questionnaire), which is an inquiry method used by IBM to compare the results with the SUS (Systems Usability Scale). The SUS will allow us to determine the acceptability of the TR platforms based on the final users' ratings. Platforms that score above 70 are considered "acceptable," while those that score between 50 and 70 are categorized as "marginal," and those below 50 are labeled "unacceptable." Our evaluation will focus on analyzing the usability of these platforms and identifying areas for improvement to enhance their overall user experience.The usability evaluation results will help us consider web platform design and development features for the TR of physical and cognitive disorders. Consequently, we encourage health professionals to adapt their job to TR, improving patients' autonomy and quality of life.The results of the usability evaluation will assist in the consideration of web platform design and development features for TR of physical and cognitive disorders. As a result, we encourage health professionals to adapt their practices to TR, ultimately enhancing patients' autonomy and quality of life.

© 2023. Published by AHFE Open Access. All rights reserved. 438

Telerehabilitation Platforms in Times of COVID-19: Usability Evaluation 439

INTRODUCTION

Following the pandemic caused by the SARS-COV-2 virus, TR has become a tool to provide health services safely. TR has positioned itself as an innovative health strategy that seeks to improve access to health by applying advances in ICT (Brennan, Mawson and Brownsell, 2009). The rise of TR, an emerging field of telehealth, brings new challenges, mainly related to supporting the physiatrist in planning, monitoring, and evaluating rehabilitation. Currently, it is common for patients who have demonstrated health-improved following hospitalization caused by the SARS-COV-2 virus to be sent home. However, many of these patients may have consequences of prolonged hospitalization and isolation, such as physical (fatigue, weakness), cognitive (delirium), and emotional complications (depression) (Stam, Stucki and Bickenbach, 2020).

Research conducted in (Tousignant et al., 2006) and (Bettger et al., 2020) has demonstrated the effectiveness of TR for physical and mental pro- blems improving patient’s quality of life by overcoming obstacles such as the distance to the institution providing care, the strain associated with these face-to-face rehabilitation appointments on family members, and the lack of homecare for rehabilitation.

Using software within TR programs supports professionals aiding the construction of a rehabilitation program and as a source of learning and knowledge (Cardoso, 2021). For the design and development of technolo- gies that improve TR is important to consider usability, a key aspect that facilitates the use of the product relating to the ability to learn, efficiency, error reduction, and satisfaction. The correct incorporation of usability must consider principles, metrics, and patterns that facilitate effectiveness, quality, and service usefulness. Lack of usability and a user-centered approach can lead to problems such as confusion, repetitive errors, or even abandonment of the TR program (Henry Pilco et al., 2019).

This work aims to analyze the usability evaluations of TR web platforms, considering efficiency, effectiveness, and satisfaction with the CSUQ. This questionnaire will allow us to get data regarding the platform’s usability, the quality of the information presented, and the quality of the interfaces.

The usability evaluation results will help us consider web platform design and development features for the TR of physical and cognitive disorders. Consequently, we encourage health professionals to adapt their job to TR, improving patients’ autonomy and quality of life.

This paper is structured as follows: section 2 presents the related work in the context of TR and usability. Section 3 describes the case study applied in this work. Section 4 proposes the usability evaluation along with the expe- rimental process developed. Finally, section 5 and 6 present the results and conclusions, respectively.

MATERIAL AND METHODS TR, an emerging field of telehealth, refers to providing synchronous or asy- nchronous rehabilitation treatment via ICT, minimizing barriers caused by distance, time, and costs (Brennan et al., 2010).

Several authors have conducted usability evaluations of TR programs to treat sequelae caused by the SARS-COV-2 virus; in these investigations were

used communication tools such as Zoom™, Viber™, or Google Meet™for TR sessions. Research presented in (Finkelstein et al., 2020) and (Lee et al., 2023) affirms the high potential for patient-centered digital health and a high acceptance of the rehabilitation program. However, in (Valera et al., 2022), the potential of TR is overshadowed by existing technical problems, mostly due to internet connection in a developing country.

Previous studies exists and contribute to the enhancing and enrichment of the telerehabilitation research. A systematic literature review was conducted in (Valverde-Martínez et al., 2023) demonstrating that the use of TR could be an effective tool for the treatment of persistent symptoms after suffering from COVID-19. No records have been found of research conducted for the treatment of the sequelae of the SARS-COV-2 virus using TR platforms.

On the other hand, the authors of (Solorzano et al., 2021) and (Pérez- Medina et al., 2020) performed usability evaluations on a TR platform for patients recovering after hip replacement surgery. The results obtained in those investigations have provided the researcher’s important suggesti- ons to improve the platform’s current features. In addition, (Hennry Pilco et al., 2019) demonstrates the necessity of performing usability evaluations throughout the software development cycle, resulting in improvements in the system’s usability, even before writing the first code lines.

The sample selected for a usability evaluation is a widely discussed topic because if conducted with many users, the costs tend to outweigh the benefits, while too small a sample may miss usability issues. The study presented in (Faulkner, 2003) shows that up to 80% of usability problems can be found using a sample of five participants. Therefore, the findings of this research imply that more participants are unnecessary.

Case Study Telerehabilitation platforms allow the rehabilitation process to be carried out in different ways. TR sessions can be carried out using any of the three modes of communication: Synchronous: delivered to the patient through video calls or virtual meetings. Asynchronous: by sending text messages, emails, audio, or videos with multimedia material. Hybrid: by incorporating the two modes.

For the development of this research were selected three web platforms for TR. The platforms were evaluated considering the following criteria: One or more modes of communication to carry out the TR sessions. Potential for physical or cognitive TR, the platform should include a set of exercises for physical or cognitive rehabilitation from the sequelae of the SARS-COV-2.

Preferably with free access or a free trial period to access the resource for a free evaluation.

The selected platforms were as follows:

  • Rehab Guru (https://www.rehabguru.com/)
  • Rehab My Patient (https://www.rehabmypatient.com)
  • FreePT (https://securelogin.freept.com/FreePt/login.do) The work process is as follows: The physiatrist personality evaluates a patient’s initial condition. After analyzing the evaluation results, a rehabilitation program is designed. The

Telerehabilitation Platforms in Times of COVID-19: Usability Evaluation 441

patient is instructed to complete the rehabilitation program using a web- based platform at home. Once the patient has conducted all the TR sessions will be able to inform the physiatrist of any program-related difficulties. In the event of finding ailments or difficulties, the physiatrist will advise the pati- ent again, accompanying the patient throughout the rehabilitation process. If no rehabilitation-related issues are identified, the physiatrist will conduct periodic examinations to monitor the patient’s improvement.

The selected platforms allowed the study to be carried out from two perspectives: Physiatrist Perspective

The web-based platform allows physiatrists to create an account to manage their patient’s TR sessions and therapy plans. Figure 1 depicts the three plat- forms that permit patient registration: (a) Rehab Guru (Rehab Guru, 2022), (b) Rehab My Patient (RehabMyPatient, 2023), and (c) FreePT (FreePT, 2023). In addition, Figure 2 evidence the creation of a template for rehabili- tating a specific ailment, in this case, to treat the sequelae of the SARS-COV-2 virus, as outlined for Rehab Guru (a) and Rehab My Patient (b). The FreePT platform does not include these workouts; the exercise programs are designed and sent by email, as seen in (c).

Finally, communication with the patient can be seen in Figure 3, where an appointment for rehabilitation is scheduled; at the appropriate time, the physiatrist will advise the patient on how to execute and supervise the com- pletion of the exercises through videoconference. Rehab Guru (a) and Rehab My Patient (b) platforms simplify the task of creating TR sessions as they automatically generate a link for a video call, even though this functionality is platform specific for Rehab My Patient. In contrast, the FreePT (c) platform makes the task more complicated by requiring the physiatrist to schedule a meeting for the rehabilitation session and email it to the patient.

Figure 1: Comparison of patient registration (own figure based on (a), (b), (c)).
Original PDF Figure 1:
Figure 2: Comparison of the creation of an exercise template (own figure based on (a), (b), (c)).
Original PDF Figure 2:
Figure 3: Comparison of a TR session (own figure based on (a), (b), (c)).
Original PDF Figure 3:

Patient Perspective Patients can review the potential rehabilitation programs proposed by their physiatrists. These programs will be communicated using one or more communication modes, simplifying the patient’s tasks. Figure 4 shows the email notifications sent to patients upon completion of therapies; the Rehab Guru (a), Rehab My Patient (b), and FreePT (c) platforms also provide the attachment of photographs and videos of the rehabilitation program.

Telerehabilitation Platforms in Times of COVID-19: Usability Evaluation 443

Figure 4: Comparison of TR sessions notifications.
Original PDF Figure 4:

EVALUATION RESULTS AND DISCUSSION This section summarizes the process corresponding to the usability evaluation performed on the TR web platforms. The usability evaluation was carried out considering the protocol proposed by Abhay Rautela (Rautela, 2018) (see Figure 5). This protocol was accompanied by a Lewis inquiry method, such as the CSUQ.

Participants The process was carried out with a group of five users since, as mentioned previously, Laura Faulkner demonstrated in (Faulkner, 2003) that, despite evaluating with just five users, the results of usability problems encountered represent between 55% and 85% of the total.

Materials and Methods

The following materials were considered for this usability evaluation:

  • Web-based telerehabilitation platforms
  • Usability Evaluation Protocol
  • CSUQ The CSUQ consists of sixteen questions using the empirically validated 7 point Likert scale, distributed in 4 categories: System Utility (CSUQ1 to CSUQ6). Information Quality (CSUQ07 to CSUQ12). Interface Quality (CSUQ13 to CSUQ15). General (CSUQ16).

Tasks and Procedure The user tasks (T) completed by a physiatrist and a TR patient are shown in Tables 1 and 2, respectively.

Figure 5: Usability Evaluation Protocol.
Original PDF Figure 5:
Table 1. Activities performed in the role of a physiatrist.

Table available in the original PDF paper.

Tasks and subtasks T1. Login to the Platform 1. Enter your e-mail address. 2. Enter the password. 3. Click the “Log In” button. T2. Register a Patient 1. Click the “Clients” or “Add New Patient” button. 2. Enter patient data. 3. Click the “Add New Patient” button. T3. Create a telerehabilitation session

1. Click the “Clients” or “Add New Patient” button. 2. Enter patient data. 3. Click the “Add New Patient” button. T4. Logout from the platform

1. Click the “Log out” button in the upper right corner of the platform

Table 2. Activities performed in the role of patient.

Table available in the original PDF paper.

Tasks and subtasks T1. Entering a telerehabilitation session

1. Login to your email. 2. Search for the e-mail with your appointment schedule. 3. Open the e-mail with your appointment schedule. 4. Click on the URL link with the appointment meeting or rehabilitation program attached.

The participants performed the proposed tasks and then completed the CSUQ, with data collected to determine the outcomes.

Telerehabilitation Platforms in Times of COVID-19: Usability Evaluation 445

DISCUSSION

For the analysis of the results were reviewed the values obtained in the usa- bility surveys were, these values ranged from 1(Strongly agree) to 7(Strongly disagree) and included the option “Not Applicable” (N/A); the lower values were interpreted as indicators of a higher degree of satisfaction. The auth- ors have decided to hide the names of the platforms to avoid affecting the platforms analyzed. In addition, some correspondence was made between the CSUQ and SUS values to transform the values between 1 to 7 of the CSUQ scale to a 0 to 100 point scale that matches SUS for interpretation. The equation for this transformation was presented in 1:

!

!

P16

!

100

Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.
Mathematical equation reproduced from the original PDF
Equation reproduced from the original PDF.

16

6

Breaking this down, the process of getting from a CSUQ score to one that matches the SUS involves subtracting 1 from the mean of the 16 individual CSUQ items and multiplying that by 100/6 to stretch it out to a 0 to 100 point scale, then subtracting from 100 to reverse the scale (Lewis, 2018).

The results obtained after this transformation are presented in Figure 6, where the equivalent score on the SUS can be seen for each question in the questionnaire and each web platform analyzed. This graph shows a clear superiority in the usability of the Platform 1, followed by Platform 2 and Platform 3.

The transformation was performed to employ the Bangor scale, presented in Figure 6, which allows us to compare adjective ratings and acceptability scores about the SUS score obtained. As we can see in the Bangor scale (Ban- gor, Kortum and Miller, 2009), the value “Acceptable” is assigned when the evaluation score is above 70, “Not Acceptable” when the value is below 50, and “Marginal” when it is between 50 and 70.

Concerning the categories of usability evaluation, Figure 7 shows the percentages obtained by each evaluated platform. As can be seen, Platform 1 once again stands out above the other platforms in areas such as the system’s usefulness and the interface’s quality; however, in aspects such as the quality of the information, it shares a high score with Platform 2. See Figure 8.

Generally, the rating of the two web platforms for TR (Platform 1 and Platform 2) following the equivalence presented in the Bangor scale is “Acce- ptable”, as they present 89.69% and 73.75%, respectively. However, the

Figure 6: Bangor SUS scale.
Original PDF Figure 6:
Figure 7: Percentage by category.
Original PDF Figure 7:
Figure 8: Percentage by category.
Original PDF Figure 8:

Platform 3 shows clear deficiencies in all the aspects evaluated since it pre- sents an average of 30.52% in the usability evaluation, equivalent to “Not Acceptable” (see Figure 7).

Telerehabilitation Platforms in Times of COVID-19: Usability Evaluation 447

CONCLUSION

The present work evaluated the usability of 3 web platforms for TR following the protocol proposed by Abhay Rautela and the incorporation of the CSUQ; this process allowed us to identify design elements for the development of future TR platforms.

The benefits that TR platforms offer to the community allow adults, youth, and children to access physical and mental rehabilitation from the comfort of their homes. However, it is essential to analyze the patient’s needs for tre- atment, hence the importance of conducting studies that provide feedback tools that meet the satisfaction conditions of end users, indicating to which treatment they have been oriented.

The results obtained provide us with important findings on the usability to be incorporated into a web platform for TR, with the aim that experts in the field of health adapt their work to a rehabilitation modality, allowing to improve the autonomy and quality of life of their patients, minimizing the barriers caused by distance, time, and costs.

ACKNOWLEDGMENT

The authors would like to thank the Corporación Ecuatoriana para el Desar- rollo de la Investigación y Academia - CEDIA for the support of this research work, through the I+D+I-XVII-2022-27 program, especially for the funding of the project: “Plataforma digital para educación terapéutica accesible hacia las necesidades de rehabilitación respiratoria”. The authors want to remark on the contribution of the “Laboratorio de Investigación de Sistemas de Infor- mación e Inclusión Digital” (LudoLab - https://ludolab.epn.edu.ec/) for the support for the research and publication of this article.

REFERENCES

Bangor, A., Kortum, P. and Miller, J. (2009) ‘Determining what individual SUS scores mean; adding an adjective rating’, Journal of usability studies, 4(3), pp. 114–23. Bettger, J. P. et al. (2020) ‘Effects of Virtual Exercise Rehabilitation In-Home’, J Bone

Joint Surg Am, 0, pp. 101–109. Brennan, D. et al. (2010) ‘A Blueprint for Telerehabilitation Guidelines’, International Journal of Telerehabilitation, (October), pp. 31–34. doi: 10.5195/ijt.2010.6063. Brennan, D. M., Mawson, S. and Brownsell, S. (2009) ‘Telerehabilitation: Enabling

the remote delivery of healthcare, rehabilitation, and self management’, Studies in Health Technology and Informatics, 145(January 2014), pp. 231–248. doi: 10.3233/978-1-60750-018-6-231. Cardoso, S. (2021) Barreras y palancas para el uso de la telerehabilitación a través de la experimentación en tres países. Humanity &. Luxemburgo: Creative Common Attribution. Faulkner, L. (2003) ‘Beyond the five-user assumption: Benefits of increased sample

sizes in usability testing’, Behavior Research Methods, Instruments, & Computers, 35(3), pp. 379–383. doi: 10.1109/DCABES.2011.32. Finkelstein, J. et al. (2020) ‘Usability of remote assessment of exercise capacity for pulmonary telerehabilitation program’, Studies in Health Technology and Informatics, 275(November), pp. 72–76. doi: 10.3233/SHTI200697.

FreePT (2023) FreePT - Secure Log in. Available at: https://securelogin.freept.com/F reePt/login.do (Accessed: 28 March 2023). Lee, J. H. et al. (2023) ‘Comparison of Functional Outcomes between Supervised

Rehabilitation and Telerehabilitation in Female Patients with Patellofemoral Pain Syndrome during The COVID-19 Pandemic’, International Journal of Environ- mental Research and Public Health, 20(3), p. 2233. doi: 10.3390/ijerph20032233. Lewis, J. R. (2018) ‘Measuring Perceived Usability: The CSUQ, SUS, and UMUX’, International Journal of Human-Computer Interaction, 34(12), pp. 1148–1156. doi: 10.1080/10447318.2017.1418805. Pérez-Medina, J. L. et al. (2020) ‘User experience assessment of a tele-rehabilitation

platform: The physiotherapist perspective’, Advances in Intelligent Systems and Computing, 959, pp. 463–473. doi: 10.1007/978-3-030-20040-4_42. Pilco, Hennry et al. (2019) ‘An agile approach to improve the usability of a physical telerehabilitation platform’, Applied Sciences (Switzerland), 9(3), pp. 1–20. doi: 10.3390/app9030480. Pilco, Henry et al. (2019) ‘Analysis and Improvement of the Usability of a Tele-

Rehabilitation Platform for Hip Surgery Patients’, 781(July 2018), pp. 3–5. doi: 10.1007/978-3-319-94334-3. Rautela, A. (2018) The Usability Testing Process (diagram) — Cone Trees, Cone- Trees. Available at: https://medium.com/conetrees/the-usability-testing-process-d iagram-cone-trees-c77941edfeeb (Accessed: 10 February 2022). Rehab Guru (2022) Exercise Prescription Software for professionals, physiothe-

rapy and patients. Available at: https://www.rehabguru.com/ (Accessed: 28 March 2023). RehabMyPatient (2023) Exercise prescription software for therapist | Rehab My Patient. Available at: https://www.rehabmypatient.com/ (Accessed: 28 March 2023). Solorzano, S. et al. (2021) ‘Tele-Rehabilitation platform ePHoRt as Serious Game:

Usability evaluation’, Proceedings - 2021 2nd International Conference on Infor- mation Systems and Software Technologies, ICI2ST 2021, pp. 54–61. doi: 10.1109/ICI2ST51859.2021.00016. Stam, H. J., Stucki, G. and Bickenbach, J. (2020) ‘Covid-19 and post intensive care syndrome: A call for action’, Journal of Rehabilitation Medicine, 52(4), pp. 19–22. doi: 10.2340/16501977-2677. Tousignant, M. et al. (2006) ‘In home telerehabilitation for older adults after disch-

arge from an acute hospital or rehabilitation unit: A proof-of-concept study and costs estimation’, Disability and Rehabilitation: Assistive Technology, 1(4), pp. 209–216. doi: 10.1080/17483100600776965. Valera, M. J. S. et al. (2022) ‘Evaluating the Usability of the Expanded Telerehabilitation Program Implemented by Philippine General Hospital during the COVID-19 Pandemic: A Cross-sectional Study’, Acta Medica Philippina, 56(4), pp. 41–50. doi: 10.47895/AMP. V56I4.4042. Valverde-Martínez, M. Á. et al. (2023) ‘Telerehabilitation, A Viable Option in Pati-

ents with Persistent Post-COVID Syndrome: A Systematic Review’, Healthcare, 11(2), p. 187. doi: 10.3390/healthcare11020187.

Cover of the proceedings volume Human Interaction & Emerging Technologies (IHIET 2023): Artificial Intelligence & Future Applications

Explore the complete digital volume

Human Interaction & Emerging Technologies (IHIET 2023): Artificial Intelligence & Future Applications

Read this individual research paper free through the original publisher, or explore the full digital proceedings book.

ISBN 978-1-958651-87-2

Explore this ISBN in the AHFE Bookstore Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional

Explore Related Research

Related research 8 matches

More papers from this volume

← Back to Research ExplorerBrowse the full library →