Julia Wilk, Piotr Falkowski, Cezary Rzymkowski (2023). Use of predictive models based on biomedical signals and motion measurements for predicting extremity kinematics. In: Emerging Technologies in Healthcare and Medicine. AHFE Open Access, vol. 116, pp. 98–106. https://doi.org/10.54941/ahfe1004361
Due to staff shortages among physiotherapists and an ageing society, there is a growing need for the dynamic development of robot-aided rehabilitation. An ideal solution is a therapy conducted remotely, requiring minimal supervision by a physiotherapist, thus saving time and increasing the number of people treated. To achieve this, a rehabilitation device equipped with intelligent systems to detect dangerous situations for patients is essential. The paper presents a methodology for constructing a predictive model for a control system dedicated to home kinesiotherapy with an exoskeleton. It involves NARX-type recurrent neural networks based on the patients' electromyographic (EMG) measurements while exercising. Within the study, simultaneous EMG measurements and motion capture of the upper extremity were performed on three participants. The collected data were divided into sets for learning and testing neural networks. The kinematics was calculated using a multibody model of the upper limb with five degrees of freedom. The position data obtained from markers were converted into joint angles. Subsequently, a neural network was modelled in MATLAB, with the EMG measurements as inputs and the rotation angles in the upper limb joints as outputs. A sequence of movements covering the entire workspace of the upper limb was adopted as the network training set, while the network's performance was evaluated based on trajectory data from five simple exercises. The reported accuracy of the results remained within the range of 0.05-1°. The study revealed differences in the quality of the result depending on whether the participant of the exercise changes between the training and validation. To optimize predictions and reduce computation time, several different networks with varying parameters were constructed, trained and compared. The quasi-optimal parameters of the models were identified, including the number of hidden neurons, samples of previous output signal values, and samples of prior input signal values.
Use of Predictive Models Based on Biomedical Signals and Motion Measurements 99
INTRODUCTION
Most of the currently available rehabilitation devices are stationary and hence, permanently set up in rehabilitation clinics. Their use requires the constant presence of a physiotherapist onsite. They can relieve the exercise burden on the rehabilitation therapist, for whom a therapy session is often a significant physical effort (Falkowski, 2022b), and turn the therapy more entertaining (Levac, 2012).
Due to the limited number of medical staff and the ageing of society (Atif, 2021), there are not enough physiotherapists to guarantee regular home visits to patients who require these. Thus, rehabilitation robots can poten- tially solve the problem of missing professionals and improve the quality of therapy. However, they are not the answer to most of the challenges of self- use, which emerged during the COVID-19 pandemic. The interruption or reduction in the intensity of therapy negatively impacts the patient’s health (Gutenbrunner, 2020). As the world seeks to automate and maximize effi- ciency and convenience, it is desirable to create solutions for rehabilitation that are remote and accessible anytime from anywhere in the world (Musleh, 2022). Nevertheless, technology that interacts directly with humans, such as rehabilitation robotics, must meet strict safety criteria before being imple- mented without supervision (Munoz, 2019). Rehabilitation robots can fulfil this requirement by equipping them with safety systems that adapt to the individual patient’s needs, in line with the trend of personalized medicine (Maughan, 2017). In kinesiotherapy of the extremity, the patient has to be prevented from entering dangerous configurations out of their range of motion. This can be realized by predicting the extremity’s kinematics in advance.
The idea presented in this article is to use recurrent neural networks (RNNs) (Pascanu, 2013), (Falkowski, 2022) based on electromyographic (EMG) recordings (Tabbori, 2020) to predict the positions of characteristic points of the upper extremity while exercising. Position prediction will occur before a hazardous situation occurs to react in advance and prevent them. This control system is ultimately to be used in the ExoReha exoskeleton, which is being developed at the Łukasiewicz Research Network – Indus- trial Research Institute for Automation and Measurements PIAP (Falkowski, 2023), (Falkowski, 2022a), (Falkowski, 2020). To prove the presented concept, an experiment without the use of any rehabilitation robot was con- ducted. This includes real-life data collection and comparison of the different architectures of RNNs. The study aimed to validate whether it is possible to design a model predicting the system’s behavior at least one second in advance with satisfactory accuracy.
METHODOLOGY
The study validates the applicability of the prediction model correlating real- life EMG and joint angles of the upper extremity. For this purpose, EMG and the upper extremity characteristic point motion were tracked simultaneously while performing specific movements.
The EMG data were collected from six muscle parts: biceps (lat. musculus biceps brachii), triceps (lat. musculus triceps brachii), pectoralis major (lat. musculus pectoralis major), quadriceps (lat. musculus trapezius), and shoul- der muscle (lat. musculus deltoideus), distinguishing between the middle and posterior parts (Leis, 2000). To do so, six MyoWare Muscle Sensors were used for data collection. They recorded raw signals but also pre-filtered them. The data were collected at a sampling rate of 3000 times per second and then filtered was post-processed with a 50 Hz clipping filter, a fourth-order high- pass Butterworth filter with a 20 Hz cut-off frequency, and a fourth-order low-pass Butterworth filter with a 500 Hz cut-off frequency (Yin, 2020). The collected data was then assigned to a float variable type and normalized by dividing each measurement by 4096 - resulting in values between 0 and 1.
The characteristic points were measured with 32 high-speed cameras (200 fps). They were tracking the Cartesian position of twelve markers placed, as shown in Figure 1. Then, they were recalculated to the joint angles with a five-degree-of-freedom kinematic model (see Figure 2) and normal- ized by dividing by the maximum values reached. The joints represent the following motions: shoulder abduction/adduction (ϕ1), shoulder flexion/ex- tension (ϕ2), shoulder rotation (ϕ3), elbow flexion/extension (ϕ4), and elbow rotation (ϕ5). The values correspond directly to the rotation of exoskeleton motors’ shafts.
Figure 1: Placement of markers.
Three participants completed a calibration phase involving randomized movements covering the entire range of motion of the elbow and shoulder without any additional load. They were repeating the standardized routine presented simultaneously in the video. Following, they performed five simple exercises:
Use of Predictive Models Based on Biomedical Signals and Motion Measurements 101
1. Arm extension with elbow flexion upwards; 2. Reaching the opposite arm; 3. Shoulder blade pull with elbow extension; 4. Overhead claps with elbow flexion; 5. Arm circles in front of the body.
Figure 2: Applied kinematic model.
These were performed at two different paces (fast or slow) and with vary- ing loads (no load, 0.5 kg load, or 1.5 kg load). The loads were supposed to simulate possible differences in individual dynamics of the extremity to design a robust model.
NEURAL NETWORK MODELLING Recurrent neural networks were modelled in Matlab software with the Neu- ral Net Time Series toolbox. They were all based on Nonlinear Autoregressive with Exogeneous Input (NARX) architecture, enabling prediction based on current and prior states. The six input signals were EMG for every muscle group, while the outputs were five degrees of freedom (refer to the example model in Figure 3).
Figure 3: Example neural network model.
Within the training process, the Levenberg-Marquardt optimization algo- rithm was used. The inputs were normalized to the range of 0 to 1, while the outputs were normalized to the range of -1 to 1, as described before. The training dataset was randomly divided into three subsets: training (70%), validation (15%), and test (15%). The hidden neurons have sigmoidal, while the output neurons have linear activation functions.
RESULTS AND DISCUSSION
As part of the search for optimal neural network parameters, various net- works were trained on the same learning set. Initially, the networks with only different numbers of neurons in the hidden layer (5 to 10) were con- structed. After analyzing the results to select the best-performing network, the impact of changing the horizon values was tested. Table 1 presents the results obtained by all the analyzed networks on the whole dataset - when predicting waveforms from the participant on which it was trained and while predicting for another participant. The “Network Name” column contains the code representing the neural network’s parameters, where the first two digits indicate the length of the EMG horizon, the next two digits indicate the angle value horizon, and the last two digits indicate the number of hidden neurons.
Table 1. Prediction results obtained by each network.
Table available in the original PDF paper.
Learning set Same participant Another participant Network name Average difference [
Based on the results and the computation, a network with the following parameters was selected as the best-performing:
Number of hidden layers: 1;
Number of hidden neurons: 9;
Hidden layer activation function: sigmoidal;
Output layer activation function: linear;
Number of epochs before reaching the minimum gradient value (stop condition): 2716;
Use of Predictive Models Based on Biomedical Signals and Motion Measurements 103
EMG horizon: 50 steps;
Angle horizon: 10 steps;
Mean squared error in the final epoch: 7.43 ∗10−10;
Gradient value in the final epoch: 2.85 ∗10−6; The network was designed to operate in two modes during modelling. Ini- tially, the first 200 steps (corresponding to the first second of movement) functioned in an open-loop form, utilizing actual output values. After this ini- tial phase, the computation transitioned to a closed-loop form for prediction, relying only on the input data (EMG) and former predictions. This simulated predicting the further motion of the multibody system at every timestamp for the second ahead.
The network’s performance was evaluated based on the differences between the predicted and real-life angle values. Figure 4 illustrates a com- parison of the predicted and real-life elbow flexion and extension angles, along with the prediction errors, for the network trained and tested on the same participant’s data. On the contrary, Figure 5 presents the results of the network trained and tested on the different participants’ data. The former has an accuracy not worse than 0.05
, while the latter’s accuracy ranges even up to 1
. Moreover, these increase while testing on a dataset acquired for the different loads. The errors with the largest amplitude occurred during rapid accelerations in the joints.
Throughout the parameter changes, including variations in the number of hidden neurons and horizons, the network consistently indicates the following trends independent of these:
The network achieves the largest errors on the learning set when predicting elbow flexion and extension angle values;
The smallest errors are achieved for the prediction of elbow rotation angle values;
The largest prediction inaccuracies rise with the increasing acceleration.
Figure 4: Comparison of the real-life and predicted elbow flexion and extension angles of the network trained and tested on the same participant’s data during a series of five simple exercises.Figure 5: Comparison of the real-life and predicted elbow flexion and extension angles of the network trained and tested on the different participants’ data during a series of five simple exercises.
Additionally, for a fixed number of hidden neurons (10), the impact of decreasing and increasing the EMG horizon was examined. A slight decrease resulted in worse results, while an increase improved the accuracy for the same participant but worsened it for a different one. However, no significant overall changes in the prediction outcomes were observed.
Furthermore, the effect of changing the length of the angle horizon was analyzed. Increasing it led to a neglectable improvement in the network’s performance but significantly increased the computation time. On the other hand, reducing the horizon to 5 drastically affected the results, particularly while testing and training the network on the different participants’ datasets. In such a case, the maximum error exceeded 41
.
To validate the methodology performance independently of the dataset, the procedure was replicated on another research team’s similar measure- ment dataset (Hubaut, 2022). The results from this replication confirmed the universality of the approach, as the obtained results were on a similar level of quality to the ones obtained in own research.
CONCLUSION
The conducted investigation proves that the proposed methodology is suit- able for predicting the kinematics of a patient exercising at least one second in advance. For this purpose, a series of trials involving networks of differ- ent architectures were performed. These were based on the collected datasets with positions of the upper limb and EMG signals of the corresponding mus- cles. The achieved accuracy at the level of 0.01 degree is considered sufficient, similar to the typical servodrives positioning accuracy. Regarding results, the proposed approach can be applied to predictive control over any rehabilita- tion exoskeleton. This can be realised by performing an individual calibra- tion of the system and then training models, each dedicated to a different user.
For practical reasons, the angle data can be recorded directly with the exoskeleton’s encoders. Thanks to this, the motion capture and multibody
Use of Predictive Models Based on Biomedical Signals and Motion Measurements 105
model will be eliminated from the procedure. However, involving these in the proof of concept eliminated potential inaccuracies of the designed exoskeleton and risks of harm to the test objects. As the final outcome, the methodology is planned to be incorporated into the predictive control of ExoReha.
REFERENCES
Atif, Muhammad, et al., “What impact does medicines shortages have on patients?
A qualitative study exploring patients’ experience and views of healthcare profes- sionals”, BMC Health Services Research, 2021. Falkowski, Piotr, “An optimisation problem for exoskeleton-aided functional rehabilitation of an upper extremity”, IOP Conference Series: Materials Science and Engineering, 2022a, DOI: 10.1088/1757-899x/1239/1/012012. Falkowski, Piotr, “Analysis of needs and requirements of kinesiotherapy in Poland for
robot design purposes”, Postepy Robotyki, t. 2, 2022b, ISBN: 9788381564106. Falkowski, Piotr, “Light Exoskeleton Design with Topology Optimisation and FEM Simulations for FFF Technology” Journal of Automation, Mobile Robotics and Intelligent Systems, JAMRIS, Vol. 15, No. 2, 2021. Falkowski, Piotr, et. al., “Study on the Applicability of Digital Twins for Home
Remote Motor Rehabilitation”. Sensors. 2023; 23(2):911. https://doi.org/10. 3390/s23020911 Falkowski, Piotr, “Predicting Dynamics of a Rehabilitation Exoskeleton with Free Degrees of Freedom”, Automation 2022: New Solutions and Technologies for Automation, Robotics and Measurement Techniques. AUTOMATION 2022. Advances in Intelligent Systems and Computing, vol. 1427. Springer, Cham. https://doi.org/10.1007/978-3-031-03502-9_23 Gutenbrunner, Christoph, et al., “Why rehabilitation must have priority during
and after the COVID-19-pandemic: A position statement of the global reha- bilitation alliance”, Journal of Rehabilitation Medicine, Volume. 52, 2020, ISSN: 16501977. DOI: 10.2340/16501977-2713. Hubaut, Remy et al., (2022), “Validation of an Embedded Motion-Capture and EMG Setup for the Analysis of Musculoskeletal Disorder Risks dur- ing Manhole Cover Handling”, Sensors, Volume 22, ISSN: 14248220. DOI: 10.3390/s22020436. Leis, A. Arturo, and Vicente C. Trapani, (2000), Atlas of electromyography. Oxford
University Press. Levac, Danielle, Miller, Patricia, Missiuna, Cheryl, “Usual and virtual reality video game-based physiotherapy for children and youth with acquired brain injuries”, Physical and Occupational Therapy in Pediatrics, Volume 32, pages: 180–195, 2012, ISSN: 01942638. DOI: 10.3109/01942638.2011.616266. Maughan, Tim, (2017), “The Promise and the Hype of ‘Personalised Medicine”’, New Bioethics, Volume 23, pages: 13–20, ISSN: 20502885. DOI: 10.1080/20502877.2017.1314886. Munoz, Karla, et al., (2019), “Remote Rehabilitation: Remote Rehabilitation: A
Solution to Overloaded & Scarce Health Care Systems”, Trends in Telemedicine & E-health, DOI: 10.31031/TTEH.2018.01.000503. Musleh, Atieh, (2022), “Remote Management: A Temporary Need or A Future Necessity?”, European Journal of Business and Management Research, Volume 7, pages: 1–5, 5, DOI: 10.24018/ejbmr.2022.7.5.1531.
Pascanu, Razvan, Mikolov, Tomas, Bengio, Yoshua, (2013), “On the difficulty of training recurrent neural networks”, Proceedings of the 30th International Confer- ence on International Conference on Machine Learning - Volume 28 (ICML’13). JMLR.org, III–1310–III–1318. Taborri, Juri et al., (2020), “Sport biomechanics applications using inertial force and
EMG sensors: A literature overview”, Applied Bionics and Biomechanics, Volume 2020, ISSN: 17542103. DOI: 10.1155/2020/2041549. Wilk, Julia, Falkowski, Piotr, “A concept of detecting patient hazards during exoskeleton-aided remote home motor rehabilitation”, Postepy Robotyki, t. 2, 2022, ISBN: 9788381564106. Yin, Gui, et al., “Processing Surface EMG Signals for Exoskeleton Motion Con-
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-92-6
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional