Towards automated video-based assessment of dystonia in dyskinetic cerebral palsy

Towards automated video-based assessment of dystonia in dyskinetic cerebral palsy: A novel approach using markerless motion tracking and machine learning

Haberfehlner, Helga (Katholieke Universiteit Leuven; Amsterdam UMC; Amsterdam Movement Sciences, Rehabilitation & Development)
van de Ven, Shankara S. (Amsterdam UMC)
van der Burg, Sven A. (Netherlands eScience Center)
Huber, Florian (Netherlands eScience Center; University of Applied Sciences, Düsseldorf)
Georgievska, Sonja (Netherlands eScience Center)
Aleo, Ignazio (Moveshelf Labs B.V.)
Harlaar, J. (TU Delft Biomechatronics & Human-Machine Control)
Bonouvrié, Laura A. (Vrije Universiteit Amsterdam)
van der Krogt, Marjolein M. (Amsterdam UMC; Amsterdam Movement Sciences, Rehabilitation & Development)
Buizer, Annemieke I. (Emma Children's Hospital Academic Medical Center, University of Amsterdam; Amsterdam UMC; Amsterdam Movement Sciences, Rehabilitation & Development)

2023

Introduction: Video-based clinical rating plays an important role in assessing dystonia and monitoring the effect of treatment in dyskinetic cerebral palsy (CP). However, evaluation by clinicians is time-consuming, and the quality of rating is dependent on experience. The aim of the current study is to provide a proof-of-concept for a machine learning approach to automatically assess scoring of dystonia using 2D stick figures extracted from videos. Model performance was compared to human performance. Methods: A total of 187 video sequences of 34 individuals with dyskinetic CP (8–23 years, all non-ambulatory) were filmed at rest during lying and supported sitting. Videos were scored by three raters according to the Dyskinesia Impairment Scale (DIS) for arm and leg dystonia (normalized scores ranging from 0–1). Coordinates in pixels of the left and right wrist, elbow, shoulder, hip, knee and ankle were extracted using DeepLabCut, an open source toolbox that builds on a pose estimation algorithm. Within a subset, tracking accuracy was assessed for a pretrained human model and for models trained with an increasing number of manually labeled frames. The mean absolute error (MAE) between DeepLabCut’s prediction of the position of body points and manual labels was calculated. Subsequently, movement and position features were calculated from extracted body point coordinates. These features were fed into a Random Forest Regressor to train a model to predict the clinical scores. The model performance trained with data from one rater evaluated by MAEs (model-rater) was compared to inter-rater accuracy. Results: A tracking accuracy of 4.5 pixels (approximately 1.5 cm) could be achieved by adding 15–20 manually labeled frames per video. The MAEs for the trained models ranged from 0.21 ± 0.15 for arm dystonia to 0.14 ± 0.10 for leg dystonia (normalized DIS scores). The inter-rater MAEs were 0.21 ± 0.22 and 0.16 ± 0.20, respectively. Conclusion: This proof-of-concept study shows the potential of using stick figures extracted from common videos in a machine learning approach to automatically assess dystonia. Sufficient tracking accuracy can be reached by manually adding labels within 15–20 frames per video. With a relatively small data set, it is possible to train a model that can automatically assess dystonia with a performance comparable to human scoring.

cerebral palsy
human pose estimation
machine learning
markerless skeleton tracking
motion capture
movement disorders

http://resolver.tudelft.nl/uuid:e4931165-63d4-4137-9b38-1d19339b3de2

https://doi.org/10.3389/frobt.2023.1108114

2296-9144

Frontiers In Robotics and AI, 10

Institutional Repository

journal article

© 2023 Helga Haberfehlner, Shankara S. van de Ven, Sven A. van der Burg, Florian Huber, Sonja Georgievska, Ignazio Aleo, J. Harlaar, Laura A. Bonouvrié, Marjolein M. van der Krogt, Annemieke I. Buizer