In clinical practice, objective and quantitative assessment of motor dysfunction is required for monitoring disease progression over time and evaluating response to therapeutic interventions. Thereby, clinicians typically want their patients to make movements to their full physical potential. Augmented reality (AR) games using 3D hand and body tracking that engage patients, could motivate them to perform repetitive tasks to the limit of their physical capabilities in a safe environment. This paper reports on different AR games developed for objective upper extremity motor dysfunction assessment of Parkinson's Disease (PD) patients and stroke patients. Quantitative and qualitative evaluations of various user studies involving 23 PD patients, 22 stroke patients and 39 healthy persons are discussed to make design recommendations for designing engaging AR games for objective assessment of upper extremity motor dysfunction.@en

We focus on the problem of estimating human hand-tremor frequency from input RGB video data. Estimating tremors from video is important for non-invasive monitoring, analyzing and diagnosing patients suffering from motor-disorders such as Parkinson’s disease. We consider two approaches for hand-tremor frequency estimation: (a) a Lagrangian approach where we detect the hand at every frame in the video, and estimate the tremor frequency along the trajectory; and (b) an Eulerian approach where we first localize the hand, we subsequently remove the large motion along the movement trajectory of the hand, and we use the video information over time encoded as intensity values or phase information to estimate the tremor frequency. We estimate hand tremors on a new human tremor dataset, TIM-Tremor, containing static tasks as well as a multitude of more dynamic tasks, involving larger motion of the hands. The dataset has 55 tremor patient recordings together with: associated ground truth accelerometer data from the most affected hand, RGB video data, and aligned depth data.

@en

In clinical practice, upper extremity motor impairments are commonly assessed with disease-specific, subjectively scored and low-resolution rating scales that often do not consider the variations in tasks and environment that are essential aspects of daily life. Augmented reality (AR) systems with contactless tracking of the hand and upper body offer opportunities for objective quantification of motor (dys)function in a challenging, engaging and patient-tailored environment. In this study, we explore the potential of AR for evaluating 1) speed and goal-directedness of movements within the individually determined interaction space, 2) adaptation of hand opening to objects of different sizes, and 3) obstacle avoidance in healthy individuals (N = 10) and two highly prevalent neurological conditions (N = 10 patients with Parkinson’s Disease and N = 10 stroke patients). We successfully implemented three AR games to evaluate these key aspects of motor function. As expected, PD patients moved slower than controls and needed more time for task completion. No differences were observed between stroke patients and controls, perhaps because motor impairments in this patient group were relatively mild. Importantly, usability of our AR system was good and considerably improved compared to our previous study due to more natural and patient-tailored interaction. Although our findings testify to the potential of AR for assessing motor impairments in patients with neurological conditions and provide starting points for further improvement, there are still many steps to be taken towards application in clinical practice.@en

Advances in technology offer new opportunities for a better understanding of how different disorders affect motor function. Our aim is to explore the potential of augmented reality (AR) using free hand and body tracking to develop engaging games for a uniform, cost-effective and objective evaluation of upper extremity motor dysfunction in different patient groups. Based on the insights from a study with 20 patients (10 Parkinson’s Disease patients and 10 stroke patients) who performed hand/arm movement tasks in AR, we created a set of different augmented reality games for upper extremity motor dysfunction assessment. @en

Proper understanding of motor control requires insight into the extent and manner in which task performance and control strategy are influenced by various aspects of visual information. We therefore systematically manipulated the visual presentation (i.e., scaling factor and optical flow density) of a visuomotor tracking task without changing the task itself, and investigated the effect on performance, effort, motor control strategy (i.e., anticipatory or corrective steering) and underlying neuromechanical parameters (i.e., intrinsic muscle stiffness and damping, and proprioceptive and visual feedback). Twenty healthy participants controlled the left-right position of a virtual car (by means of wrist rotations in a haptic robot) to track a slightly curved virtual road (presented on a 60” LED screen), while small torque perturbations were applied to the wrist (1.25–20 Hz multisine) for quantification of the neuromechanical parameters. This visuomotor tracking task was performed in conditions with low/medium/high scaling factor and low/high optical flow density. Task performance was high in all conditions (tracking accuracy 96.6%–100%); a higher scaling factor was associated with slightly better performance. As expected, participants did adapt their control strategy and the use of proprioceptive and visual feedback in response to changes in the visual presentation. These findings indicate that effects of visual representation on motor behavior should be taken into consideration in designing, interpreting and comparing experiments on motor control in health and disease. In future studies, these insights might be exploited to assess the sensory-motor adaptability in various clinical conditions.@en

Advances in technology offer new opportunities for a better understanding of how different disorders affect motor function. In this paper, we explore the potential of an augmented reality (AR) game implemented using free hand and body tracking to develop a uniform, cost-effective and objective methods for evaluation of upper extremity motor dysfunction in different patient groups. We conducted a study with 20 patients (10 Parkinson’s Disease patients and 10 stroke patients) who performed hand/arm movement tasks in four different conditions in AR and one condition in real world. Despite usability issues mainly due to non-robust hand tracking, the patients were moderately engaged while playing the AR game. Our findings show that moving virtual objects was less targeted, took more time and was associated with larger trunk displacement and a lower variability of elbow angle and upper arm angle than moving real objects. No significant correlations were observed between characteristics of movements in AR and movements in the real world. Still, our findings suggest that the AR game may be suitable for assessing the hand and arm function of mildly affected patients if usability can be further improved. @en