An Intelligent Decision Support System for Stroke Rehabilitation Assessment
Min Hun Lee · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019) · doi:10.1145/3308561.3356106
Summary
This student research abstract presents an interactive multimodal machine learning approach for automatically assessing upper-limb stroke rehabilitation exercises and supporting therapist decision-making. Physical rehabilitation is critical for people recovering from stroke to regain functional ability, but assessment requires direct therapist observation, which is expensive and limits access to timely, comprehensive rehabilitation. Existing computational approaches fall into two categories: rule-based models (where therapists articulate if-then rules for motion correctness, offering interpretability but being time-consuming to configure for individual patients) and statistical models (neural networks trained on labelled sensor data, offering accuracy but lacking explainability). This paper proposes combining both through an interactive Multimodal Model (MM) that uses a weighted average ensemble of a Prediction Model (PM, neural network trained on labelled kinematic data) and a Knowledge Model (KM, therapist-derived rules). The system identifies salient features of assessment using Sequential Forward Search with Neural Networks, then presents user-specific analysis through a web interface that visualises how the stroke survivor’s motion compares to their unaffected side. A therapist can review this analysis, provide feature relevance feedback, and the system automatically generates additional personalised rules to update the KM.
Key findings
The system was evaluated on data collected from 15 stroke survivors (Fugl-Meyer Scores of 37±21, indicating a range of impairment severity) and 11 subjects without motor impairment, performing three upper-limb exercises: bringing a cup to the mouth, switching a light on, and moving forward with a cane. Motion was captured using a Kinect v2 sensor, with each stroke survivor performing 10 repetitions per exercise with both affected and unaffected sides. Two therapists independently annotated the recorded videos. Using Leave-One-Subject-Out cross validation, the interactive Multimodal Model (MM2) achieved average F1-scores of 0.9041, 0.8161, and 0.8493 across the three exercises respectively — consistently outperforming the non-interactive multimodal model (MM1: 0.8504, 0.7186, 0.7446), the standalone prediction model (PM: 0.8806, 0.8090, 0.8115), and the rule-based knowledge model alone (KM: 0.6148, 0.6707, 0.4626). The key insight is that statistical and rule-based approaches complement each other: the PM captures complex patterns in motion data while the KM incorporates therapist expertise that the data alone cannot express. Iteratively accommodating therapist feedback through the interactive web interface improved the generic model into a personalised assessment tool. The approach normalises the stroke survivor’s affected side motion against their own unaffected side rather than against a population norm, enabling personalised assessment.
Relevance
This work addresses the scalability gap in stroke rehabilitation: there are not enough therapists to provide the frequent, intensive assessment that optimal recovery requires, particularly in the critical first three months post-stroke. The interactive approach is valuable because it keeps the therapist in the loop rather than replacing clinical judgement — the system presents data-driven analysis and the therapist validates, corrects, and personalises it. For accessibility practitioners working on rehabilitation technology, the specific design of the human-AI interaction is instructive: rather than presenting a black-box prediction, the system visualises salient kinematic features and allows therapists to provide feature-level feedback that generates new personalised rules. The use of the Kinect sensor (a low-cost, widely available depth camera) rather than clinical motion capture equipment is a practical choice that could enable home-based assessment. As a student research abstract, this is preliminary work with a small dataset, but the interactive multimodal framework and the demonstrated improvement from therapist-in-the-loop feedback point toward a promising direction for AI-assisted rehabilitation assessment.
Tags: stroke rehabilitation · machine learning · human-AI interaction · decision support · motor disability · kinematic analysis · interactive machine learning · Kinect