Comparing Two Safe Distance Maintenance Algorithms for a Gaze-Controlled HRI Involving Users with SSMI
Vinay Krishna Sharma, L. R. D. Murthy, Pradipta Biswas · 2022 · ACM Transactions on Accessible Computing · doi:10.1145/3530822
Summary
This paper presents an eye gaze-controlled augmented reality interface for operating a tabletop robotic arm, designed specifically for people with severe speech and motor impairment (SSMI). Users with SSMI often cannot manipulate physical objects due to spasticity but are familiar with eye-pointing communication. The system enables them to control a DOBOT Magician robotic arm for tasks like picking and placing objects, fabric printing, drawing, and painting. The interface uses a video see-through AR approach: users see the real robotic arm and workspace through a tablet camera, with control overlays superimposed. The system employs a custom appearance-based gaze estimation algorithm (AGE-Net) that works with a standard webcam—avoiding the need for expensive specialized eye trackers. The interface is deliberately language-agnostic, using only four directional controls without text to ensure accessibility across different language speakers. A critical safety challenge is that users may place their hands on the table while operating the robot, and the arm must never collide with body parts. The paper integrates MediaPipe Hands for real-time hand detection and compares two safe distance maintenance algorithms: a naive geometric circle-arc approach and an intelligent Markov Decision Process (MDP) algorithm that uses value iteration to find optimal paths while avoiding detected hands.
Key findings
The MDP-based intelligent algorithm significantly outperformed the naive circle-arc approach in simulation studies, reducing movement distance by a statistically significant margin (t = -13.1, p < 0.001) while maintaining safe separation. The intelligent algorithm also accounts for the periodic hand movements typical of users with spasticity by remembering previous hand positions and anticipating that hands may return to earlier locations. A user study with 13 participants (6 users with SSMI who were quadriplegic wheelchair users, plus 7 able-bodied university students) demonstrated that all participants could complete pick-and-place tasks. Users with SSMI could bring the robotic arm to any designated point within its working envelope in under 3 minutes. The average separation distance between the robot's end effector and users' palms was 25 cm, and it never fell below 13 cm—confirming the safety algorithms worked effectively. Task completion times improved significantly across trials for both groups (p < 0.05), demonstrating learnability. The participant type had a significant main effect (F(1,11) = 106.16, p < 0.05, η² = 0.906), with able-bodied users completing tasks faster, but both groups showed consistent improvement with practice.
Relevance
This research addresses a critical gap in assistive robotics: enabling people with the most severe motor impairments to manipulate physical objects independently. The population—people with SSMI who cannot control their limbs voluntarily due to conditions present from birth, accident, disease, or genetic disorders—often cannot use even standard assistive technologies due to their severity of impairment. For accessibility practitioners, several design decisions are instructive: the language-agnostic interface ensures global accessibility; the use of standard webcams rather than expensive specialized hardware lowers barriers to adoption; and the explicit attention to safety through collision avoidance addresses a real concern when robots operate in personal spaces. The work demonstrates that eye gaze can be a viable control modality for complex manipulation tasks, not just communication. Future applications could include fabric printing, painting, playing with toys, and other rehabilitation activities. The safe distance algorithms could be extended to avoid other obstacles or adapted to different robot configurations. Limitations include the current 2D workspace constraint and the need for users to maintain a fixed distance from the camera.
Tags: eye gaze tracking · robotics · severe motor impairment · human-robot interaction · augmented reality · assistive technology · collaborative robots
Standards referenced: ISO/TS 15066 · ISO 10218-1