Eyelid Gestures on Mobile Devices for People with Motor Impairments
Mingming Fan, Zhen Li, Franklin Mingzhe Li · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020) · doi:10.1145/3373625.3416987
Summary
This paper presents a novel approach to mobile device interaction for people with severe motor impairments: using eyelid gestures detected by the smartphone's front-facing camera. While existing eye-based interaction research has primarily focused on gaze direction and simple blinks using specialized equipment like dedicated eye trackers, this work exploits the fact that two eyelids can open and close independently, for different durations, and in different sequences — creating a rich gesture vocabulary using only commodity smartphone hardware. The researchers designed a taxonomy of nine eyelid gestures based on four primitive eyelid states (both open, both closed, right closed only, left closed only) combined with two duration levels (short and long). Gestures range from simple single-eye closures (R for right, L for left, B for both) to compound sequences (B-R- for both close then right close held, B-L- for both then left held, BOB for double blink). The detection algorithm uses Google Mobile Vision API to obtain real-time eye-open probabilities from the front camera at 30fps, then employs a cascade of Support Vector Machine classifiers to identify eyelid states, segment gestures, classify duration, and recognize specific gestures. The system was implemented on standard Android phones (Samsung S7, Huawei P20) without any additional hardware.
Key findings
Two user studies were conducted. Study 1 with 12 able-bodied participants (ages 23-35) in two different office environments achieved .76 user-dependent and .68 user-independent overall gesture accuracy, demonstrating robustness across environments and postures. Study 2 with four participants with severe motor impairments — three with cervical spinal cord injuries (C5-C6) using wheelchairs, and one with bilateral forearm amputation using prosthetics — achieved .76 user-dependent and .69 user-independent accuracy, comparable to able-bodied results. The eyelid state classifier alone reached .85 accuracy for the motor-impaired group and .997 when simplified to a two-state (open/closed) classifier. The researchers also designed and evaluated a gesture-to-navigation mapping scheme where simple gestures (R, L) control fine-grained in-app navigation while complex gestures (B-R-, B-L-) handle higher-level app switching. All four participants with motor impairments learned the mapping in under five minutes and found it intuitive. Participants reported wanting to use eyelid gestures across multiple devices (phones, TVs, PCs) and in situations where hands are unavailable, such as lying down, cooking, or using the bathroom. Key design recommendations include: assess individual eyelid control before assigning gestures, allow customizable hold durations, reserve complex gestures for rare/high-error-cost actions, let users define a hard-to-perform "trigger" gesture to prevent false activations, and allow personalized gesture definitions.
Relevance
This research opens a significant new input channel for people with severe motor impairments who cannot reliably use touch, voice, or traditional switch-based interfaces. The use of standard smartphone cameras rather than specialized eye-tracking hardware makes the approach potentially accessible to anyone with a smartphone — a meaningful advantage given the cost and portability limitations of dedicated assistive devices. For accessibility practitioners and developers, the work demonstrates that the front-facing camera is an underutilized accessibility sensor, and that even a relatively small gesture vocabulary (nine gestures) can support full app navigation when mapped thoughtfully. The participant feedback highlights important real-world considerations: individual variation in eyelid control (one participant had difficulty with her right eyelid), fatigue from sustained gestures, and the need for false-positive prevention especially for wheelchair users who may accidentally trigger recognition while positioning their phone. Limitations include the small sample size (N=4 motor-impaired participants), short study duration that did not assess long-term fatigue, and the exclusion of half-closed eyelid states.
Tags: motor accessibility · mobile accessibility · input methods · gesture interaction · eye tracking · assistive technology · spinal cord injury · computer vision