How Accessible are Virtual Reality Freehand Gestures? Understanding Barriers for Users with Upper Limb Motor Impairments
Lauren Pococke, Crescent Jicol, Christof Lutteroth, Christopher Clarke · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746402
Summary
This paper provides the first empirical examination of how accessible default freehand gesture implementations in virtual reality are for people with upper limb motor impairments. The researchers recruited 8 adults with conditions including lupus, rheumatoid arthritis, carpal tunnel, spinal injury (C4 incomplete), psoriatic arthritis, and post-stroke impairments (ages 18-91), alongside a control group of 16 participants without diagnosed impairments (ages 17-80, including older adults). Using a Meta Quest Pro headset with the Meta Interaction SDK, participants completed 28 VR tasks covering four fundamental freehand gesture types: pinch (selection and manipulation), grasp (picking up and rotating objects), extended finger (pressing buttons and scrolling), and open palm (swiping and pressing). The study collected comprehensive data including objective physical hand measurements (grip strength, pinch strength, finger range of motion, wrist range of motion, Kapandji thumb opposition test, and Jebsen-Taylor Hand Function Test), subjective impairment questionnaires (QuickDASH), visual analogue scale ratings for pain, effort, and discomfort after each gesture block, video recordings of both external interactions and internal headset views, and semi-structured interviews. The analysis combined observational coding of interaction challenges with thematic analysis of interview data, identifying 41 initial codes refined to 17 codes across 236 data points.
Key findings
Five core themes of barriers emerged. First, recognition of precise hand poses caused cascading issues—the system expected specific finger positions that participants with limited range of motion could not achieve, and five participants performed pinching by pressing their thumb against the side of their straightened index finger rather than tip-to-tip, which the system did not recognize. Second, interactions relied too heavily on the index finger unnecessarily—pinch, trigger, and extended finger gestures all required the index finger, creating a single point of failure for anyone with index finger impairments. One participant with arthritis noted their index finger was "stuck in a claw position" making pinch gestures impossible as designed. Third, interaction issues caused a domino effect where self-adaptations to overcome one barrier (e.g., compensating for limited wrist ROM by using elbow/shoulder movements) created secondary problems including increased fatigue, self-occlusion of hand tracking cameras, and reduced accuracy. Fourth, the lack of physicality in VR both helped and hindered—some participants appreciated reduced joint stress, while others struggled without the tactile feedback they relied on for grip control. Fifth, speed requirements for repeated selections excluded participants who could not maintain accuracy under time pressure. Median pain scores for participants with impairments were dramatically higher than controls: pinch 1.50 vs 0.00, grasp 8.50 vs 0.00. Critically, physical measurements did not reliably predict interaction difficulties, suggesting that standard assessments alone cannot determine which adaptations a user needs.
Relevance
This research fills a critical gap in VR accessibility by demonstrating that even when users retain hand functionality, default freehand gesture implementations cause significant pain, fatigue, and frustration. The finding that pinching—the most widely used gesture across Meta Quest, Apple Vision Pro, and Microsoft HoloLens—is also the most problematic gesture for users with motor impairments has major implications for the XR industry. The paper challenges the assumption that freehand interactions are inherently more accessible than controllers, showing that rigid gesture thresholds and index finger dependence create unnecessary barriers. For accessibility practitioners, the key takeaway is that VR systems need flexible, adaptive gesture recognition that accepts natural variation in how people perform movements, rather than requiring exact replication of predefined poses. The observation that physical measurements do not predict interaction difficulties underscores the need for dynamic, in-context assessment approaches rather than one-time calibration.
Tags: virtual reality · motor accessibility · freehand gestures · hand tracking · upper limb impairments · XR accessibility · interaction design
Standards referenced: WHO International Classification of Functioning, Disability and Health