A Performance Comparison of On-Hand versus On-Phone Nonvisual Input by Blind and Sighted Users
Uran Oh, Leah Findlater · 2015 · ACM Transactions on Accessible Computing (TACCESS) · doi:10.1145/2820616
Summary
This study investigates whether using one's own hand as an input surface offers performance advantages over touchscreen phones for blind users performing nonvisual interaction. On-body input is particularly attractive for blind users because it provides enhanced tactile and proprioceptive feedback compared to flat touchscreens, and the hand is always available without needing to retrieve a phone from a pocket. The researchers conducted a controlled lab study with 12 sighted and 11 blind participants (average age ~52 years, all with touchscreen experience). Two tasks were evaluated: a pointing task requiring participants to touch 20 target locations on the hand or phone, and a shape-drawing task involving five gestures (circle, triangle, square, plus sign, and equals sign). Sighted participants were blindfolded to ensure nonvisual conditions. The experimental apparatus used camera tracking, color markers on the pointing finger, and capacitive touch sensors to enable consistent measurement across both surfaces. The hand condition leveraged natural anatomical landmarks—fingertips, finger webs, palm edges, and wrist—to define target regions. This design choice made the hand condition more ecologically valid but meant target sizes varied across locations, unlike the uniform grid on the phone.
Key findings
For the pointing task, the hand was significantly faster than the phone (3.59s vs 3.98s average selection time), confirming prior work with sighted users. More notably, first-contact accuracy—whether the finger landed within target bounds on initial touch—was substantially higher on the hand (57.0%) than the phone (34.8%). This suggests proprioception provides location information even before tactile contact occurs. Target location significantly affected performance on the hand. Fingertips were the fastest and most accurate targets, followed by the top row of the palm. The fifth row (near the wrist) and third column were slowest and least accurate. For the phone, outer edge columns were fastest. These findings have direct design implications: frequently-used commands should be placed on fingertips. For shape gestures, the hand resulted in significantly higher gesture recognition rates than the phone, though overall recognition remained relatively low (around 60-70%) for practical deployment. Contrary to the researchers' hypothesis, blind participants did not show greater performance benefits from on-hand input than sighted participants, despite research showing blind individuals have superior tactile acuity. The practical differences in tactile sensitivity may simply be too small to matter at this scale. Subjective preferences diverged: 9 of 12 sighted participants preferred the hand for pointing, but blind participants were evenly split, likely due to their existing familiarity with touchscreen phones.
Relevance
This research provides empirical evidence that on-body input is a viable alternative input modality for accessible mobile computing. For practitioners designing wearable or hands-free interfaces, the key insight is that proprioceptive feedback—knowing where your body parts are in space—provides meaningful accuracy benefits over flat touchscreens, even before tactile contact. The finding that fingertips offer the fastest and most accurate pointing has direct design implications: gestural shortcuts or frequently-accessed controls should target fingertip regions. The variation in accuracy across different fingers (index vs. ring finger) also suggests considering finger-specific acuity when placing targets. While the lab setup with stabilized hands doesn't reflect real-world conditions, emerging wearable sensing technologies (infrared reflective sensors, flexible skin overlays) may soon enable practical on-body input. For blind users, on-hand interaction could complement existing touchscreen accessibility by providing quick audio control without retrieving the phone—particularly valuable when hands are otherwise occupied (carrying groceries, using a cane).
Tags: on-body interaction · mobile accessibility · blindness · touchscreen accessibility · gesture input · proprioception · tactile feedback · nonvisual interaction · alternative input · user study