Leveraging Proprioception to Make Mobile Phones More Accessible to Users with Visual Impairments
Frank Chun Yat Li, David Dearman, Khai N. Truong · 2010 · Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2010) · doi:10.1145/1878803.1878837
Summary
This paper extends Virtual Shelves — an interaction technique that leverages proprioception (the sense of body position and movement) to provide eyes-free access to mobile phone shortcuts — for users with visual impairments. The core concept is elegant: users hold their phone in front of them, press and hold a button, move the device to a specific spatial position around their body (like reaching for items on an imaginary shelf), then release to activate the associated shortcut. This eliminates the need for screen readers or voice commands, both of which have significant limitations in mobile contexts — screen readers occupy the user's hearing needed for safety awareness, and voice commands fail in noisy environments. The research consisted of two studies. Study 1 measured the directional accuracy of 9 visually impaired participants (8 blind, 1 low vision, ages 20s-50s) pointing a Nokia N93 phone at 14 virtual targets arranged in theta (horizontal) and phi (vertical) planes. Study 2 evaluated a functional prototype with 13 visually impaired participants (7 blind, 6 low vision) who customized the placement of seven shortcuts across 15 spatial regions. The prototype used a Nintendo Wii MotionPlus accelerometer and gyroscope housed in a custom iPhone-shaped case, communicating via Bluetooth.
Key findings
Study 1 revealed that visually impaired participants had significantly greater selection error and slower selection times than sighted participants from prior work — the median age difference (40-50 vs. 26-35) may have contributed, as proprioceptive accuracy degrades with age. To accommodate higher error rates, the researchers expanded the spatial grid from 7x4 to 7x5 regions with 30-degree spacing, creating 15 shelves in a revised 3x5 layout. In Study 2, participants achieved 81.8% correct selections overall, with 88.3% accuracy when including selections of neighboring regions. The north-middle region (home position, directly in front) was the fastest and most accurate. Middle vertical regions outperformed top and bottom regions significantly. No two participants chose the same seven regions for their shortcuts, confirming the importance of personalization. Participants placed the most frequently used items (calling contacts, directions) closest to the home position. Semi-structured interviews revealed participants found Virtual Shelves faster than their existing mobile phone interfaces, with comments like "oh it feels like one of those iPhones" about the prototype's form factor.
Relevance
This research demonstrates a creative alternative input modality for mobile accessibility that bypasses the visual channel entirely — an approach that remains relevant as touch screens dominate mobile devices. The key insight for accessibility practitioners is that proprioception is a reliable and underutilized sense for interaction design. Users can build spatial muscle memory for frequently used functions, reducing cognitive load compared to navigating screen reader menus. The personalization finding — that no two users chose identical layouts — reinforces the importance of customizable interfaces rather than one-size-fits-all designs. While modern smartphones have largely addressed some of these challenges through improved VoiceOver/TalkBack and gesture systems, the concept of spatial, proprioceptive shortcuts remains a promising direction for reducing the interaction overhead that screen reader users face with complex mobile applications.
Tags: visual impairment · proprioception · mobile accessibility · interaction techniques · eyes-free interaction · screen reader