Ally: Understanding Text Messaging to Build a Better Onscreen Keyboard for Blind People
Danielle Lottridge, Chris Yoon, Darren Burton, Chester Wang, Jofish Kaye · 2022 · ACM Transactions on Accessible Computing · doi:10.1145/3533707
Summary
This research presents Ally, a smartphone onscreen keyboard designed specifically for blind users who need to enter text messages when voice input or external keyboards are unavailable. The authors recognized that blind smartphone users employ heterogeneous input methods—voice dictation for speed, external Bluetooth keyboards for accuracy, braille input for those fluent in braille, and the standard QWERTY soft keyboard as a backup. However, QWERTY is poorly optimized for non-visual input, requiring sequential auditory search where users slide their finger across keys while VoiceOver announces each letter. The design process began with corpus analysis, comparing 241,776 text messages from modern SMS corpora (MTurk and NUS datasets) against standard English text from books and news. Text messages showed distinct character frequencies—notably higher use of "o" and lower use of "t" compared to standard English—driven by words like "you," "to," and informal shorthand. The authors algorithmically evaluated 104 possible keyboard layouts, optimizing for character frequency at easy-to-locate positions: bottom corners (near phone case edges), top corners, and center bottom (above the home button). The final Ally design arranges the alphabet counterclockwise around the screen edges starting with "a" in the top-left, maintaining alphabetical order for easy recall while placing frequent SMS characters at tactile landmarks.
Key findings
Two user studies evaluated Ally with blind VoiceOver users. The pilot study (13 participants, average age 39, mean 4.9 years iPhone experience) showed significant speed improvement between first and last five trials (p=0.004), with participants improving from 4.08 wpm to 5.48 wpm after just 11 trials (~18 minutes). Two participants matched their QWERTY baseline speed. The larger user study (10 participants, average age 31.2) found that after approximately one hour of practice across five 20-minute sprints, half of participants reached or exceeded their QWERTY typing speed. Baseline QWERTY speed for blind participants was only 8.03 wpm—dramatically lower than the 38 wpm typical for sighted users on soft keyboards. Ally achieved 5.3 wpm overall with comparable error rates (6.6% total error rate vs. 8.2% for QWERTY). Critically, blind participants reported using multiple input methods depending on context: voice for speed but with privacy concerns and accuracy issues in noisy environments; external Bluetooth keyboards for accuracy but inconvenient to carry; braille input for those fluent (only ~10% of US blind population can read braille fluently). The onscreen keyboard serves as a necessary backup when other methods are unsuitable—particularly for text messaging, the most-used smartphone app category.
Relevance
This research addresses a practical gap in mobile accessibility: blind users need a usable fallback keyboard for situations where their preferred input methods (voice, external keyboards) are inappropriate. The "perpetual novice" design philosophy is particularly valuable—rather than optimizing for expert performance, Ally prioritizes rapid learning and ease of use for occasional users. For accessibility practitioners, the corpus-based optimization approach demonstrates how to design for specific use cases. Text messages differ linguistically from formal writing, and keyboard optimization should reflect actual usage patterns. The use of physical phone landmarks (case edges, home button area) as tactile guides shows how hardware affordances can enhance software accessibility. The study also documents the heterogeneous input strategies blind users employ—valuable context for understanding real-world smartphone accessibility needs. Limitations include the lack of text prediction and autocorrect in Ally, features that significantly aid QWERTY performance, and testing with phrase sets rather than actual SMS composition. Future work combining speech and keyboard input could further improve blind text entry.
Tags: blindness · text entry · mobile accessibility · keyboard design · VoiceOver · touchscreen · smartphone accessibility