Typing Slowly but Screen-Free: Exploring Navigation over Entirely Auditory Keyboards
Reeti Mathur, Aishwarya Sheth, Parimal Vyas, Davide Bolchini · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3308561.3353789
Summary
This paper introduces the concept of "keyflows" — entirely auditory keyboards that operate without a reference screen, enabling blind users to type while untethered from their mobile device. The core problem is that current accessible keyboards require blind users to constantly hold and interact with their phone screen, using VoiceOver or TalkBack to find visual keys they cannot see. This is particularly impractical when walking with a cane or guide dog, multitasking, or in situations where pulling out a phone is inconvenient or unsafe. A keyflow is a looping auditory stream of TTS characters spoken at a rate of two characters per second, arranged alphabetically (A-Z), which users navigate and select from using hand gestures on a MYO armband worn near the elbow. Five gestures control the keyflow: double-tap (select character), wave-in (skip ahead by chunks of 5 letters), wave-out (backtrack letter-by-letter), fingers-spread (delete), and fist (read back typed letters). The design went through iterative refinement with blind co-researchers: a probabilistic letter ordering (by frequency) was rejected because it created unpredictability and cognitive fatigue — users strongly preferred the alphabetical order despite its theoretical inefficiency. A "chunking" mechanism groups letters into bursts of five with 1.5-second pauses between chunks to align with short-term memory capacity. A two-character rewind offset compensates for the delay between hearing a target letter and executing the selection gesture.
Key findings
Twenty screen-reader users (18 blind, 1 blind with low vision in one eye, 1 extremely low vision) from BOSMA Enterprises participated in the study, typing the words "BUS" and "MARSH" plus optional additional words. 14 of 20 participants typed at least one complete word. For "BUS," 10 participants succeeded with an average time of 1 minute 23 seconds (31.6% error rate). For "MARSH," only 3 participants succeeded with an average of 2 minutes 4 seconds (57% error rate). Typing was dramatically slower than screen-based methods — but this was expected for a first exposure to an entirely new interaction paradigm. Three distinct navigation strategies emerged: (1) listening and waiting — patiently following the alphabetical flow and selecting on the second loop for safety (3 participants); (2) skipping chunks and fine-tuning — jumping ahead by chunks to approximate the target letter's location, then scanning letter-by-letter (13 participants, the most common strategy); (3) retracing — overshooting past the target and backtracking letter-by-letter (4 participants). Despite the slow speed, most participants preferred keyflows to voice input and screen-based typing for short on-the-go messages. Fourteen participants said they would use keyflows outdoors while walking with a cane or guide dog. Participants valued keeping their phone in their pocket, typing hands-free, and maintaining awareness of environmental sounds. Privacy was also cited — voice typing exposes information to bystanders. Error sources included gesture misrecognition (the armband frequently failed), wrong character selection due to reaction time delays, gesture confusion, and system errors.
Relevance
This research opens a fundamentally new design space for accessible text entry by challenging the assumption that keyboards require a visual or spatial reference point. The conceptual contribution — moving the keyboard from a location in space to a flow in time — has broad implications for accessibility. For blind users, being "tethered" to the phone screen is not merely inconvenient but creates real barriers to mobility, safety, and multitasking. The finding that participants overwhelmingly preferred the predictable alphabetical ordering over the theoretically more efficient probabilistic ordering provides an important design lesson: controllability and predictability matter more than optimization, especially for novel interaction paradigms. The three navigation strategies (wait, skip-and-scan, overshoot-and-retrace) reveal how users naturally adapt to serial auditory information — insights applicable to any auditory navigation interface. The practical trade-off the paper identifies is clear: keyflows are much slower than screen-based typing but offer untethered, screen-free, hands-free typing that is valuable for specific contexts like walking, multitasking, or quick short messages. As wearable input devices improve beyond the limited MYO armband (smart rings, finger-worn devices), the keyflow concept could become significantly more practical. The paper also highlights that 25% of participants had comorbid conditions (autism, short-term memory loss, muscular degeneration, diabetic neuropathy) that compounded the interaction challenges, underscoring that accessible design must account for intersecting disabilities.
Tags: text entry · blindness · visual impairment · screen reader · auditory interface · screenless interaction · wearable technology · gesture-based interaction · mobile accessibility · eyes-free interaction