Designing a Text Entry Multimodal Keypad for Blind Users of Touchscreen Mobile Phones
Maria Claudia Buzzi, Marina Buzzi, Barbara Leporini, Amaury Trujillo · 2014 · Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS) · doi:10.1145/2661334.2661354
Summary
This paper presents MTITK (Multimodal Text Input Touchscreen Keypad), a software-only text entry system for blind users of touchscreen phones that combines the familiar 12-key telephone keypad layout with multimodal audio-tactile feedback. The work is grounded in the personal experience of one of the authors, Barbara Leporini, who is blind and uses both a Nokia N95 (physical keypad) and iPhone 4S daily. The paper provides detailed first-person observations of the challenges blind users face with touchscreen text entry: accidental character insertion from slight finger movements, difficulty distinguishing phonetically similar letters (m/n) in TTS, problems with text editing and cursor placement, and the iPhone's proximity sensor disabling the screen during calls. MTITK uses the standard ITU-T telephone keypad with multitap character entry (pressing a key multiple times to cycle through its assigned letters). Keys are classified into five groups with distinct vibration patterns and audio tones: cardinal-point keys (2,4,6,8), intercardinal-point keys (1,3,7,9), centre key (5), zero, and symbol keys (*,#). Users explore the screen with one finger to identify keys through haptic-audio feedback, then tap anywhere on screen the required number of times to select a character. Gestures based on single right-angled swipes provide editing functions with granularity semantics: swipe left-then-up deletes last character, extend to swipe left-then-down-then-right to delete a phrase. The system supports English, French, Italian, and Spanish character repertoires per ETSI ES 202 130. The prototype was implemented on Android (API level 17) and tested on Galaxy Nexus, Nexus 4, and Nexus 5 devices.
Key findings
Usability testing proceeded in two phases. A preliminary assessment with two blind participants (including the blind co-author) using think-aloud protocol showed that MTITK was well received — participants could accurately identify keys through audio-tactile feedback even in noisy environments and appreciated the familiar keypad layout. The co-author, who retained physical keypad muscle memory, was better at remembering character assignments. A pilot test with 14 visually impaired participants (6 female, 8 male; ages 26-70, median 44; recruited from three local centres for blind and low-vision people in Italy) revealed mixed-to-positive reception. All participants preferred combined voice and haptic feedback over voice alone, particularly in noisy settings. However, half of the users had difficulty differentiating vibration patterns for key identification — haptic feedback alone was insufficient, and the current vibration pattern distinctions needed to be more pronounced. Participants instinctively lifted their finger between taps (mimicking physical key presses), which conflicted with the touchscreen interaction model. Editing gestures with steep angles were difficult, and people blind from early age had more difficulty with gesture shapes. Participants particularly valued MTITK for: private and discrete text entry where TTS dictation would be inappropriate, and telephone number composition (a simpler task where haptic key identification was sufficient). iOS was overwhelmingly preferred (11 of 16 used iPhones) due to VoiceOver's maturity, though the researchers chose Android for its openness and customizability.
Relevance
This paper provides a uniquely grounded perspective on touchscreen text entry accessibility, combining formal research with authentic lived experience from a blind co-author. The first-person account of juggling two phones — a physical keypad phone for texting and calls, a touchscreen phone for apps and web browsing — vividly illustrates the accessibility trade-offs blind users make daily. For accessibility practitioners, several insights stand out. First, the familiar telephone keypad layout leverages existing knowledge that many blind users already have, reducing the learning burden compared to novel input methods. Second, noisy environments are a critical and often overlooked use case — audio-only feedback (the basis of VoiceOver and TalkBack) becomes unreliable in noise, making haptic feedback essential as a complementary channel. Third, the observation that vibration patterns need to be more distinct reveals a practical limitation of current smartphone haptic hardware, which typically offers limited vibration variety. Fourth, the finding that participants preferred iOS despite the research being Android-based underscores VoiceOver's significance in the blind mobile user community and the real-world platform choices that accessibility quality drives. The software-only approach requiring no additional hardware makes MTITK deployable on any touchscreen device, though the current vibration limitations highlight the need for richer haptic capabilities in mobile devices.
Tags: mobile accessibility · text entry · blindness · multimodal interaction · haptic feedback · touchscreen accessibility · Android
Standards referenced: ITU-T E.161 · ISO 9995-8 · ETSI ES 202 130