BrailleSketch: A Gesture-based Text Input Method for People with Visual Impairments
Mingzhe Li, Mingming Fan, Khai N. Truong · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17) · doi:10.1145/3132525.3132528
Summary
This paper presents BrailleSketch, a gesture-based text input method for touchscreen smartphones designed for people with visual impairments who already know the Braille alphabet. Rather than requiring users to tap specific screen locations or use multi-finger chording (as with Perkins Brailler-style methods), BrailleSketch lets users sketch a single-finger path that connects all the dots in the corresponding Braille code for each letter. Users can place their finger anywhere on the screen to begin and draw the Braille pattern in whatever direction feels natural — the system computes a 5x3 grid of dots centered at the initial touch point and uses the gesture path to determine which Braille dots are intended. The system provides two types of audio feedback: a sine-wave tone indicating whether each new dot is in a cardinal or diagonal direction from the previous one (helping users detect errors mid-gesture), and word-level speech output when a space is entered. Crucially, BrailleSketch deliberately omits immediate letter-level audio feedback — inspired by research showing that delaying feedback to the word level increases typing speed. An auto-correction algorithm based on minimum string distance against a 10,000-word dictionary corrects errors at the word level. The system was implemented as an Android application and tested on a Huawei Ascend Mate7 phone with a 6.0-inch screen.
Key findings
Ten legally blind participants from CNIB (Canadian National Institute for the Blind), with an average of 32.3 years of Braille experience, completed five typing sessions each. In the 3-phrase typing tests, average speed grew from 6.56 wpm in the first session to 14.53 wpm in the final session, with the best individual performance reaching 19.32 wpm. The total error rate decreased from 14.8% to 10.6% across sessions. Importantly, speed had not plateaued by the fifth session, suggesting continued improvement with practice. BrailleSketch outperformed most other Braille-based touchscreen input methods in speed (compared to BrailleType at 1.49 wpm, SingleTapBraille at 4.71 wpm, Perkinput at 6.05 wpm, and EdgeBraille at 7.17 wpm), though BrailleTouch expert users reached 23.2 wpm with its six-finger chording approach. The auto-correction algorithm successfully corrected 55% of mistyped words. Substitution errors were the most common error type, particularly for letters with similar Braille codes like G/N, S/P, and Y/G, where the direction of the initial gesture path caused confusion. The reduced audio feedback strategy was validated: a comparison test with one participant showed that immediate letter-level feedback slowed typing to 8.37 wpm versus 11.53 wpm without it.
Relevance
BrailleSketch demonstrates a practical approach to leveraging existing Braille literacy for efficient touchscreen text entry, which is significant given that text input remains one of the most time-consuming mobile tasks for people with visual impairments. The design is notable for its flexibility — users do not need to target specific screen locations, learn a new alphabet, or use multiple fingers simultaneously, removing key barriers found in competing methods. For accessibility practitioners, this research reinforces that input method design must consider the full interaction cost, including audio feedback overhead, not just gesture mechanics. The finding that reducing letter-level feedback actually improves speed challenges common assumptions about the necessity of continuous audio confirmation in non-visual interfaces. The work also highlights a limitation: BrailleSketch requires Braille knowledge, which not all people with visual impairments possess, particularly those with late-onset blindness. This positions BrailleSketch as a complement to, rather than replacement for, other input approaches like speech or screen reader keyboards.
Tags: text entry · braille · gesture interaction · visual impairment · mobile accessibility · touchscreen accessibility · auto-correction