GestureCalc: An Eyes-Free Calculator for Touch Screens
Bindita Chaudhuri, Leah Perlmutter, Justin Petelka, Philip Garrison, James Fogarty, Jacob O. Wobbrock, Richard E. Ladner · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3308561.3353783
Summary
This full research paper presents GestureCalc, an eyes-free, target-less gesture-based calculator for touch screens, along with a rigorous three-session longitudinal evaluation with eight screen reader users. The system replaces the spatial button layout of conventional calculators with metaphor-based gestures performed anywhere on the screen: digits use taps (1-3 fingers) and directional swipes with an additive prefix-free coding scheme (0 = down swipe, 1 = one-finger tap, 2 = two-finger tap, 3-6 use three-finger tap prefix, 6-9 use upward swipe prefix), while operations follow the "more is higher" metaphor (+ = two-finger up, - = two-finger down, * = three-finger up, / = three-finger down, = = two-finger right swipe, delete = one-finger left, clear = two-finger left, decimal = long tap). The coding scheme averages 1.7 gestures per digit, fewer than DigiTaps (1.8) and BrailleTap (2.5), and does not require Braille knowledge — an important consideration since only about 10% of legally blind people can read Braille. The design was informed by a formative pilot study with four participants that validated the memorability and usability of the gesture codes. The system was developed for iOS in Swift and compared against ClassicCalc, a recreated version of the default iOS calculator using standard VoiceOver interaction (seek with finger, double-tap or split-tap to activate).
Key findings
Eight screen reader users (ages 23-58, 6 male, 2 female) participated in three one-hour sessions separated by at least four hours. Each session included both GestureCalc and ClassicCalc (counterbalanced), with three blocks of 10 recorded trials per calculator. Total dataset: 1,440 trials. Character entry speed was 40.5% faster with GestureCalc (0.753 vs 0.536 characters per second, p < .0001). GestureCalc showed significant improvement across all three sessions (p < .0001 between sessions 1 and 3) while ClassicCalc performance plateaued, suggesting GestureCalc has a higher performance ceiling with practice. Uncorrected error rate was 59.8% lower with GestureCalc (0.92% vs 2.29%, p < .001), and erroneous calculations were 52.2% fewer (33 vs 69, p < .001, Fisher's exact test). However, corrected error rate was 138.1% higher with GestureCalc (5.31% vs 2.23%, p < .0001), meaning participants made more errors but caught and fixed them more frequently. The speed advantage came primarily from eliminating seek time — with ClassicCalc, users must first find each button on screen before activating it, while GestureCalc gestures can be performed anywhere. NASA TLX scores showed no significant differences across any dimension, though GestureCalc trended higher on mental demand (learning new gestures) and ClassicCalc trended higher on effort and frustration. Interviews revealed participants valued eliminating the "guesswork" of spatial search (P5: "gestures are good; they take the guesswork out"), appreciated the low verbosity compared to VoiceOver (P8: "it wasn't overly wordy"), and recognized the learning investment required while expressing confidence they would become fluent. Three-finger gestures were the most physically challenging due to screen width limitations, with 2 of 8 participants preferring landscape mode. Participant P5 highlighted the real-world impact: "Right now, I'm studying to take the test to get into math classes. I haven't been able to take the test yet because I don't have a decent calculator."
Relevance
This paper provides rigorous longitudinal evidence that target-free gesture-based interfaces can substantially outperform spatial button-based interfaces for screen reader users on touchscreens — a finding with implications far beyond calculators. The 40.5% speed improvement and 52.2% fewer erroneous calculations are practically significant gains for an everyday task. The longitudinal design is particularly valuable: it shows that GestureCalc's performance continues to improve over sessions while ClassicCalc plateaus, suggesting the initial learning investment in gesture codes pays off increasingly over time. For accessibility practitioners, the conceptual metaphor approach to gesture design offers a generalizable principle: grounding gesture codes in intuitive physical concepts ("up means more," quantity maps to finger count) makes them learnable without visual reference. The finding that only 10% of legally blind people read Braille — making Braille-based input methods inaccessible to 90% of the target population — underscores the importance of designing accessible interfaces that do not assume Braille literacy. The decomposition of typing time into think time, seek time, and gesture time provides a useful framework for understanding where accessible interfaces create bottlenecks: GestureCalc's advantage comes from eliminating seek time entirely, even though individual gestures take slightly longer than simple taps. The paper also demonstrates that accessibility research should measure not just error rates but the distinction between uncorrected errors (which produce wrong results) and corrected errors (which indicate the interface supports error recovery).
Tags: gesture-based interaction · touchscreen accessibility · blindness · visual impairment · screen reader · eyes-free interaction · mobile accessibility · calculator · input methods · conceptual metaphor · longitudinal study