Tactile Graphics with a Voice: Using QR Codes to Access Text in Tactile Graphics
Catherine M. Baker, Lauren R. Milne, Jeffrey Scofield, Cynthia L. Bennett, Richard E. Ladner · 2014 · ASSETS '14: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2661334.2661366
Summary
This paper presents Tactile Graphics with a Voice (TGV), a system that makes text labels in tactile graphics accessible by replacing Braille labels with QR codes that can be scanned and read aloud using a smartphone. The motivation is significant: while tactile graphics are essential for blind students accessing textbook figures, the text labels on these graphics are typically embossed in Braille—yet less than 40% of the functionally blind population in the US is fluent in Braille. TGV uses a smartphone camera to scan QR codes placed on tactile graphics and speaks the encoded text aloud. The system offers three feedback modes for aiming the camera: Silent (no guidance, chimes on success), Verbal (spoken feedback about QR code positions), and Finger Pointing (detects the user's finger via skin-color detection to select which QR code to scan). The authors conducted formative studies including a survey of 22 blind and low vision people and follow-up interviews, then a six-session longitudinal evaluation with ten participants.
Key findings
Across all modes, TGV achieved 88-89% accuracy on math-related tasks (finding intercepts, coordinates, hypotenuse lengths, and bar chart values), comparable to the 88% accuracy achieved by Braille-literate participants using Braille labels. Four participants who could not read Braille were able to successfully answer questions using TGV that they could not answer with Braille equivalents—demonstrating TGV's value for non-Braille readers. Average task completion time was 40.9 seconds with QR codes versus 28.6 seconds with Braille, though this gap narrowed significantly over the six sessions as participants gained experience. QR codes can encode 45% more text than equivalently-sized Braille labels, and QR code labels averaged 225mm² compared to 327mm² for Braille. Feedback preferences varied widely: four participants preferred Silent mode, four preferred Finger Pointing, and two preferred Verbal, highlighting the importance of offering multiple interaction options.
Relevance
This research addresses a critical gap in accessible education: making tactile graphics usable for the majority of blind people who are not fluent in Braille. For accessibility practitioners, TGV demonstrates how mainstream technology (smartphones and QR codes) can be repurposed as low-cost assistive tools without requiring specialized hardware. The system is particularly relevant for STEM education, where graphical information is essential but often inaccessible. The finding that feedback preferences varied significantly among participants reinforces the importance of providing multiple modalities rather than a one-size-fits-all approach. The study also reveals practical challenges in creating tactile graphics—experts reported that a quarter to a third of all graphics require key-and-legend systems because Braille labels cannot fit on the graphic itself, a problem QR codes could help solve.
Tags: visual impairment · blindness · tactile graphics · QR codes · Braille · education · assistive technology · non-visual feedback