Leveraging Dynamic Audio-Tactile UIs to Assist Visually Impaired Users in Exploring Line Charts through Tactile Graphic Readers
Gaspar Ramôa, Karin Müller · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746398
Summary
This paper presents the Melodic Line Tracing Exploration UI, a novel dynamic audio-tactile user interface designed to help blind and visually impaired (BVI) individuals interpret complex line charts through tactile graphic readers. The research addresses a significant gap: while tactile graphic readers like the Tactonom Reader allow BVI users to explore raised-line graphics with audio feedback, the state-of-the-art tap-to-hear interaction method struggles with complex charts containing multiple overlapping lines and intersections. The melodic UI was developed through a three-month participatory design process with two blind audio engineers and introduces three key components: (1) a synth melody environment that assigns each line a distinct synthesized instrument (piano, organ, violin, trombone) and pitch-mapped melody note, allowing users to aurally distinguish lines in the multi-line overview; (2) pitch trace guidance that provides continuous pitch-based feedback reflecting the user's vertical finger position along a selected line, creating an intuitive spatial audio representation; and (3) line boundary feedback using gong-like percussion sounds and pitch shifts to identify line edges, maxima, and minima. The system was implemented on the Tactonom Reader, a 5.3 kg tactile graphic reader with a magnetic surface that combines swell paper tactile graphics with SVG-linked audio elements detected via an overhead camera and fingertip tracking. A within-subjects user study with 10 BVI participants (6 congenitally blind, 4 visually impaired) compared the melodic UI against the standard tap-to-hear interface using four realistic, complex line charts from renewable energy data, each containing eight lines with numerous intersections and overlaps.
Key findings
The melodic UI was significantly more effective than tap-to-hear, with participants completing an average of 8.15 out of 9 tasks correctly per chart pair compared to 2.05 for tap-to-hear (p < 0.001) — a four-fold improvement. The melodic UI achieved this without increasing interaction times; mean task duration was 720 seconds vs. 819 seconds for tap-to-hear (no significant difference, p = 0.154). Cognitive load was significantly lower with the melodic UI (NASA-TLX score of 25.50 vs. 46.58, p = 0.002), and user satisfaction was significantly higher (SUS score of 86.75 vs. 69.50, p = 0.002). Utility ratings averaged 4.5/5 for the melodic UI vs. 3.0/5 for tap-to-hear. All three melodic UI components received high individual ratings: synth melody environment 4.80/5, pitch trace guidance 4.7/5, and line boundary feedback 4.9/5. Crucially, the UI empowered both beginners and advanced users equally, with minimal variation in performance (standard deviation of 1.09 tasks). The most challenging task for both UIs was determining the number of distinct intersections between lines (T-Intersect), highlighting a remaining frontier in chart accessibility. Participants reported that the melodic UI fundamentally shifted their perception of chart complexity, with users expressing that previously inaccessible complex charts now felt manageable. Participants suggested broader applications including floor plans, maps, mathematical graphs, and educational games.
Relevance
This research makes a compelling case for moving beyond basic tap-to-hear interfaces in tactile graphic readers toward richer, context-specific audio-tactile interactions. The four-fold improvement in effectiveness without time penalties challenges the assumption that more feature-rich assistive interfaces necessarily increase cognitive load or interaction time. For accessibility practitioners, the key takeaway is that generic interaction paradigms are insufficient for complex graphical content — specialized UIs designed for specific chart types can dramatically improve comprehension. The participatory design approach with blind audio engineers demonstrates how domain expertise from BVI professionals can drive innovation in assistive technology design. The work has practical implications for STEM education, data journalism, and workplace accessibility, where complex data visualizations remain a major barrier for BVI individuals. The Tactonom Reader platform and the broader applicability to other tactile graphic readers and refreshable pin displays suggest potential for real-world deployment. Limitations include the small sample size (10 participants), single-hand interaction constraint of the device, and the focus on basic comprehension tasks rather than higher-level analytical activities.
Tags: data visualization · tactile graphics · sonification · audio-tactile interface · blindness and visual impairment · participatory design · assistive technology · line charts