Using an Audio Interface to Assist Users Who are Visually Impaired with Steering Tasks
Robert F. Cohen, Valerie Haven, Jessica A. Lanzoni, Arthur Meacham, Joelle Skaff, Michael Wissell · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169008
Summary
This paper presents results from an ongoing study of PLUMB (exPLoring graphs at UMB), a software system that uses audio feedback and a pen-based Tablet PC interface to make relational graphs and diagrams accessible to users who are blind. Graphs — consisting of vertices (nodes) and edges (lines connecting them) — are fundamental to computer science education and many professional fields, yet are typically presented visually, putting students with visual impairments at a severe disadvantage. PLUMB displays a drawn graph on a Tablet PC screen and generates audio cues as the user moves a stylus over graph elements. When the stylus contacts an edge, a continuous musical tone is played; the tone is modified with a vibrato effect that increases in intensity as the user approaches a vertex, indicating proximity to endpoints. Variations in loudness represent distance from the central axis of the edge. When the stylus enters or exits a vertex, a sound is played and speech announces the element's name. Right-clicking provides more detailed spoken descriptions. The system was built using C# and .NET, with DirectX for sound, Microsoft Speech API for voice output, and MIDI for musical tones. Graph data is stored in GXL (Graph eXchange Language), an XML-based format.
Key findings
Human trials with ten participants without usable vision (three congenitally blind, seven adventitiously blind) evaluated steering performance across edges varying in width (20, 40, 60 pixels), angle, direction, and screen position. Key findings: wider edges were significantly easier to follow — mean errors dropped from 18.72 (small) to 7.53 (medium) to 4.20 (large width), while mean traversal time decreased from 31.22 to 21.85 to 17.74 seconds. Edge angle and direction were not statistically significant factors in performance. Full-screen edges produced significantly higher error rates and exploration times than small-quadrant tasks. Congenitally blind participants spent more time (31.95 vs 20.93 seconds) and made more errors (13.14 vs 9.96) than adventitiously blind participants, and were much more sensitive to edge width — their accuracy degraded sharply as edges narrowed. However, the error-to-time ratio was similar between groups (0.41 vs 0.47), suggesting comparable efficiency. An interesting observed strategy was "tacking" — zigzagging along an edge like a sailboat, which produced many exits/errors but allowed confident navigation. The relationship between errors and distance followed the Accot-Zhai steering law, which models the time to traverse a constrained tunnel.
Relevance
This research addresses a fundamental gap in STEM accessibility: the inability of blind users to access graphical and diagrammatic information that is central to computer science, mathematics, and many other fields. Rather than simply describing graphs verbally, PLUMB enables active tactile-audio exploration, allowing users to build spatial mental models through direct interaction. The finding that the system required minimal training — participants could use it after a brief introduction — is encouraging for practical adoption. The significant performance differences between congenitally and adventitiously blind participants highlight that visual experience (even if no longer present) influences spatial reasoning strategies, an important consideration when designing non-visual interfaces. The study's systematic evaluation of graph parameters (edge width, angle, screen position) provides concrete design guidelines: wider edges, shorter lengths, and smaller display areas improve accessibility. These findings extend beyond graph exploration to any audio-guided steering task in non-visual interfaces.
Tags: visual impairment · sonification · graph accessibility · audio interface · tablet PC · steering task · non-visual interaction · education