Designing Auditory Cues to Enhance Spoken Mathematics for Visually Impaired Users
Emma Murphy, Enda Bates, Dónal Fitzpatrick · 2010 · Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2010) · doi:10.1145/1878803.1878819
Summary
This paper presents a novel auditory math system designed to make spoken mathematical equations more accessible to visually impaired users. The core challenge addressed is that visual mathematical notation conveys structure — such as fractions, superscripts, subscripts, and nested brackets — in a two-dimensional spatial layout that is inherently difficult to represent through the linear channels of synthesized speech and Braille. Spoken mathematics frequently introduces ambiguity; for example, the phrase "one over x plus 4" could represent either a fraction containing "x plus 4" in the denominator or a fraction "one over x" followed by "plus 4." The authors's system combines three types of auditory cues to disambiguate mathematical structure: non-speech sounds (earcons such as beeps and noise sweeps to indicate brackets), modified speech called spearcons (shortened spoken phrases like "frac," "sup," and "sub" to label mathematical operators), and binaural spatialization (panning sounds left and right to indicate opening and closing of structural elements). The research follows a participatory design methodology, involving both visually impaired and sighted users in the creative design process through an online survey with 56 participants (22 female, 34 male), including 35 sighted users and 21 visually impaired screen reader users aged 18 to 64.
Key findings
Spearcon recognition accuracy was significantly higher at slower speeds (70% of original) compared to 50% speed, with rates around 80% for visually impaired users. For full equations with non-speech sound, overall accuracy was 67%, though sighted users performed significantly better than visually impaired users, likely because screen reader users found the verbose multiple-choice format more tedious. Partial equations with spatial attributes achieved 61% accuracy with no significant difference between user groups. Users showed a clear preference for bracket sounds: 57% preferred the beep with glissando over noise sweeps, describing it as "clear" and "short," though some users found the noise sweep easier to spatialize and more aesthetically pleasing. Qualitative feedback revealed creative alternative suggestions from participants, including metaphors like a door opening and closing for brackets, USB disconnect sounds, and the idea of using "power" or "to the power of" instead of the confusable "superscript" label. Users confirmed that left-right binaural panning intuitively conveyed opening and closing of structural elements.
Relevance
This paper demonstrates the significant accessibility barriers that mathematical content poses for screen reader users and offers a practical, user-centered approach to solving them. The participatory design methodology — involving blind and sighted users collaboratively in the creative process — is a model for inclusive design that remains relevant today. The findings highlight that even well-designed auditory interfaces face challenges around cognitive load, user training, and the inherent difficulty of linearizing two-dimensional notation. For accessibility practitioners, the work underscores that mathematical accessibility requires more than just reading equations aloud; structural and spatial information must be conveyed through complementary auditory channels. The research also reveals practical challenges in creating accessible online surveys for screen reader users, a concern still relevant for user research today.
Tags: mathematical accessibility · auditory display · non-speech sound · spearcons · visual impairment · screen readers · participatory design · sonification
Standards referenced: Nemeth MathSpeak