SoundCells: Designing a Browser-Based Music Technology for Braille and Print Notation
William Payne, Fabiha Ahmed, Michael Gardell, R. Luke DuBois, Amy Hurst · 2022 · Proceedings of the 19th International Web for All Conference (W4A) · doi:10.1145/3493612.3520462
Summary
This paper from New York University presents SoundCells, a free, open-source, browser-based music notation tool designed to be accessible to screen reader users while outputting music in three formats simultaneously: audio playback, visual print scores, and braille music notation. The work addresses a fundamental gap in music technology: existing notation software (Finale, Sibelius, MuseScore) relies on visual WYSIWYG interfaces with drag-and-drop positioning that are largely inaccessible to blind musicians, while the few braille-specific tools (Braille Music Editor, Duxbury) produce only braille output and cost hundreds of dollars. Converting between print and braille music typically requires a complicated multi-step toolchain involving 4+ software applications, vision ability for early steps, and braille expertise for later steps. SoundCells takes a fundamentally different approach: users type music in ABC notation — a plain ASCII text syntax — within a web-based code editor (CodeMirror 6), and the system renders print scores (via Open Sheet Music Display), braille scores (via music21), and audio (via tone.js and Web Audio) in real time. The QWERTY keyboard functions like a MIDI piano, playing notes as they are typed. "Tell-me" commands (typing ? followed by a character) provide contextual information about notes and measures via ARIA live regions.
Key findings
The tool was co-designed over two months with two expert blind braille musicians from the Filomen M'D'Agostino Greenberg (FMDG) Music School in New York City, through four remote meetings, and then evaluated in Design Probes with five additional blind and low-vision musicians. Reactions were "immediately and overwhelmingly positive" — both co-designers found the ABC syntax approachable and independently composed music after short instruction, with one improvising a blues version and the other composing the entire tutorial tune using copy-and-paste. Key findings include: (1) Text input separates musical content from presentation, making it inherently more accessible than WYSIWYG — ABC text is directly readable by screen readers and always semantically explicit. (2) Multi-modal output served different purposes — braille for proofreading, audio for verification, and co-designers used a combination that matched their individual preferences. (3) The design facilitated unexpected remote collaboration between sighted researchers and blind musicians — ABC text could be emailed, copy-pasted, and edited across teams without requiring conversion between formats. (4) Educators envisioned SoundCells lowering barriers for music students, noting that students who lose vision especially struggle with print-based notation software. Several accessibility challenges emerged with CodeMirror 6's screen reader support, cross-browser ARIA live region inconsistencies, and the difficulty of managing competing audio streams (music playback vs. screen reader speech).
Relevance
SoundCells demonstrates how text-based approaches to creative tools can fundamentally improve accessibility over visual GUI paradigms — a principle applicable far beyond music notation. The parallel to accessible web development is explicit: just as semantic HTML is more accessible than visually-styled content, text-based music notation (ABC) is more accessible than visually-positioned scores. For accessibility practitioners, the co-design methodology with blind expert users is exemplary — the researchers worked with musicians who are both end users and domain experts, ensuring the tool addresses real workflow needs rather than imagined ones. The multi-modal output approach (simultaneous text, audio, and tactile) provides a model for accessible creative tools generally. The finding that the tool enabled cross-ability remote collaboration — sighted and blind musicians sharing ABC text seamlessly — illustrates how accessibility can improve workflows for everyone, not just users with disabilities. The challenges around competing audio streams (music vs. screen reader) and CodeMirror's incomplete screen reader support highlight persistent infrastructure-level accessibility gaps in web development tools.
Tags: music accessibility · blindness · visual impairment · braille · screen readers · co-design · creative accessibility · web accessibility
Standards referenced: WAI-ARIA · WCAG