Empowering Blind Musicians to Compose and Notate Music with SoundCells
William Christopher Payne, Fabiha Ahmed, Michael Zachor, Michael Gardell, Isabel Huey, Amy Hurst, R. Luke DuBois · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22) · doi:10.1145/3517428.3544825
Summary
This paper presents a six-week remote study in which six blind and visually impaired (BVI) musicians used SoundCells, a text-based music notation web application, to compose original short works culminating in a live performance by a professional musician. SoundCells addresses a significant gap in music technology: commercial notation software relies on graphical user interfaces and produces only visual print scores, while existing braille music tools cannot generate print notation for sighted collaborators. SoundCells uses ABC text — an ASCII-based music notation syntax — as input and simultaneously generates audio playback, visual print scores, and braille music output. The system was co-designed with experienced blind musicians and built on open-source tools including CodeMirror 6 for screen-reader-accessible text editing and music21 for braille and print conversion. The six participants ranged widely in vision ability (blind, visually impaired, low vision), music experience (from decades of professional work to complete novices), notation background (braille music experts to no prior notation experience), and assistive technology preferences (JAWS, VoiceOver, NVDA, magnification). Over the course, participants learned ABC text syntax, engaged with weekly challenges (notation, remix, and composition activities), and created 8-18 measure compositions that were performed live on soprano saxophone. The study used three semi-structured interviews to capture evolving reactions, experiences, and design recommendations.
Key findings
All six participants successfully composed original music with SoundCells despite their diverse backgrounds, demonstrating the system's accessibility across a range of vision abilities and experience levels. Participants engaged deeply with multiple output modalities in complementary ways: music playback was the most popular feature, with participants using real-time audio feedback to verify their notation; braille displays were crucial for navigation and tracking position in complex scores; text-to-speech helped identify errors and understand measure completeness; and one low-vision participant primarily used the visual print score alongside the text. A major finding was the problem of "ocularcentrism" in ABC text syntax — characters like commas, apostrophes, underscores, and carets were chosen for their visual position relative to pitch letters on screen, making them unintuitive and hard to remember for users who cannot see the spatial relationship. This revealed how visual assumptions can be deeply embedded even in ostensibly text-based, screen-reader-accessible formats. Rhythm notation proved the most challenging aspect, requiring participants to mentally track fractional durations to ensure measures were complete — a cognitive burden that GUI-based tools handle automatically. Participants suggested improvements including a "musical typing" mode mapping the QWERTY keyboard to piano keys, auto-completion for rhythms, measure-by-measure braille display layout, and more descriptive screen reader output that interprets syntax rather than reading raw characters. Five of six participants planned to continue using SoundCells.
Relevance
This research fills a critical gap in accessibility work by focusing on creative expression and content creation rather than content consumption — most prior music accessibility research has addressed making existing scores available in braille or large print, not supporting blind musicians in composing new work. The study demonstrates that accessible creative tools can achieve high engagement when they combine genuine expressive opportunities with clear, motivating goals like a live performance. For accessibility practitioners, the finding about ocularcentrism in ABC syntax is broadly applicable: even text-based, nominally accessible formats may embed visual assumptions in their design that create barriers for non-visual users. The multimodal output approach — simultaneously providing audio, braille, print, and text-to-speech — offers a model for designing systems that serve users with diverse abilities and preferences without requiring them to choose a single modality. The study also provides practical evidence that remote, longitudinal research with BVI participants can yield rich data and high engagement, offering a methodological model for future accessibility studies.
Tags: music accessibility · blindness · braille music · creative expression · assistive technology · text-based interface · co-design · longitudinal study