Playing with Feeling: Exploring Vibrotactile Feedback and Aesthetic Experiences for Developing Haptic Wearables for Blind and Low Vision Music Learning
Leon Lu, Jin Kang, Chase Crispin, Audrey Girouard · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2023) · doi:10.1145/3597638.3608397
Summary
This paper explores how musical haptic wearables (MHWs) using vibrotactile feedback can support music learning for blind and low-vision (BLV) people, and investigates the role of material aesthetics in shaping the user experience of such devices. The researchers conducted five two-hour in-person co-design workshops with 10 BLV music teachers and learners (ages 25-72) at The Filomen M. D'Agostino Greenberg (FMDG) Music School in New York City, a school for blind musicians. Participants included four completely blind, four low-vision, and two deafblind individuals who played a range of instruments including piano, saxophone, trumpet, cello, and voice. The study addressed two research questions: how BLV musicians envision using MHWs with vibrotactile feedback to support music learning, and what material preferences BLV musicians have for MHW design. Workshops were structured in three sections: exploring vibrotactile feedback through a proof-of-concept prototype with six preset vibration patterns (alerts, repeated intervals, varying intensity, and varying speed/pattern), exploring eleven material swatches common in music environments (fabrics, silicon, wood, metal, rubber, cork), and ideating through making with craft materials. The prototype consisted of vibration motors in 3D-printed coin-sized casings attached via Velcro bands, controlled by a Bluetooth-enabled microcontroller.
Key findings
The study revealed three categories of vibrotactile feedback applications for BLV music learning. First, vibrotactile alerts (simple on/off vibrations) were well-suited for communicating instructional information — signaling when to start and stop playing, indicating whose turn it is during improvisation, and allowing teachers to discreetly correct students (e.g., indicating they are flat or sharp) without interrupting the flow of music. Second, variations in vibration intensity, speed, and pattern could communicate musical concepts like dynamics and articulation, and participants envisioned using vibration-based notation systems to supplement braille music reading — potentially allowing musicians to keep their hands on their instruments while receiving musical information. Third, vibrotactile alerts placed near body parts involved in producing sound could support technical guidance, such as teaching fingering patterns or complex rhythm patterns like polyrhythm. However, participants also identified limitations: perceiving tempo changes through vibration was challenging, complex music scores would create cognitive overload, and the system was better suited for simple rather than complex pieces. Regarding materials, findings were categorized into sensorial (breathability, malleability, distinguishability, sound dampening, durability), interpretive (associations with musical instruments, weight-quality connection, form for functionality), and affective (curiosity, enjoyment of textures, comfort of familiarity) levels. Sound dampening was particularly important since vibration motors produce buzzing noise that could interfere with music.
Relevance
This research advances the design of assistive technology for BLV musicians by taking a holistic approach that considers both functionality and aesthetics — an area often neglected in AT design. The finding that material experiences evoke positive emotions in BLV participants, which can increase intention to use and sustain long-term engagement, challenges the purely functional approach that dominates AT development. For AT designers, the study provides concrete guidance: wearable devices for BLV users should prioritize breathability, sound dampening, and malleability; they should feel discreet in public performance settings; and their materials should evoke positive associations. The co-design methodology with a blind researcher as a core team member demonstrates best practices for participatory research. The paper also highlights an underserved area — while much accessibility research focuses on screen-based information access, BLV musicians face unique challenges around non-verbal communication, music notation, and technical instruction that require body-worn, tactile solutions rather than screen reader adaptations.
Tags: blind and low vision · assistive technology · haptic feedback · vibrotactile · music learning · co-design · wearable technology · material experience