Playing Music with the Head
Emeline Brulé · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16) · doi:10.1145/2982142.2982146
Summary
This short paper describes the co-design process leading to an open source music-playing software for M., a person living with severe motor, dexterity, and speech impairments in a nursing home in France. The system uses an infrared camera mounted on top of a computer to track the reflections of a sticky dot placed on the user's forehead, translating head movements into cursor control. The software, built with Pure Data, divides the screen into a musical zone where head position controls different musical parameters depending on the selected instrument — for example, with percussion, the Y axis controls tempo while the X axis controls sound level, and with keyboard instruments, the X axis maps to musical notes. Top and bottom screen areas serve as menus for selecting instruments. The project emerged from the e-fabrik initiative, which pairs teenagers (apprentices), educators (accomplices), and people with impairments (associates) to use local fablabs and workshops for creating assistive solutions. The design process began with participant observations in the nursing home, followed by iterative co-design sessions that used Wizard of Oz techniques and adapted communication methods to enable M.'s participation despite her speech impairments.
Key findings
The paper focuses primarily on the co-design methodology rather than formal evaluation results. The iterative process required adapting typical design methods to accommodate M.'s communication abilities — using regular iterations to facilitate communication and enable meaningful participation. The resulting system allows M. to play music either alone or with others, addressing both individual creative expression and social participation. The hardware uses readily available components (infrared camera, reflective dot, standard computer), and the software is open source, making it replicable and adaptable for similar cases. The project illustrates how participation in the design process itself can reinforce broader participation in other areas of life, aligning with French disability policy (the 2002 healthcare modernization bill) that mandates personal projects for institutional residents and requires annual renewal with caregiver and resident input. The nursing home caregivers had engaged in multiple DIY assistive technology projects through collaborations with non-profit organizations when commercial accessible products did not meet residents' needs.
Relevance
This paper contributes to two important intersections in accessibility: music as a form of creative participation for people with severe motor disabilities, and DIY/open source assistive technology as a response to the gap between commercial products and individual needs. For accessibility practitioners, it demonstrates that head tracking — typically associated with computer access — can be repurposed for creative musical expression, opening up cultural participation for people whose motor abilities are limited to head movement. The co-design approach is particularly notable because it addresses the challenge of including people with severe speech impairments in the design process, using iterative prototyping and Wizard of Oz techniques rather than traditional verbal methods. The open source nature of the solution is significant: by making the software freely available and using commodity hardware, the project avoids the cost and availability barriers that often prevent people in institutional settings from accessing adapted music technology. The project also highlights the role of maker communities and fablabs in bridging assistive technology gaps.
Tags: music accessibility · motor disability · co-design · participatory design · DIY assistive technology · open source · alternative input · creative arts · head tracking