Evaluation of the Use of Real-time 3D Graphics to Augment Therapeutic Music Sessions for Young People on the Autism Spectrum
J. J. McGowan, I. McGregor, G. Leplâtre · 2021 · ACM Transactions on Accessible Computing · doi:10.1145/3445032
Summary
CymaSense is a real-time 3D visualization application based on cymatics—the study of making sound visible through vibration patterns—designed to augment therapeutic music sessions for autistic individuals. The system maps audio parameters to visual elements: amplitude controls shape scale, pitch determines the cymatic shape and color lightness, and timbre affects surface quality. The researchers conducted a 12-week study with eight autistic adults at TouchBase (a Sense Scotland charity center), using a single-case experimental design with AB and BAB reversal phases. Three prototype configurations were tested: a single microphone visualizing room sounds, two microphones showing separate projections for participant and tutor, and an interactive table providing tangible interaction. Sessions lasted approximately 30 minutes each, facilitated by an experienced music tutor who worked with improvisational music therapy techniques. The study combined quantitative behavioral coding using the CRASS (Checklist of Communicative Responses/Acts Score Sheet) methodology with qualitative data from behavior change questionnaires and semi-structured interviews with parents and the music tutor. In total, 88.5 hours of video footage were analyzed to track changes in musical and non-musical communicative behaviors across intervention and non-intervention phases.
Key findings
The study provided quantitative evidence that CymaSense increased communicative behaviors for the majority of autistic participants. Statistical analysis using Wilcoxon signed-rank tests showed significant increases in both musical and non-musical communicative behaviors during intervention phases compared to non-intervention phases. The BAB reversal design in Phase 2 further validated these findings, demonstrating that the positive effects were attributable to the intervention rather than familiarity with the music tutor or environment. The Behavior Change Questionnaire yielded positive mean scores of 5.6-5.7 on a 7-point scale. Importantly, participants missed the CymaSense application during withdrawal phases, actively seeking the visual feedback they had grown accustomed to. Interview data from three parents and the music tutor revealed that participants showed increased independence, sense of agency, and improved communication both during and after sessions. The tutor observed reduced stereotypical and repetitive behaviors, greater enthusiasm, and more reciprocal musical interaction. Parents reported positive behavioral changes extending to home environments.
Relevance
This research demonstrates the potential of multi-sensory audio-visual technology to enhance therapeutic interventions for autistic individuals. The study provides a model for designing accessible interactive systems that facilitate communication without requiring verbal skills or complex motor control. The emphasis on sense of agency—allowing participants to control and customize their visual experience—aligns with best practices in accessible design. The findings support the value of tangible and natural user interfaces in accessibility contexts, particularly for individuals with limited verbal communication. For practitioners, the study highlights that audio-visual tools can serve as a bridge to encourage social interaction and turn-taking, potentially reducing reliance on caregivers. The research also demonstrates that adults with autism benefit from such interventions, addressing a gap in autism research that typically focuses on children.
Tags: autism · music therapy · multi-sensory · audio-visual · cymatics · tangible user interface · natural user interface · communication