Encouraging Speech and Vocalization in Children with Autistic Spectrum Disorder
Joshua Hailpern · 2007 · SIGACCESS Accessibility and Computing · doi:10.1145/1328567.1328576
Summary
This paper reviews existing HCI research on autism spectrum disorder and proposes a new direction: using technology to encourage speech and vocalization in children with ASD who have not acquired spoken language. The author identifies a gap in the field — while HCI researchers have worked on early detection tools, virtual environments for social skills training, and technology-mediated play with robots and interactive displays, almost no work has targeted using technology to directly teach low-functioning children with ASD to vocalize and speak. The paper reviews three drawbacks of traditional applied behavior analysis (ABA) approaches to speech therapy: the need for many sessions with scarce trained professionals, the requirement for intense practitioner attention, and the inherent difficulty of requiring human interaction from children who experience anxiety in social contact. The proposed approach uses real-time visual and auditory feedback generated in response to a child's vocalizations, leveraging the observation that children with ASD are often strongly motivated by technology and that real-time visualizations can act as "social mirrors" influencing communication. The system presents approximately a dozen visualization types across four categories — Falling, Spinning, Found Imagery, and Flashy — chosen to align with stimuli that engage children on the spectrum, paired with four different sound feedback metaphors.
Key findings
Phase One of the research had been completed at the time of writing, with initial qualitative results described as promising. Children enrolled in six sessions where approximately eight different combinations of visual and auditory feedback were presented over 40-minute periods, with play and relaxation breaks between trials. Many children appeared to have preferred forms of feedback that elicited far greater numbers and frequency of sounds. Parents reported that their children became excited or elated when arriving for sessions, suggesting the visual and auditory stimuli were intrinsically engaging. One child showed particularly notable results, exhibiting signs of turn-taking with auditory feedback and mimicking generated sounds as if playing a call-and-response game — prompting researchers to plan additional sessions using a Wizard-of-Oz controlled visualization requesting specific sounds. The quantitative analysis plan included measuring time in chair, percent of intervals with positive emotive state, diversity of phonetic repertoire, frequency of utterances, variation in utterance duration, and exploration of utterance volume. The researchers also developed Vcode, a video coding tool to facilitate analysis of session recordings with interrater reliability checking. Future phases planned to explore controlling vocalization frequency, instructing children to make specific sounds, and word formation, as well as portable form factors for learning outside clinical settings.
Relevance
This research highlights an important distinction in assistive communication technology: most AAC work focuses on providing alternative communication methods (picture boards, speech-generating devices), whereas this work aims to use technology as an instructional tool to develop spoken language itself. For accessibility practitioners, this reframes technology's role from compensating for a deficit to actively building a capability — a meaningful conceptual shift. The approach of using computer-generated sensory feedback to motivate vocalization in children who find human interaction stressful is a thoughtful design decision that works with, rather than against, common characteristics of ASD. The research also demonstrates the importance of individualized responses in accessibility technology, as different children showed preferences for different visualization and sound combinations. While the results are preliminary and qualitative, the work laid groundwork for an area that has since grown considerably, with multimodal feedback systems now increasingly explored in speech therapy and autism intervention contexts.
Tags: autism spectrum disorder · speech disorder · vocalization · visual feedback · auditory feedback · augmentative and alternative communication · applied behavior analysis · social robot