visiBabble Demo
Harriet Fell, Joel MacAuslan, Jun Gong, Josh Ostrow · 2005 · Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '05) · doi:10.1145/1090785.1090829
Summary
This paper presents a demonstration of visiBabble, a computer-based system designed to encourage and reinforce pre-speech vocalizations in infants at risk of being nonspeaking due to neurological or oral/motor impairments. The system consists of a notebook computer, microphone, flat-panel display, and custom software that responds to an infant's syllable-like productions with large, vibrant animations. visiBabble uses the Stevens Landmark Theory for acoustic-phonetic analysis, identifying landmarks — points in an utterance where listeners extract information about distinctive features through acoustically abrupt events. The system analyzes signals across multiple frequency bands calibrated for infant vocal-tract dimensions, detecting consonantal landmarks through simultaneous peaks in energy rate-of-change across bands. It then identifies syllables based on landmark sequences and timing constraints, requiring a voiced segment of sufficient length, and groups syllables into utterances based on inter-syllable gaps of no more than 200 milliseconds. The system responds to three types of vocal behavior: syllable production, pitch variation, and syllable or utterance complexity. Each session is digitally recorded, and detailed acoustic-phonetic data is saved for later analysis, including syllable types, durations, pitch contours, and utterance complexity metrics. visiBabble supports ABA single-case study formats for rigorous behavioral comparison across baseline and active phases.
Key findings
The 2005 demonstration showcased significant improvements over the initial version presented at Assets 2004. The team extended the system's feature detection capabilities, enabling more accurate identification of infant vocalizations versus environmental noise. The graphic response repertoire was expanded with larger, more varied animations designed to better engage infants. Reporting and analysis functionality was enhanced to provide richer data output, and sound feedback was added alongside the visual animations. The system's noise discrimination is particularly notable — it automatically ignores loud noise segments that lack voicing characteristics while retaining faint babble sounds that contain well-defined voicing and sufficient duration. This distinction is critical for real-world deployment where environmental noise is common. Field testing was planned at Northeastern University and the University of Nebraska-Lincoln, building on earlier pilot work with Professor Cynthia Cress. The project was funded by an NIH STTR grant (R42 DC005534), indicating both its clinical promise and its potential for commercialization as an assistive technology product.
Relevance
visiBabble represents an important intersection of speech technology and early intervention for children with disabilities. Research consistently shows that infant vocalizations predict later articulation and language abilities, yet traditional speech therapy struggles to provide sufficient practice and feedback for children with atypical speech patterns. By automating the reinforcement of syllabic utterances through engaging visual and auditory feedback, visiBabble addresses a real gap in pediatric assistive technology. The system's approach — using acoustic-phonetic analysis to identify meaningful vocalizations rather than simply responding to any sound — makes it a principled tool rather than a simple toy. For accessibility practitioners, this work highlights the importance of designing technology interventions that begin in infancy, long before conventional assistive communication tools become relevant. It also demonstrates how careful signal processing can make technology responsive to the specific communication abilities of users who produce non-standard speech.
Tags: early intervention · pre-speech vocalizations · speech technology · visual feedback · assistive technology · child development · AAC · acoustic analysis