Using Dynamic Audio Feedback to Support Peripersonal Reaching in Young Visually Impaired People
Graham Wilson, Stephen A. Brewster · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16) · doi:10.1145/2982142.2982160
Summary
This paper investigates whether dynamic audio feedback — sound that changes based on the proximity between a reaching hand and target objects — can improve the accuracy of peripersonal reaching (reaching within arm's length) for blind and visually impaired young people. Congenitally or early blind children develop motor skills more slowly than sighted peers and engage less with objects in their immediate environment, due to reduced awareness of object locations and slower development of object permanence. While sounding toys and sensory resource boxes have been used to encourage reaching, they provide only static audio cues. The research builds on the ABBI (Audio Bracelet for Blind Interaction) project, which uses a wrist-worn device to provide movement-based audio feedback. The apparatus used a Microsoft Kinect v2 depth camera for hand tracking, seven small KitSound speakers (six as targets placed on 10cm-high beakers, one worn on the wrist), and a Windows PC with USB soundcards. Three feedback designs were compared: a Geiger counter (increasing tempo of pluck tones as proximity increases), an increasing pitch design (ascending C major scale from C4 to C5 based on distance), and a constant unchanging tone. Each design was tested in both an Individual mode (sound from either object or wrist alone) and a Coalescent mode (alternating sounds from both object and wrist that merge as the hand approaches the target).
Key findings
Study 1 with six legally blind young adults (ages 18-22) found that both dynamic feedback designs (Geiger and Pitch) produced significantly fewer reaching errors (12.0% and 12.7% respectively) than the Constant design (24.1%, p < 0.001). The Control condition (no ongoing feedback, only an initial target cue) had the highest error rate (36.1%). The Pitch design was significantly faster than Geiger and Constant designs (3237ms vs. 4103ms and 3488ms). Subjective ratings showed dynamic designs created a stronger perceived "connection" between hand and object (6.0-6.3 out of 7) compared to Constant (4.7). Sound from the Object speaker alone resulted in the lowest error rates and fastest times, with Both speakers providing no additional accuracy benefit — participants found dual-source sound distracting rather than helpful. Study 2 with eight blind and visually impaired children (ages 12-17) showed higher overall error rates (30.3% vs. 17.7% for adults) but similar patterns across conditions. The Geiger counter produced the best accuracy for children (24.3% error) while Pitch was significantly worse (36.1%). Children preferred the Geiger counter (3 of 8) while adults preferred Pitch (3 of 6), possibly reflecting differences in musical ability and pitch interpretation. The Coalescent two-speaker design did not provide added benefit in either age group, contrary to hypotheses based on developmental psychology literature recommending redundant, interconnected information for blind children.
Relevance
This research has direct implications for early intervention and rehabilitation for blind and visually impaired children, demonstrating that dynamic, proximity-based audio feedback can meaningfully improve reaching accuracy and may encourage self-initiated environmental engagement. For accessibility practitioners and assistive technology developers, several design insights are notable: dynamic (changing) feedback significantly outperforms static sounds for guiding spatial action; sound emanating from the target object is more effective than sound from a wrist-worn device; adding redundant audio from multiple sources can actually be distracting rather than helpful; and age/musical ability influences which sonification design is most effective, suggesting that personalization is important. The finding that children showed less benefit from dynamic feedback than adults — while still showing the same general pattern — highlights that younger children may need different approaches, possibly involving simpler or more familiar sounds. The work contributes to the broader goal of using technology to support spatial cognitive development in blind children, addressing a critical developmental window where engagement with the physical environment shapes spatial understanding.
Tags: blindness · visual impairment · sonification · spatial cognition · child development · rehabilitation · haptic technology · auditory interface · sensory substitution