Using Dynamic Audio Feedback to Support Peripersonal Reaching in Visually Impaired People
Graham Wilson, Stephen A. Brewster · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2700648.2811339
Summary
This demonstration paper explores how dynamic audio feedback can help blind and visually impaired people locate and reach for objects in their immediate (peripersonal) space. The research addresses an important developmental challenge: blind children engage less with objects in their environment than sighted children, partly due to reduced awareness of object locations and slower development of object permanence concepts. The authors hypothesize that providing real-time audio feedback about both the environment and the person's movements within it could encourage more self-initiated exploration and support spatial cognitive learning. The study tested three audio feedback designs with five blind and visually impaired young adults aged 18-19. The first design used a "Geiger counter" approach where pluck tones played with increasing temporal frequency as the hand approached the target object. The second used increasing pitch, moving up a C major scale as proximity increased. The third was a constant sound that played continuously regardless of hand position. Each design was tested with sound emanating from the target object alone, a wrist-worn speaker alone, or both combined. The system used a Microsoft Kinect v2 depth camera to track hand position relative to six small speaker targets arranged in peripersonal space.
Key findings
Dynamic audio feedback significantly improved reaching accuracy compared to constant or no feedback. The Geiger counter design achieved only 12.4% error rate compared to 27.7% for constant sound and 32.2% for the control condition with no feedback. The increasing pitch design performed similarly well at 13.1% error. Participants strongly agreed that dynamic designs created a perceptual connection between their hand and the target object (mean 6.2/7 on Likert scale), while constant sound scored notably lower (4.4/7). Three of five participants preferred the pitch-based design, finding the changes more obvious and easier to distinguish. One preferred the Geiger counter for being more "intuitive," noting that people without musical training might struggle with pitch-based feedback. Sound from the target object supported faster reaching than wrist-based sound, and combining both speakers provided no additional benefit over object-only feedback. All participants agreed that dynamic changes were necessary for accurate reaching—constant feedback was viewed as unhelpful.
Relevance
This research has implications for designing audio-based assistive technologies that support spatial awareness and motor development in blind individuals. The finding that dynamic, proximity-based audio feedback significantly outperforms static audio is relevant for developers creating navigation aids, educational tools, or therapeutic interventions for blind children. The preference split between pitch-based and rhythm-based (Geiger) feedback suggests that offering multiple audio modes could accommodate different user preferences and abilities. The study also validates the concept of using environmental sonification to encourage self-initiated exploration, which could inform the design of smart environments, audio games, or wearable devices for blind users. While this was a small demonstration study, it provides a foundation for larger-scale research on audio-guided reaching and spatial learning.
Tags: visual impairment · audio feedback · sonification · spatial awareness · child development · reaching · assistive technology