VisAural: A Wearable Sound-Localisation Device for People with Impaired Hearing
Benjamin M. Gorman · 2014 · ASSETS '14: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2661334.2661410
Summary
This paper presents VisAural, a wearable device that converts sound direction information into visual cues for people with hearing impairments. The system addresses a gap left by conventional hearing aids, which help with general sound detection but do not preserve spatial information needed for sound localisation. VisAural uses an array of four microphones mounted on a pair of eyeglasses to detect sounds in the environment. Using a delay-and-sum beamforming algorithm, the system computes the direction a sound arrives from by measuring delays between microphones and a reference point. LEDs fixed to the left and right of the eyeglass frame light up to indicate which direction a sound is coming from, placing visual cues at the periphery of the user's visual field. The system checks for sounds at 15-degree increments from 15 to 90 degrees on either side.
Key findings
Testing with nine participants who had hearing impairments showed that the approach holds promise but needs improvement. Participants were asked to locate a sound source in a room while wearing the device. Most participants found the concept valuable and could understand the directional cues. However, the system's 0.2-second sampling buffer created responsiveness issues—participants sometimes moved faster than the system could update, leading to confusion. The study identified that reducing latency and improving the refresh rate of directional feedback are critical for real-world usability. The work also found that vision is a suitable alternative sensory channel for sound localisation, as both hearing and vision enable spatial understanding of the environment.
Relevance
This research explores sensory substitution as an accessibility strategy—converting auditory information into visual information for people with hearing loss. With approximately 17% of the UK population having some form of hearing impairment and hearing aid uptake at only 14%, alternative approaches to environmental sound awareness are important. The wearable form factor (eyeglasses) is practical and socially acceptable, which is significant for adoption. For accessibility practitioners, this work highlights the importance of latency in assistive devices and the need for user testing with the target population early in the design process. While the prototype needs refinement, the concept of peripheral visual cues for sound direction has potential applications in safety-critical situations where sound awareness matters.
Tags: hearing impairments · wearable technology · sound localisation · sensory substitution · assistive technology