WatchOut: Obstacle Sonification for People with Visual Impairment or Blindness
Giorgio Presti, Dragan Ahmetovic, Mattia Ducci, Cristian Bernareggi, Luca Ludovico, Adriano Baratè, Federico Avanzini, Sergio Mascetti · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019) · doi:10.1145/3308561.3353779
Summary
This paper presents WatchOut, a sonification technique that conveys real-time information about obstacle characteristics to people who are blind or visually impaired (BVI) through non-verbal sound, enabling them to avoid obstacles while walking. The white cane, the primary obstacle detection tool, is limited to approximately 1 metre range, cannot detect elevated obstacles (such as overhanging branches), and should not be used to hit certain obstacles like standing people. WatchOut addresses these limitations by using computer vision (Apple’s ARKit 2, running on an iPhone 7+ worn on the chest) to detect obstacles at distances up to 3 metres and encoding four obstacle characteristics into distinct auditory dimensions: distance is mapped to sound repetition rate (mimicking parking sensors — faster beeping for closer obstacles), horizontal position (left/centre/right) is mapped to stereo panning, width (small/large) is mapped to polyphony (single tone versus three-note chord cluster), and height (walkable/circumventable) is mapped to base sound pitch (low for step-over, high for go-around). The base sound is a synthetic sine wave with a percussive impulse, deliberately chosen to be clearly distinguishable from environmental sounds. The system was designed through two iterative cycles of online questionnaires with BVI participants (22 in the first iteration, 9 in the second), each informing design improvements before a real-world evaluation.
Key findings
The iterative design process significantly improved sonification accuracy. In the first questionnaire, participants could accurately identify distance (0.92) and position (0.92) from sound alone but struggled with height (0.65) and width (0.64). For the second iteration, width was redesigned from a violin-versus-double-bass timbre mapping to a single-pitch-versus-chord mapping, and height was changed from a filter cut-off frequency to base pitch — both leveraging more cognitively robust ecological metaphors. This improved global accuracy from 0.34 to 0.53, with height rising to 0.70 and width to 0.74, while distance (0.99) and position (0.99) reached near-perfect scores. Importantly, the improved mappings for height and width also improved accuracy for the unchanged distance and position parameters, suggesting the redesigned sonification was globally more intuitive and less cognitively demanding. A real-world evaluation with 13 BVI participants walking outdoor paths with unknown obstacles showed the system effectively guided users to avoid 85% of obstacles (precision 0.82, recall 0.87). The system received a SUS usability score of 72.5 ("good"), was rated 4.25/5 for usefulness and 4.25/5 for effectiveness. Blind participants reported they would use the system more frequently than low vision participants, which the authors attribute to blind users having greater need since low vision users can rely on residual sight. Seven participants requested adding vibration as an additional feedback modality, and five wanted verbal cues naming specific obstacles.
Relevance
WatchOut makes a significant contribution to the assistive navigation field by focusing on the often-overlooked problem of how to convey obstacle information, rather than just how to detect obstacles. Most prior obstacle detection systems for BVI users either provide only binary alerts (obstacle present/absent) or use speech messages that are slow and distracting during walking. WatchOut’s approach of simultaneously encoding four obstacle properties into a single synthesised sound — enabling users to understand at a glance whether to step over, go around left, or go around right — is novel and practically valuable. The use of bone-conducting headphones is a critical design choice, keeping ear canals open so users can hear ambient environmental sounds (traffic, people, echoes) that are essential for safe mobility. The iterative, user-centred design methodology demonstrates the value of involving BVI participants not just in evaluation but in refining the sonification mappings themselves. For accessibility practitioners, the specific mapping choices (parking-sensor metaphor for distance, stereo panning for position, chord clusters for width, pitch for height) provide a reusable vocabulary for obstacle sonification design.
Tags: sonification · obstacle avoidance · blind · low vision · computer vision · orientation and mobility · ARKit · mobile accessibility · auditory display · navigation