Exploring the Impacts of Background Noise on Auditory Stimuli of Audio-Visual eHMIs for Hearing, Deaf, and Hard-of-Hearing People
Wenge Xu, Foroogh Hajiseyedjavadi, Debargha Dey, Tram Thi Minh Tran, Mark Colley · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26) · doi:10.1145/3772318.3791557
Summary
This CHI 2026 paper investigates a critical but overlooked accessibility question in the design of external Human-Machine Interfaces (eHMIs) for automated vehicles (AVs): how do Deaf and Hard-of-Hearing (DHH) pedestrians experience audio-visual eHMIs, and how does real-world urban background noise affect the auditory components of these interfaces? Prior eHMI research has largely excluded disabled pedestrians (an estimated 430 million DHH people globally) and has typically tested systems in idealized soundscapes that do not reflect noisy streets, construction sites, or busy markets where AV-pedestrian communication must actually function. The authors built a Unity-based VR simulation of an urban mid-block crossing without traffic signals, featuring 13 AVs driving at 40-50 km/h on a two-lane road. Each AV was fitted with a standardized audio-visual eHMI: a pulsing cyan light strip, a text display showing "STOPPING"/"STOPPED", and one of three auditory stimuli conditions — Baseline (only the vehicle's Acoustic Vehicle Alerting System and tire-surface sound), Bell (non-speech), or Speech ("I'm stopping"/"I'm stopped"). Two background noise levels were tested: Quiet (~58 dB of ambient urban ambience) and Loud (~70 dB with construction machinery and dense market chatter). The study used a 2x3 within-subjects design with 36 participants — 25 Hearing and 11 DHH (hearing loss ranging from moderate to profound across one or both ears, all using hearing aids or cochlear implants during the session; three BSL users had a sign language interpreter present). Sessions used a Varjo XR-4 headset with built-in eye tracking and a Cyberith Virtualizer Elite 2 treadmill for walking. Outcomes included trust, acceptance, perceived safety, mental workload (NASA-TLX), eye-gaze dwell on eHMI components, step-into-road time, early-step counts, and thematic analysis of semi-structured interviews.
Key findings
Three main results emerged. (1) DHH crossing experiences differ significantly from Hearing pedestrians: DHH participants showed lower and more variable ratings across trust, usefulness, satisfying, and perceived safety, and a substantial gap in auditory access — all 25 Hearing participants could perceive at least one auditory eHMI under both noise conditions, versus only 7 of 11 DHH participants and only 2 DHH who could perceive both auditory eHMIs under both noise levels. Crucially, hearing technology mediated access unevenly: one participant's hearing aid filtered the Bell as background noise, and another only heard "Stopped" from "I'm stopped." (2) Loud background noise significantly impaired crossing experience (trust, usefulness, safety) and raised mental workload, but did not change objective crossing behavior (step-into-road time, early-step counts, gaze dwell). (3) Adding an auditory eHMI (Bell or Speech) improved experience ratings over the Baseline, with Speech ranked first overall (19/36 ranked Speech first) and Bell second; no significant experience or behavior differences between Bell and Speech emerged. Qualitative findings flagged that some DHH participants found high-pitched robotic voices hard to hear and suggested deeper voices or alternative modalities (e.g., haptics) for those unable to access auditory eHMIs; five Hearing participants linked Bell to tram or pedestrian-crossing signals while 15 others found its meaning unclear ("sounds like a church bell"). Four practical implications: involve disabled pedestrians in eHMI research; use background noise in future testing to match realistic soundscapes; enable audio-visual (multi-modal) eHMI so one channel compensates when another is masked; and collaborate with hearing-technology manufacturers to design eHMI signals that pass through (rather than being filtered by) hearing aids and cochlear implants.
Relevance
For accessibility practitioners, this paper is one of the first rigorous empirical investigations of how AV-pedestrian communication fails DHH users in realistic urban soundscapes. Its findings directly challenge an implicit assumption in the eHMI literature — that auditory cues are a universally accessible add-on — by demonstrating that hearing aids and cochlear implants may filter or distort eHMI signals in ways designers have not anticipated. The call to work with hearing-technology manufacturers on "hearing-technology-friendly auditory eHMI" is a concrete cross-industry action item. The paper also contributes methodologically, showing how VR simulation with varied realistic background noise can surface accessibility failures that idealized lab studies mask. Limitations include a modest DHH sample (N=11) with uneven group sizes, a single non-signalized crossing scenario, a single country (UK), and no low-vision or blind participants. A closely related companion paper by the same authors (Xu et al. 2026, DOI 10.1145/3772318.3790738) explores the visual-only eHMI question for DHH pedestrians. Together these works open an important inclusive-mobility research thread as AVs move toward deployment.
Tags: deaf and hard of hearing · external human-machine interface · automated vehicles · pedestrian safety · multimodal interaction · accessibility · virtual reality · auditory icons
Standards referenced: ISO/TR 23049:2018