Evaluating Existing Audio CAPTCHAs and an Interface Optimized for Non-Visual Use
Jeffrey P. Bigham, Anna C. Cavender · 2009 · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2009) · doi:10.1145/1518701.1518983
Summary
This paper presents the first large-scale evaluation of audio CAPTCHAs — the purportedly accessible alternative to visual CAPTCHAs — revealing that they are dramatically more difficult for both blind and sighted users. The study gathered 10 types of audio CAPTCHAs from major websites (AOL, Google, Facebook, PayPal, reCAPTCHA, and others) and tested them with 162 participants (89 blind, 73 sighted) in a remote study. The research identifies a fundamental design problem: audio CAPTCHA interfaces are direct translations of visual interfaces rather than being designed for non-visual use. When solving a visual CAPTCHA, the image and answer box can be perceived simultaneously, but with audio CAPTCHAs, blind users must navigate between the play button and the answer box using a screen reader — a process that causes them to miss content, triggers the screen reader to talk over the CAPTCHA audio, and requires memorizing the entire sequence before typing. The paper then introduces an optimized interface that integrates playback controls directly into the answer text box using keyboard shortcuts: period (.) for play/pause, comma (,) for rewind one second, and forward slash (/) for fast-forward. This eliminates the need to navigate between separate controls, allowing users to listen, type, pause, review, and rewind all from within a single interface element.
Key findings
Blind participants solved only 43% of audio CAPTCHAs on the first try, while sighted participants solved 39% of audio CAPTCHAs but nearly 80% of visual CAPTCHAs on the first try. Blind participants took an average of 50.9 seconds per audio CAPTCHA — more than 5 times longer than sighted users took for visual CAPTCHAs (9.9 seconds) and more than twice as long as sighted users on audio CAPTCHAs (22.8 seconds, p < .0001). Second and third attempts rarely helped, with many CAPTCHAs remaining unsolved after three tries. The optimized interface increased blind participants' first-try success rate from 42.9% to 68.5% — a 59% improvement (F(1,100) = 22.3, p < .0001) — and reduced average attempts from 2.21 to 1.56 (p < .0001). Critically, the optimized interface achieves this improvement without changing the underlying CAPTCHA, meaning it does not affect security. The study also documented that over 2,600 web users had signed a petition asking Yahoo to provide an accessible CAPTCHA, and that none of fifteen blind high school students in an introductory programming class could solve the audio CAPTCHA required for the course.
Relevance
This research exposed a critical web accessibility barrier that affected millions of blind users' ability to create accounts, make purchases, and access online services. The paper articulates a broadly applicable accessible design principle: the most usable audio interfaces are often not direct translations of visual interfaces. This insight extends far beyond CAPTCHAs to any situation where developers add an audio or non-visual alternative by simply mapping visual interface patterns to audio. The finding that interface design — not just the CAPTCHA content itself — is a major barrier has practical implications for any developer implementing accessible alternatives. The optimized interface approach of integrating controls into the answer element, rather than requiring navigation between separate controls, is a design pattern applicable to many non-visual interaction scenarios. While CAPTCHAs have evolved significantly since 2009 (with invisible CAPTCHAs and other alternatives), the underlying research contribution about designing interfaces specifically for non-visual use, rather than adapting visual patterns, remains a fundamental principle of accessible design.
Tags: blind and low vision · web accessibility · screen readers · CAPTCHA · authentication · non-visual interfaces · usability testing · accessibility barriers