Evaluating Alternatives for Better Deaf Accessibility to Selected Web-Based Multimedia
Brent N. Shiver, Rosalee J. Wolfe · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2700648.2809857
Summary
This research addresses the accessibility gap created by the proliferation of uncaptioned video content online—a particular problem for deaf adults who use American Sign Language as their primary language and view English as a second language. While television captioning has matured through legislation and technology, online video largely remains inaccessible. The high cost of manual captioning ($9-30 per minute) makes comprehensive captioning economically unfeasible, prompting investigation into automatic speech recognition (ASR) as a lower-cost alternative. The researchers conducted two studies. The first was an exploratory interview study with 20 deaf and hard-of-hearing participants (19 profoundly deaf, mostly deaf from birth or early childhood) conducted via videophone by a deaf facilitator. This study identified Internet usage patterns and accessibility priorities. The second study with 95 deaf participants evaluated four captioning conditions: ASR captions with color-coded confidence visualization, ASR captions without visualization, no captions, and manually-created perfect captions. Participants viewed simulated news videos based on standardized 8th-grade reading tests (to control for prior knowledge) and answered comprehension questions.
Key findings
The exploratory study revealed that online news videos are the highest-priority accessibility concern for deaf Internet users—17 of 20 participants cited lack of captions on news clips as their top frustration. Participants strongly preferred captions over transcripts (19/20), finding transcripts harder to use because they require switching attention between video and separate text, while synchronized captions are "always in sync with the video." Participants reported daily frustration with inaccessible content, with one comparing the feeling to when "the Internet is down." The visualization study demonstrated that ASR-generated captions significantly improve comprehension compared to no captions, even with word error rates of 12-20%. When asked directly, 85 of 95 participants said they would choose ASR captions over no captions. The color-coded confidence visualization (highlighting uncertain words) showed no measurable benefit or detriment to comprehension—68 of 95 disagreed that color-coding helped understanding. However, when asked hypothetically whether they would want such visualization, 73 of 95 said yes, revealing a disconnect between stated preference and actual utility. Notably, performance on perfectly-captioned video was not significantly higher than ASR-captioned video, possibly due to participant fatigue from the lengthy study protocol.
Relevance
This research provides empirical support for a pragmatic accessibility position: imperfect ASR captions are substantially better than no captions at all. For organizations unable to afford professional captioning for all content, this finding justifies deploying automatic captions as a minimum accessibility baseline rather than leaving content entirely inaccessible. The overwhelming majority preference (89%) for ASR captions over nothing is a powerful data point for accessibility advocates and decision-makers. The study also highlights that news content should be prioritized for captioning investment given its importance to deaf users. The finding that transcripts are not an acceptable substitute for captions—despite being cheaper to produce—is valuable for content strategy decisions. However, the researchers note a legitimate concern: advocacy groups worry that organizations might use ASR captions to meet legal obligations when human captioning should be the standard. The research represents a snapshot of 2015 ASR technology; modern systems have substantially lower error rates, making the case for automatic captioning even stronger today.
Tags: deaf accessibility · captions · automatic speech recognition · ASR · multimedia accessibility · web accessibility · online video · news accessibility · American Sign Language
Standards referenced: WCAG · Section 508