WebInSight: Making Web Images Accessible
Jeffrey P. Bigham, Ryan S. Kaminsky, Richard E. Ladner, Oscar M. Danielsson, Gordon L. Hempton · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169018
Summary
This paper introduces WebInSight, a system that automatically generates and inserts alternative text for web images that lack it. The authors first establish the scale of the problem through a series of web studies examining five groups of important websites: the 500 most high-traffic international sites, 158 computer science departments, the top 100 international universities, 137 U.S. federal agencies, and all 50 U.S. states plus D.C. The studies distinguished between significant images (those conveying content — multicolored, larger than 10 pixels, clickable, or linked) and insignificant images (decorative or structural). Results showed that only 39.6% of significant images on high-traffic sites had alternative text, rising to 82.5% for U.S. state websites. A separate traffic-weighted study at the University of Washington CSE department over one week captured nearly 12 million images, finding 63.2% of the 4.9 million significant images had alternative text. WebInSight addresses these gaps through a transformation proxy architecture with three labeling modules: web context labeling (extracting text from the title and h1 tags of pages linked to by images), enhanced OCR with color segmentation preprocessing, and human labeling for images that automated methods cannot handle. The system caches labels in a database keyed by MD5 hash of the image, enabling reuse across pages and sessions.
Key findings
The web studies revealed significant variation in alt text compliance across website categories: U.S. federal agencies (74.8% of significant images labeled) and state sites (82.5%) performed best, likely due to Section 508 requirements, while high-traffic commercial sites performed worst (39.6%). A common error was assigning no alternative text at all to insignificant images rather than the recommended zero-length alt attribute — only 7.3% of insignificant images were correctly marked. When evaluated on 2,500 randomly selected unlabeled significant images, WebInSight automatically provided alternative text for 1,079 (43.2%), with 94.1% accuracy as verified by manual review. Context-based labeling proved particularly reliable for linked images because the title and h1 tags of destination pages often accurately describe the image. The OCR module, enhanced with a color segmentation preprocessing step that creates black-and-white highlight images for each dominant color, achieved 65% accuracy on a test set of 100 multicolored web images (compared to 52% without segmentation). OCR output was validated through a multi-tiered verification pipeline using dictionary lookup, Google spell-check API, and web search. The system successfully recovered 52 of 65 email addresses rendered as images on a university faculty page.
Relevance
WebInSight was a pioneering system in automated alternative text generation, predating modern AI-based image description by nearly two decades. Its core insight — that useful alternative text can often be derived from surrounding web context rather than from the image content itself — remains relevant today. The web studies provide valuable baseline data on alt text compliance across different website sectors, demonstrating the positive impact of legal requirements like Section 508 on accessibility outcomes. The distinction between significant and insignificant images, and the argument that each requires different alt text treatment, is a practical framework that still applies. For modern practitioners, the paper illustrates that automated approaches need not be perfect to be valuable — providing correct alt text for 43% of unlabeled images with 94% accuracy meaningfully improves the browsing experience. The human labeling module anticipated later crowdsourced accessibility efforts. The challenges identified — copyright concerns, misuse potential, quality control — are the same ones facing today's AI-generated image descriptions.
Tags: web accessibility · alternative text · image accessibility · OCR · optical character recognition · transformation proxy · blind users · screen readers · automated labeling · crowdsourcing
Standards referenced: WCAG · Section 508