Exploratory Analysis of Collaborative Web Accessibility Improvement
Daisuke Sato, Hironobu Takagi, Masatomo Kobayashi, Shinya Kawanaka, Chieko Asakawa · 2010 · ACM Transactions on Accessible Computing · doi:10.1145/1857920.1857922
Summary
This paper presents a detailed analysis of the Social Accessibility Project, an experimental crowdsourced service from IBM Research-Tokyo that enables volunteers to fix web accessibility problems without requiring site owners to modify their pages. Launched in July 2008, the service uses collaborative metadata authoring: screen reader users report accessibility barriers they encounter, volunteer "supporters" create external metadata (alternative text, heading structure, landmarks), and client-side browser tools apply this metadata to web pages in real-time. The system architecture separates the concerns of problem identification (by users who experience barriers), solution creation (by volunteers with accessibility knowledge), and deployment (through browser extensions that fetch and apply metadata automatically). This approach sidesteps the traditional dependency on site owners to fix accessibility issues—a process that can take months or never happen at all. The metadata is stored in a public repository called Accessibility Commons, allowing fixes created for one user to benefit all users accessing the same pages. Over 20 months of operation, the researchers collected extensive data on user behavior, volunteer productivity, metadata quality, and system sustainability. The paper provides detailed case studies of successful collaborations (a hospital website fixed within 7 hours, a disability organization site with 82 accessibility fixes) and a technical failure case (an embedded map widget that couldn't be described with current tools), illustrating both the potential and limitations of the crowdsourcing approach.
Key findings
The service accumulated approximately 500 participants (150 end users, 350 supporters) who created 19,398 pieces of metadata for over 3,000 web pages based on 355 user requests. Volunteer productivity exceeded expectations: 83.4% of requests were resolved, with 43.2% solved within 24 hours. The most common metadata types were alternative text (66.7%) and heading structure (31.9%). No malicious metadata was detected, and supporter work quality was generally good. However, the study revealed a critical barrier: users often don't know what accessibility problems they're facing. The researchers termed this "they don't know what they don't know"—screen readers silently skip inaccessible content (images without alt text, opaque Flash movies), so users remain unaware of information they're missing. WebAIM survey data showed half of screen reader users "don't know how accessible these technologies are." This awareness gap is fundamental because approximately 70% of improvements in the system are triggered by user requests; if users can't articulate problems, the request-driven model breaks down. Metadata robustness presented another challenge. After 20 months, only 44.5% of metadata could still be applied successfully—web pages had changed, URLs had moved, and DOM structures had evolved. The visual "page map" authoring tool dramatically increased supporter participation when released, demonstrating that lowering technical barriers (eliminating need to write XPath expressions) enables broader volunteer involvement.
Relevance
This research provides empirical evidence for both the promise and pitfalls of crowdsourcing approaches to accessibility. The core insight—that users often can't report problems they're unaware of—has profound implications for any system that relies on user feedback to prioritize accessibility work. Automated accessibility checkers can detect technical violations, but matching technical issues to user-experienced barriers requires understanding what users actually encounter. For accessibility practitioners, the metadata types and request categories offer a data-driven view of what problems real screen reader users face most often: missing alt text and missing headings account for over 60% of requests. The case studies demonstrate that motivated volunteers can address substantial accessibility issues quickly (one supporter fixed 3,500 errors across 82 pages in one hour using site-wide metadata templates), suggesting that tooling and volunteer coordination may be more limiting factors than volunteer availability. The "collaborative accessibility improvement" model explored here has influenced subsequent work including browser-based accessibility overlays and community labeling initiatives. The fundamental tension the paper identifies—between user-driven request models and users' limited awareness of their own barriers—remains relevant as AI-powered accessibility tools emerge. Understanding what users experience (versus what automated tools detect) is essential for prioritizing accessibility work effectively.
Tags: crowdsourcing · web accessibility · social computing · metadata · screen readers · alternative text · collaborative accessibility · blind users · transcoding
Standards referenced: WAI-ARIA