Accessibility Challenges and Tool Features: An IBM Web Developer Perspective
Shari Trewin, Brian Cragun, Cal Swart, Jonathan Brezin, John Richards · 2010 · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806029
Summary
This highly cited (50 citations) survey of 49 IBM web developers explores the barriers they face in creating accessible rich internet applications and what features they value in accessibility testing tools. IBM mandates accessibility through Corporate Instruction 162, with a corporate accessibility team (Human Ability and Accessibility Center) that harmonises Section 508 and WCAG 2.0 into unified checklists, provides tools, training, and testing support. Despite this institutional support, developers still find accessibility challenging. The sample included 14 self-reported novices, 15 intermediates, and 20 experts in accessibility, with an average of 7.3 years of web development and 4.1 years of rich internet application experience. For testing tools, 69% used both automated tools and assistive technology, 16% used only assistive technology, 14% used only automated tools, and 8% used no tools at all. Primary information sources were IBM's CI162 checklists (68%), W3C web pages (35%), and Google searches (14%).
Key findings
The three most difficult aspects of accessible development were each cited by about a third of respondents: designing accessible solutions (10), using test tools to diagnose problems (10), and finding workarounds for toolkit/browser/third-party issues (9). Critically, testing and finding workarounds were also the most time-consuming aspects, while design was not — suggesting that once developers know what to do, the struggle shifts to verifying it works and dealing with platform inconsistencies. Developer quotes reveal frustration with assistive technology inconsistencies ("we don't even know if it is the browser, Flash, or it is the technology that we are using is having the problem"), ARIA support gaps ("ARIA doesn't seem fully supported yet, making it near impossible to know if its working right"), and tool limitations for dynamic content ("tools appear to be useful for simple HTML pages, but it's very difficult to diagnose the cause of problems if the HTML page is complex"). When asked about 15 potential tool features, all groups agreed that a checklist of automatically detected problems and an explanation of each problem were the most important. Experience level significantly affected feature valuations (Chi-Square=22.773, p=0.004): experts particularly valued the ability to visualise a site as someone with a disability would experience it, and the ability to pinpoint problems on the rendered page view — features that novices and intermediates considered less important, likely because they hadn't yet experienced the scenarios where these features prove valuable. 78% of respondents wanted to improve accessibility further but were limited by time (48%), technology limitations (30%), lack of knowledge (15%), budget (15%), and conflicts with other requirements (11%).
Relevance
With 50 citations and 1,250 downloads, this is one of the most influential studies on developer experience with accessibility tools. The findings remain remarkably current despite being from 2010. For tool developers, the key insight is that existing tools are "unclear, cumbersome, and incomplete with respect to the standards that must be met" — and that the gap is not in automated detection but in explanation, diagnosis, and actionable guidance. Developers don't just need to know something is wrong; they need to understand why specific code causes problems, see example solutions, and pinpoint errors in their rendered pages and source code. The finding that testing and workarounds consume more time than design contradicts the assumption that "designing for accessibility" is the hard part — in practice, making things work consistently across browsers and assistive technologies is harder. For organisations, the paper reinforces that even with strong institutional mandates, dedicated accessibility teams, and comprehensive tooling (IBM's level of accessibility infrastructure is exceptional), developers still struggle — highlighting that accessibility requires ongoing investment in tools, training, and time allocation, not just policy. The developer wish for tools that simulate assistive technology experience anticipates features now emerging in browser DevTools and design tools.
Tags: developer awareness · accessibility testing · evaluation tools · software development · accessibility training · developer experience · WCAG compliance · assistive technology
Standards referenced: WCAG 2.0 · Section 508 · WAI-ARIA