Programmer-Focused Website Accessibility Evaluations
Chris Law, Julie Jacko, Paula Edwards · 2005 · Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '05) · doi:10.1145/1090785.1090792
Summary
This paper argues that accessibility evaluation and reporting processes have historically focused on the needs of end-users with disabilities while overlooking the actual audience for evaluation reports: the programmers who must implement the fixes. The authors observe that the W3C's preliminary evaluation process and tools like Bobby and WAVE generate voluminous reports containing hundreds of pages of line-item fixes that programmers simply do not have the capacity or inclination to process. Programmers face competing demands — security fixes, database work, design requests, content updates — and accessibility fixes must compete for their limited time. The paper also highlights organisational dynamics: management often delegates both authority and responsibility for accessibility to a specialist, which is counterproductive because programmers have the real control over code changes, while managers control priorities and resources. The authors propose SERPA (Streamlined Evaluation and Reporting Process for Accessibility), a seven-step programmer-centric methodology that restructures how accessibility evaluations are conducted and communicated to increase the likelihood that fixes are actually implemented.
Key findings
SERPA's seven steps reframe accessibility evaluation around programmer workflows: (1) discuss project goals with programmers and team members first to understand constraints; (2) establish an agreed scope and conformance level rather than attempting to fix everything at once; (3) prepare a programmer-centric report template that leads with site-specific fixes rather than generic guidelines; (4) evaluate the website using visually capable methods first (screen magnification, font resizing, colour/contrast checks) before using screen readers and automated tools; (5) report progress using the Accessibility Progress Formula (APPF = A/(A+B)+(T-C)), which tracks the ratio of applicable items addressed; (6) hand over only what is necessary — concise reports with pragmatic recommendations rather than exhaustive lists; and (7) follow up after programmer changes to verify fixes and address new issues. The paper emphasises practical advice: list specific fixes first and attach guideline checklists as appendices; separate content problems from presentation problems since different people may be responsible; present evaluation results in a form programmers can scan (not lengthy prose); and distinguish between easy fixes and more difficult ones so programmers can show quick progress.
Relevance
This paper identified a problem that persists two decades later: the gap between accessibility knowledge and implementation often lies not in what needs to be fixed but in how fixes are communicated to developers. The programmer-centric perspective remains highly relevant as organisations adopt accessibility practices. Many modern accessibility platforms still generate overwhelming reports that developers struggle to action. SERPA's core insights — scope the work incrementally, understand developer constraints, lead with specific actionable fixes rather than abstract guidelines, and track progress quantitatively — anticipate best practices now being adopted in accessibility maturity models and agile accessibility workflows. The paper's recognition that organisational dynamics (who has authority, who has responsibility, who has capacity) determine accessibility outcomes more than technical knowledge alone is a lesson many organisations are still learning.
Tags: accessibility evaluation · web accessibility · accessibility reporting · developer tools · organizational accessibility · WCAG · automated testing
Standards referenced: WCAG 1.0