Analysis and integration of web accessibility metrics
Maia Naftali · 2010 · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1805996
Summary
This Google Student Award paper presents OceanAcc, a semi-automatic accessibility evaluation and analysis tool developed as a thesis project at the University of Buenos Aires. OceanAcc addresses a practical gap: while several accessibility metrics had been published in research papers (UWEM, Failure Rate, WAB Score), most common accessibility validators did not implement them, leaving practitioners without quantitative measurement capabilities. OceanAcc integrates the AChecker validator with these established metrics and Giorgio Brajnik's barrier-checkpoint mapping from the Barrier Walkthrough method. The tool generates multiple analytical reports including metric evolution of a page over time, most violated checkpoints, a barrier ranking, and the percentage of users with different disabilities affected by the page's accessibility issues. These reports are designed to optimize subsequent manual testing by directing human testers to the most impactful problems first.
Key findings
The tool demonstrates that semi-automatic evaluation — combining automated checking with minimal user intervention to filter false positives and context-dependent results — can produce meaningful quantitative accessibility data. The author acknowledges that fully automatic assessment is inherently insufficient, as elements like tables can be flagged as failures by automated tools while actually being accessible in context. User intervention is specifically needed to filter these false positives. The integration of multiple metrics into a single tool allows evaluators to view accessibility from different analytical perspectives simultaneously, rather than relying on a single score. The barrier ranking and disability-impact reports provide actionable prioritization for remediation work, making accessibility testing more cost-effective by focusing limited manual testing resources on the highest-impact issues. The tool's goal of positioning accessibility as a measurable quality attribute reflects an important strategy for encouraging organizational adoption.
Relevance
OceanAcc represents an early effort to bridge the gap between accessibility research metrics and practical tooling — a gap that persists in some form today. The core idea of integrating multiple metrics into a single dashboard that provides different analytical views of accessibility remains valuable, particularly for organizations trying to track accessibility improvement over time. The disability-impact reporting feature — showing what percentage of users with different disabilities are affected — anticipated the user-impact-focused approach to accessibility that has gained traction in recent years. For practitioners, the tool's design reinforces that automated testing should be seen as a starting point for manual review rather than a complete solution, and that smart prioritization of manual testing effort is essential for cost-effective accessibility work. The project also demonstrates how existing tools (like AChecker's API) can be composed into more sophisticated evaluation workflows.
Tags: accessibility metrics · accessibility evaluation · automated testing · barrier walkthrough · WCAG compliance · accessibility tools
Standards referenced: WCAG · UWEM