Towards Optimised Population Sourcing for Web Accessibility Evaluation
Alexander Hambley · 2023 · Proceedings of the 20th International Web for All Conference (W4A '23) · doi:10.1145/3587281.3587701
Summary
This doctoral consortium extended abstract presents a PhD project proposing a novel framework and prototypical tool for optimising web accessibility evaluation through statistically representative page sampling. The work is the single-author companion to the multi-author extended abstract "OPTIMAL-EM" (doi: 10.1145/3587281.3587962) presented at the same conference. The framework addresses the problem that web accessibility evaluations — whether automated or manual — require selecting pages from potentially thousands on a website, yet current practice relies on ad-hoc, heuristic-based page selection. The proposed framework uses six metrics divided into three quantitative (coverage, complexity, accessibility) and three qualitative (representativeness, popularity, freshness) measures, adapted from census studies, web crawling research, accessibility metrics, the Unified Web Evaluation Methodology (UWEM), and WCAG-EM. The framework exploits modern templated web development processes, recognising that once pages are clustered by structural similarity, a sample drawn from a cluster represents the entire cluster because accessibility issues in shared templates will be shared across pages using that template.
Key findings
The framework is implemented as a three-stage tool: stage one crawls and downloads pages, processes them, and performs preliminary clustering; stage two calculates population complexity; and stage three determines sample size and produces a final report identifying representative pages for auditors to review. The researcher evaluated several population sourcing methods including server log files, web crawling, and the Common Crawl dataset, each with trade-offs. An evaluation on Studentnet (University of Manchester) found that a large proportion of crawled pages — using both depth-first and breadth-first approaches — were not actually accessed by users, highlighting the gap between crawlable pages and pages that matter for accessibility. The framework is designed to significantly reduce the human cost of accessibility evaluation while reducing the risk of discrepant or unrepresentative results that arise from ad-hoc page selection.
Relevance
This doctoral consortium paper provides additional context on the motivations and theoretical foundations of the OPTIMAL-EM framework. The explicit connection to census methodology and the distinction between quantitative and qualitative metrics adds conceptual clarity. The key insight that templated web development creates natural clusters of structurally similar pages — where fixing an accessibility issue in the template fixes it across all pages using that template — is a practical observation that accessibility teams can apply even without the full tooling. For organizations conducting WCAG-EM conformance evaluations, this work offers a principled alternative to the ad-hoc page selection that the methodology calls for but provides limited guidance on implementing. As a doctoral consortium paper, it represents an early-stage research vision whose full validation remains as future work.
Tags: automated accessibility testing · web accessibility · accessibility evaluation · web crawling · machine learning · WCAG compliance · accessibility standards · large-scale analysis
Standards referenced: WCAG · WCAG-EM