Hera-FFX: a Firefox add-on for semi-automatic web accessibility evaluation
José L. Fuertes, Ricardo González, Emmanuelle Gutiérrez, Loïc Martínez · 2009 · Proceedings of the 2009 International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1535654.1535661
Summary
This paper presents Hera-FFX, a Firefox browser add-on for semi-automatic web accessibility evaluation against WCAG 1.0, developed as a complete redesign of the earlier Hera online tool. The authors first establish a comprehensive list of 11 desirable features for accessibility evaluation tools — automatic preliminary evaluation, manual filling support, page presentation modification, annotated code view, local page evaluation, restricted-access page evaluation, rendered-page evaluation, report generation, training support, multi-session capacity, and flexibility to integrate other guidelines. They then survey 15 free evaluation tools and demonstrate that none covers all these features. Hera-FFX was designed to fill this gap by combining all features in a single browser-based tool. The architecture separates the accessibility specification from the tool itself through an XML configuration file that defines guidelines, checkpoints, and JavaScript-based automated tests, making the tool extensible to other standards. As a browser extension, Hera-FFX gains three key capabilities over online tools: it can evaluate local (unpublished) pages, restricted-access (password-protected/HTTPS) pages, and the rendered version of pages including dynamically generated content — all evaluated against the browser's DOM representation rather than the raw source code.
Key findings
Hera-FFX was the first tool to cover all 11 desirable features for web accessibility evaluation. The tool's three-phase evaluation process — automatic preliminary analysis, manual evaluation support, and report generation — addresses the reality that accessibility evaluation cannot be fully automated, as many WCAG 1.0 checkpoints require human judgment. The automatic analysis assigns each of the 65 WCAG 1.0 checkpoints a result of pass, fail, verify, or not applicable, with a priority ordering (Fail > Partial > Verify > Pass > Don't know > Not applicable) for aggregation. Three visualization services support manual inspection: a simulated page view highlighting elements requiring inspection, an annotated code view showing relevant HTML with color-coded results, and external service validation results. A preliminary evaluation with 6 expert users comparing Hera-FFX to Hera online found increased efficiency (automatic saving of manual evaluation results eliminated the need for explicit "record" actions), same effectiveness, increased user satisfaction, and slightly increased correctness due to evaluating rendered pages rather than source code. The tool's XML-based guideline definition and JavaScript test implementation meant both the guidelines and evaluation logic could be modified by users.
Relevance
This paper, together with its 2011 follow-up on WCAG 2.0 support, documents the evolution of accessibility evaluation tooling at a critical period. The 11-feature framework for evaluating accessibility tools remains a useful benchmark — modern tools can still be assessed against these criteria. The key insight that browser-based evaluation produces more accurate results than source-code-based evaluation (because it assesses what users actually experience, including dynamically generated content) anticipated a shift that is now standard practice. The distinction between automatic and manual evaluation phases reflects a reality that persists: automated tools can detect certain issues reliably but many accessibility requirements — whether alt text is meaningful, whether content is logically ordered, whether interactions are intuitive — fundamentally require human judgment. For practitioners selecting accessibility testing tools, the comprehensive feature comparison table provides a methodical approach to evaluating tool capabilities rather than relying on marketing claims. The tool's extensible XML architecture also demonstrated that accessibility evaluation tools should be adaptable to evolving standards rather than hardcoded to a single specification version.
Tags: accessibility evaluation · evaluation tools · semi-automatic evaluation · browser extensions · WCAG 1.0 · manual evaluation
Standards referenced: WCAG 1.0 · EARL