HEUA: A Heuristic Evaluation with Usability and Accessibility Requirements to Assess Web Systems
Ana Luiza Dias, Renata Pontin de Mattos Fortes, Paulo Cesar Masiero · 2014 · Proceedings of the 11th Web for All Conference (W4A) · doi:10.1145/2596695.2596706
Summary
This paper from the University of Sao Paulo presents HEUA (Heuristic Evaluation with Usability and Accessibility), a questionnaire-based instrument that integrates usability and accessibility assessment into a single evaluation framework for web systems. The authors argue that existing approaches evaluate usability and accessibility separately, making it difficult for practitioners to understand how these concerns interact, and that automated accessibility testing tools alone are insufficient since many checkpoints require human judgement. HEUA consists of 93 requirements drawn from three sources: Montero's interface design patterns for usability, all WCAG 2.0 success criteria for accessibility, and Lara's formalised requirements for older users. These 93 requirements are organised under Nielsen's 10 usability heuristics, a framework already well understood by HCI specialists. Each requirement includes an example, motivation, and source references to help evaluators apply it consistently. The evaluation produces a quantitative score (MHEUA) ranging from -100 to +100, where higher values indicate better combined usability and accessibility.
Key findings
The feasibility study compared two versions of a university scheduling system: the legacy Agendamento system and its redesigned successor AgendAloca. The older system scored MHEUA -21 (61% of applicable requirements not satisfied), while the redesigned system scored +45.5 (only 32% not satisfied). The per-heuristic breakdown revealed specific problem areas: Agendamento scored -100 on help documentation (Q10) and -43 on matching user mental models (Q2), while AgendAloca achieved 100 on system status feedback (Q1) and 75 on consistency (Q4). Both systems struggled most with error prevention (Q5), scoring -28 and -12 respectively. The evaluation took approximately four hours per system for a single specialist. The quantitative scoring system enables organisations to compare different systems or track improvements across versions, and the per-heuristic breakdown provides a roadmap for prioritising remediation efforts based on which areas have the most negative scores.
Relevance
HEUA addresses a practical gap that many web teams face: usability and accessibility are typically evaluated using different methods, different specialists, and different frameworks, making it hard to prioritise improvements holistically. By mapping WCAG 2.0 success criteria and older user requirements onto Nielsen's familiar heuristic framework, HEUA makes accessibility evaluation more approachable for HCI practitioners who already understand heuristic evaluation but may be less familiar with WCAG. The quantitative scoring provides something accessibility evaluations often lack — a single comparable metric that can track progress over time and justify investment in improvements. For practitioners managing legacy web systems, the per-heuristic scoring offers a structured way to identify where combined usability and accessibility effort should be focused first. The limitation is that the study involved only one evaluator and two systems, so broader validation is needed.
Tags: accessibility testing · usability testing · heuristic evaluation · evaluation methods · web accessibility · WCAG compliance · quality assurance
Standards referenced: WCAG 2.0 · WCAG 1.0