A multi-method evaluation of university website accessibility: Foregrounding user-centred design, mining source code and using a quantitative metric
Tahani Alahmadi · 2017 · Proceedings of the 14th International Web for All Conference (W4A) · doi:10.1145/3058555.3058580
Summary
This extended abstract presents a multi-method model for evaluating the accessibility of university websites, with particular attention to the needs of students with sensory disabilities including deaf, visually impaired, and deafblind users. The research addresses a significant gap in how accessibility evaluation is typically conducted: most automated evaluation tools (AETs) and standards treat all web-based systems identically, failing to account for the specific priorities of educational platforms. For instance, the accessibility of embedded PowerPoint files and other learning materials is critical in university systems but receives low priority in general-purpose evaluation frameworks. The proposed model combines subjective and objective evaluation methods. The subjective component is grounded in user-centred design (UCD) theory, drawing on accessibility and usability statements derived from WCAG, SUMI (Software Usability Measurement Inventory), IBM Usability Evaluation guidelines, Section 508, and PDF/MS Office accessibility standards. The objective component employs automated tools, human evaluation, source code mining for media content analysis, and a quantitative metric called the A3 formula. The research was conducted at Griffith University in Australia and also involved Princess Nora University, suggesting a cross-institutional scope.
Key findings
The research identifies several critical limitations in current accessibility evaluation approaches for educational websites. Automated evaluation tools devote less attention to voice recognition functionality compared to screen reading features, and most do not flag inaccessible PDF, Word, and PowerPoint links as webpage errors — a significant oversight for university systems where these document types are central to learning. The proposed model introduces source code mining as a method for detecting accessibility-related behaviours in media content such as images, video, and audio files, going beyond what automated tools can detect. The study also maps relationships between usability and accessibility statements to create more comprehensive evaluation questionnaires. A key innovation is the quantitative metric grounded in the priority features of university systems and the specific characteristics of deaf, visually impaired, and deafblind students, allowing evaluations to be tailored rather than generic.
Relevance
This research highlights an important but often overlooked aspect of web accessibility: evaluation methods need to be context-sensitive. A university website serving students with sensory disabilities has fundamentally different accessibility priorities than a commercial site. The emphasis on embedded document accessibility (PDFs, PowerPoint, Word files) is particularly relevant for any organization that publishes learning materials online. The multi-method approach — combining automated testing, source code analysis, human evaluation, and user-centred design — provides a useful framework for organizations seeking more thorough accessibility assessments. The focus on deafblind users alongside deaf and visually impaired populations is noteworthy, as deafblindness is frequently underrepresented in accessibility research. However, as an extended abstract, the paper presents the proposed model without full results, limiting the ability to assess empirical validity.
Tags: web accessibility evaluation · higher education · user-centred design · web mining · assistive technologies · sensory disabilities · quantitative metrics
Standards referenced: WCAG 2.0 · Section 508