Subjective Evaluation of Website Accessibility and Usability: A Survey for People with Sensory Disabilities
Tahani Alahmadi, Steve Drew · 2017 · Proceedings of the 14th International Web for All Conference (W4A) · doi:10.1145/3058555.3058579
Summary
This paper presents a novel subjective evaluation model for assessing web accessibility and usability from the perspective of students with sensory disabilities, applied to Australian university websites. The model integrates accessibility criteria from WCAG 2.0 and Section 508 with usability criteria from SUMI (Software Usability Measurement Inventory) and IBM's computer usability satisfaction questionnaires. The key innovation is mapping the relationship between accessibility and usability by using WCAG 2.0's four principles (perceivable, operable, understandable, robust) as a bridge between the standards. The model identifies priority characteristics for each sensory disability group: for visually impaired users, the priority chain flows through image, video, voice, and text content with associated transmission and display needs (Braille, screen reader, voice); for deaf and hard of hearing users, video captions and sign language are prioritized; for deafblind users, Braille code is central. The evaluation questionnaire contains 66 multiple-choice questions and 2 open-ended questions organized into four categories: general system interaction, media file issues, document file issues, and assistive software issues. A five-phase process guides the model from preparation through classification, accessibility mapping, usability analysis, to final accessible-usable statement design.
Key findings
Early results from 67 visually impaired and deaf Australian university students and their assistants revealed significant accessibility barriers. 55% of participants believe the accessibility of their current website content negatively affects their study. In media content evaluation, 59% agreed that most non-textual elements lack descriptive text. Half disagreed that audio/video files have captions, subtitles, or sign language, while 20% were neutral and 20% agreed. 40% agreed that most captions, subtitles, or sign language provided are inaccurate. For document files, 58% agreed that navigation aids like bookmarks, headings, and tables of content are unavailable in most PDF files. 41% said captions for inserted audio/video in document files are not included. Regarding assistive software, 78% believe their university website does not consider screen reader issues. 69% said they cannot understand bulleted lists on educational pages. 70% believe web pages are not well structured for navigation by learners with sensory disabilities. For voice recognition users, 54% cannot interact effectively with controls using direct voice commands, and 54% said the website does not provide keyboard command instructions.
Relevance
This research is valuable for its user-centered approach to accessibility evaluation — rather than relying solely on automated tools or expert audits, it centers the actual experiences and perceptions of students with disabilities. The findings paint a concerning picture of Australian university website accessibility, particularly around media content (lacking alt text, captions, sign language), document accessibility (PDF files without navigation structure), and assistive technology compatibility (screen reader and voice recognition issues). For higher education institutions, the specific findings provide an actionable audit checklist. The model's integration of accessibility and usability evaluation is methodologically important, as these are often treated separately despite being deeply interconnected for disabled users. The heuristic priority mapping for different sensory disability groups helps institutions prioritize remediation efforts based on their student population. Limitations include the preliminary nature of the results (the paper presents early-stage data) and the focus on sensory disabilities to the exclusion of cognitive and motor disabilities. The survey instrument itself is a contribution that other institutions could adapt for their own evaluations.
Tags: accessibility evaluation · usability · university websites · education accessibility · sensory disabilities · visual impairment · Deaf and hard of hearing · deafblindness · screen readers · voice recognition · assistive technology · user-centered design · survey · Australia · online learning · PDF accessibility · document accessibility · LMS accessibility
Standards referenced: WCAG 2.0 · Section 508 · SUMI