A Web Accessibility Report Card for Top International University Web Sites
Shaun K. Kane, Jessie A. Shulman, Timothy J. Shockley, Richard E. Ladner · 2007 · Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1243441.1243472
Summary
This paper from the University of Washington presents a comprehensive multi-method accessibility evaluation of the home pages of the top 100 international universities as ranked by the Times Higher Education Supplement 2006 World University Rankings. The study combined four automated evaluation tools — WebXACT/Bobby, Cynthia Says, the Functional Accessibility Evaluator (FAE), and WebInSight — with manual analysis of accessibility policies, alternate-language content, and text-only versions. The universities spanned 21 countries, with 48 located in the US or UK. The automated tools measured WCAG 1.0 Priority 1, 2, and 3 violations, while FAE evaluated five functional categories (navigation and orientation, text equivalents, scripting and automation, styling, and HTML standards). WebInSight was used specifically to classify images as significant or insignificant (decorative), providing a more nuanced assessment of alt text quality than simple presence/absence checks. Manual evaluation covered accessibility policy quality, searching for statements through webmaster contact, homepage links, and site search. Data collection occurred in January 2007. The authors noted that prior studies had shown fewer than 25% of university home pages met minimum accessibility criteria, and that some existing sites were actually becoming less accessible over time.
Key findings
The 100 university pages contained a total of 937 accessibility errors, with a mean of 4.68 errors per page. Only 36 pages had no Priority 1 errors, and just 2 universities (Michigan and Queensland) were completely free of all Priority 1-3 errors. Geographic differences were statistically significant (p < .05): Australian universities performed best (2.71 average P1-P3 errors), followed by UK (3.17), US (4.61), Netherlands and France (5.5 each), and Switzerland (5.6). By continent, Oceania led (3.28), followed by Europe (4.45), North America (4.81), and Asia (5.93). There was no significant difference between US public and private universities, and no correlation between university ranking and accessibility. The FAE analysis revealed navigation and orientation as the weakest area (36.07% score, rated "Not Implemented"), with all other categories rated "Partially Implemented" (51-70%). For images, 35% of 3,155 total images had alt text, but when filtering to significant images only (42% of total), 71% had alt text — suggesting universities did better with meaningful images than raw numbers indicate. Only 46 of 100 sites had findable accessibility policies; of these, only 14 stated the policy was required or current, while 20 were drafts or in progress. Sites with accessibility policies had significantly fewer violations (3.98) than those without (5.36, p < .01). Non-English pages of Asian-language sites showed significantly more accessibility errors than their English counterparts (p < .05). Only 14 sites offered text-only versions, and only 3 North American sites featured alternate language links.
Relevance
This study established an important baseline for university web accessibility that has been widely cited (63 citations). Its multi-method approach — combining multiple automated tools with manual evaluation — set a methodological standard for accessibility audits by demonstrating that no single tool captures the full picture. The finding that automated tools correlate well with each other (r² = 0.94 for Bobby and Cynthia) but still produce different results justifies using multiple tools. The geographic disparities are striking and raise questions about the role of legislation, culture, and resources in driving accessibility: UK universities, subject to SENDA legislation, performed well, while legislation alone did not explain all differences. The correlation between having an accessibility policy and having fewer violations (even if causation is unclear) provides ammunition for organizational accessibility advocates. The image accessibility analysis using WebInSight's significant/insignificant classifier was methodologically innovative, showing that raw alt text statistics can be misleading. For current practitioners, the core finding that even top-ranked, well-resourced universities struggle with basic accessibility remains depressingly relevant, and the paper's framework of combining automated testing, policy analysis, and manual evaluation provides a replicable audit methodology.
Tags: web accessibility · education · accessibility testing · WCAG compliance · automated testing · accessibility policy · global accessibility · higher education
Standards referenced: WCAG 1.0 · Section 508