Evaluating Web Accessibility for Specific Mobile Devices
Markel Vigo, Amaia Aizpurua, Myriam Arrue, Julio Abascal · 2008 · Proceedings of the 2008 International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1368044.1368059
Summary
This paper addresses a fundamental limitation in early mobile web accessibility testing: the reliance on a single Default Delivery Context (DDC) that assumed all mobile devices shared the same baseline capabilities. The W3C mobileOK Basic tests, designed to verify compliance with Mobile Web Best Practices 1.0, evaluated web pages against a fixed device profile specifying 120-pixel screen width, XHTML Basic 1.1 support, JPEG/GIF image formats, 256 colors, CSS Level 1, and no scripting. The authors from the University of the Basque Country argued that this one-size-fits-all approach produced false positives for more capable devices (flagging errors that would not actually occur) and false negatives for less capable devices (missing errors that would occur). To solve this, they developed a tool that extends the mobileOK Basic tests to account for the actual hardware and software characteristics of specific mobile devices. The system retrieves device features from heterogeneous device description repositories — UAProf profiles (an RDF-based industry standard from the Open Mobile Alliance) and WURFL (an open-source community-maintained XML database of device capabilities). These features are merged into a CC/PP (Composite Capability/Preference Profiles) compliant profile, which then parameterizes the evaluation tests. The tool uses a Uniform Guidelines Language (UGL) to express test definitions with slots that get filled by device-specific values, and automatically generates XQuery sentences to evaluate XHTML documents against those tailored tests.
Key findings
The case study evaluated nine websites (including Google, YouTube, Flickr, Amazon, Gmail, Facebook, Yahoo, Twitter, and Wikipedia) against three devices with varying capabilities: a Nokia 3590 (below DDC), a Samsung SGH-E100 (similar to DDC), and a Sony Ericsson P990 (above DDC). Results demonstrated that device-tailored evaluation produced meaningfully different outcomes compared to DDC-only testing. For example, the more capable Sony Ericsson P990 showed fewer errors because it supported frames, JavaScript, and a wider range of image formats. The Nokia 3590, despite having fewer capabilities than the DDC, sometimes produced fewer warnings than the Samsung due to its CSS support. Image format support, CSS capability, and script support were identified as the primary factors driving evaluation differences across devices. The tool successfully reduced false positives by not flagging errors for capabilities that more advanced devices actually supported, and reduced false negatives by catching issues on legacy devices with fewer capabilities than assumed by the DDC. Mobile-optimized websites consistently produced far fewer accessibility issues than desktop-oriented sites viewed on mobile devices.
Relevance
Although mobile device capabilities have evolved dramatically since 2008, this paper establishes an important principle that remains relevant: accessibility evaluation must account for the actual delivery context rather than assuming a generic baseline. The core insight — that one-size-fits-all testing produces both false positives and false negatives — applies directly to modern responsive web design testing across smartphones, tablets, smart TVs, and wearables. The technical approach of parameterizing evaluation tests based on device profiles anticipated the responsive design era where content must adapt to vastly different screen sizes, input methods, and browser capabilities. The overlap between mobile usability barriers and disability-related accessibility barriers that the authors highlight (small displays paralleling low vision, limited keyboards paralleling motor disabilities, no images paralleling blindness) remains a foundational concept in inclusive design. For current practitioners, this work reinforces the importance of testing across multiple device contexts rather than relying solely on desktop-based automated checkers.
Tags: mobile accessibility · automated testing · evaluation tools · device independence · mobile web
Standards referenced: WCAG 1.0 · Mobile Web Best Practices 1.0 · mobileOK Basic Tests 1.0 · CC/PP · UAProf