The Crowd Work Accessibility Problem
Saiganesh Swaminathan, Kotaro Hara, Jeffrey P. Bigham · 2017 · Proceedings of the 14th International Web for All Conference (W4A) · doi:10.1145/3058555.3058569
Summary
This paper examines a largely overlooked accessibility problem: the inaccessibility of crowdsourcing tasks themselves. While crowd work platforms like Amazon Mechanical Turk (AMT) offer potential employment benefits for people with disabilities — flexible scheduling, remote work, no commute, reduced need for in-person social interaction — the tasks posted on these platforms are overwhelmingly inaccessible. The researchers sampled 100 survey tasks from AMT and analyzed their compliance with WCAG 2.0 Level A using HTML CodeSniffer, an automated accessibility checker. They also manually examined a smaller sample of 6 image and audio transcription tasks. The study is framed against the stark disability employment gap: while 75.4% of people without disabilities were employed, only 34.4% of people with disabilities held jobs (2015 data). Crowdsourcing could help close this gap, but only if the work itself is accessible. The paper contextualizes this within both disability employment law (ADA Titles I and V) and web accessibility standards, noting that while public web content is mandated to be accessible under Section 508, crowdsourcing task interfaces created by individual requesters fall into a regulatory grey area.
Key findings
Only 2 out of 100 survey tasks (2%) complied with WCAG 2.0 Level A — the minimum accessibility standard. Eight of the 25 Level A guideline categories were violated. The most pervasive issue was Name, Role, Value (SC 4.1.2), affecting 95% of surveys, where form elements like dropdown menus lacked labels or ARIA attributes, making them unusable with screen readers. Info and Relationships (SC 1.3.1) violations appeared in 93% of surveys, with form labels not programmatically associated with their inputs. On Input (SC 3.2.2) violations affected 56% of surveys due to missing submit buttons, preventing keyboard-only form submission. Language of Page (SC 3.1.1) was missing in 34% of surveys, affecting screen reader pronunciation. Among the manually examined transcription tasks, additional problems emerged: 7 of 12 had dynamic content inaccessible to screen readers, 5 could not be completed by keyboard alone, and even tasks whose core content was accessible (like audio transcription for blind workers) often included inaccessible instructions or controls presented as images without alt text.
Relevance
This research raises an important and often invisible equity issue: as the economy shifts toward gig and crowd work, people with disabilities risk being excluded from these new employment opportunities not because they cannot do the work, but because the interfaces through which work is delivered are inaccessible. For accessibility practitioners and platform designers, the paper highlights that accessibility is not just about end-user-facing websites — it extends to tools and platforms used for work. The three proposed research directions remain relevant: encouraging requesters to create accessible tasks, building recommendation engines that filter tasks by accessibility, and developing automated tools to fix accessibility issues in task interfaces. The finding that even tasks whose core work is accessible often fail on interface accessibility underscores how pervasive and systemic the problem is — it is not about inherent task limitations but about careless interface design.
Tags: web accessibility evaluation · crowdsourcing · employment · disability employment · WCAG compliance · screen readers · keyboard accessibility
Standards referenced: WCAG 2.0 · Section 508 · ADA