Misfitting With AI: How Blind People Verify and Contest AI Errors
Rahaf Alharbi, Pa Lor, Jaylin Herskovitz, Sarita Schoenebeck, Robin N. Brewer · 2024 · ASSETS '24: Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675659
Summary
This paper presents an in-depth qualitative study with 26 blind participants examining how they encounter, verify, and contest errors produced by AI-enabled visual assistance technologies (AI VAT) such as Seeing AI, Be My Eyes, and ChatGPT. While blind people increasingly rely on these AI tools to gain visual access in daily life — reading documents, identifying objects, navigating environments — the technologies are embedded with errors that are inherently difficult to verify without sight. The study used semi-structured interviews to explore three dimensions: what types of errors blind people encounter, how they make sense of and verify AI output, and what they want from explainable AI (XAI) features. The researchers frame their analysis through Rosemarie Garland-Thomson's disability studies concept of "misfitting" — the idea that disability emerges from a mismatch between a body and its environment, rather than from the body itself. Applied to AI, misfitting describes the gap between what AI VAT assumes about its users (sightedness, English literacy, Western cultural context) and the actual diverse needs of blind users. The study recruited participants across a range of ages, races, education levels, and technology experience, with particular attention to including non-English speakers and people from diverse cultural backgrounds — populations often excluded from AI accessibility research.
Key findings
Participants reported errors across several categories: AI VAT frequently failed with complex document layouts (tables, multi-column formats, forms), producing scrambled or incomplete text; it struggled with non-English languages, particularly Arabic, which involves right-to-left text and diacritical marks; and it misidentified cultural artifacts and foods from non-Western cultures due to training data bias. Verification strategies fell into four main approaches: experimenting with AI VAT (trying different apps, angles, lighting, or prompts to see if results changed), employing non-visual skills (using touch, smell, sound, or prior knowledge to cross-check AI output), strategically involving sighted people (calling family members or using human-powered services like Be My AI), and cross-referencing with other devices (comparing output from multiple AI tools). Participants expressed strong desire for contestation affordances — the ability to flag errors, provide corrections, and have their feedback actually incorporated into AI improvement. They wanted confidence scores, source attribution, and the ability to ask follow-up questions about why AI produced specific output. The misfitting framework revealed that AI VAT design embeds assumptions of sightedness (visual XAI features like heatmaps), English dominance, and Western cultural norms, systematically disadvantaging blind users from marginalised backgrounds.
Relevance
This research makes critical contributions to responsible AI practice by centring the experiences of blind users — a population that is simultaneously among the most dependent on AI for daily access and most vulnerable to AI errors they cannot independently verify. The misfitting framework provides accessibility practitioners with a powerful lens: rather than asking how to make AI work for blind people, it asks what assumptions in AI design create barriers. The documented verification strategies demonstrate that blind people are not passive recipients of AI output but active, sophisticated evaluators who develop creative workarounds — a finding that should inform XAI design to support rather than replace these existing competencies. The intersectional findings around language and cultural bias are particularly important, revealing that AI accessibility failures compound for blind people who are also linguistic or cultural minorities. For the broader AI field, the call for contestation affordances — mechanisms for users to challenge and correct AI output — represents a concrete step toward more accountable AI systems that learn from disabled users' expertise rather than treating them solely as beneficiaries.
Tags: blind users · artificial intelligence · visual assistance technology · explainable AI · AI errors · verification · responsible AI · disability studies · misfitting