What Is the Point of Fairness? Disability, AI and the Complexity of Justice
Cynthia L. Bennett, Os Keyes · 2020 · SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386301
Summary
This paper offers a critical disability studies challenge to the dominant "fairness" framing of AI ethics, arguing that fairness is insufficient and potentially harmful when applied to disability, and that justice must be centred instead. Drawing on Anna Lauren Hoffmann's critique, the authors identify four limitations of fairness: it aims to improve the status of disadvantaged groups without addressing the privileges of powerful ones; it relies on traits being well-defined when disability is fluid and contested; it historically addresses one identity axis at a time while people are multiply marginalized; and it frames marginalization only in relation to specific outcomes like employment rather than addressing systemic oppression. The paper presents two case studies from computer vision. The first examines AI systems that use facial recognition to diagnose autism in children by analysing facial expressions and repetitive behaviours. Through a fairness lens, the concern is that such systems replicate existing diagnostic biases around race, gender, and class — training data based on those already diagnosed (predominantly white, male, middle-class children) means the system may fail to identify autistic children from marginalized groups. But through a justice lens, deeper problems emerge: the system reinforces medical gatekeeping by institutionalizing psychiatrists' diagnostic authority through technology, it assumes early diagnosis is inherently beneficial when diagnosis carries harmful associations (financial penalties, behavioural conversion therapies, and as Mitzi Waltz writes, "autism = death" narratives), and it expands surveillance of neurodivergent children without questioning the power structures behind diagnostic categories.
Key findings
The second case study examines computer vision systems designed to help blind people "see" — object recognition, scene description, and facial recognition tools. A fairness analysis surfaces that these systems encode biases from training data developed in white, Western, middle-class contexts, failing to recognize objects common in non-Western environments. But a justice analysis goes further: computer vision for blind people cements vision as a superior sense and legitimizes surveillance technology under the guise of assistive benefit. The authors ask: how can technology to assist a blind person be kept separate from policing technologies? Who decides that the blind person won't become a subject of the same surveillance systems? The paper argues that computer vision, even when deployed fairly, shifts the centre of analysis and judgment away from the user and toward expensive, proprietary, black-boxed technology. The authors draw a sharp distinction between fairness (treating disabled sub-populations more equally within existing systems) and justice (questioning whether those systems should exist in their current form and who holds power through them). They call for integrating Disability Studies conversations about technology, justice, and power into AI ethics, and for centring multiply marginalized disabled people — not just otherwise-privileged disabled people — in these conversations.
Relevance
This is arguably the most theoretically challenging paper in our collection on AI and disability, and it serves as a necessary counterpoint to the more reform-oriented approaches of Guo et al., White, and Findlater et al. in the same SIGACCESS issue. Where those papers ask how to make AI fairer for disabled people, Bennett and Keyes ask whether fairness itself is the right goal. For accessibility practitioners, the paper's value lies in its insistence on asking uncomfortable structural questions: Does this technology reinforce medical gatekeeping? Does it expand surveillance? Does it benefit multiply marginalized disabled people or only those who are otherwise privileged? The autism diagnosis case study is particularly relevant as AI diagnostic tools proliferate — the authors demonstrate that "helping" through automated diagnosis can cause concrete harm when divorced from the social context in which diagnostic labels carry consequences. The call to move from fairness to justice aligns with the broader disability justice movement and challenges the technology sector to engage with power dynamics rather than treating bias as merely a technical problem to be optimized away.
Tags: AI fairness · disability justice · critical disability studies · computer vision · autism diagnosis · surveillance · medicalization · intersectionality · structural oppression