Disability-Centered Evaluation

An approach to evaluating AI systems, tools, or research artefacts that places disabled people's lived experiences, information needs, and failure contexts at the centre of study design — including which data are collected, how ground truth is annotated, which models are tested, and which metrics decide success. In contrast to evaluations built on general-purpose benchmarks (e.g., ImageNet, MS COCO) that over-represent clean, non-disabled-produced inputs, disability-centered evaluation foregrounds realistic inputs (such as blurry or misframed photos taken by blind users), disability-relevant tasks (identifying a specific medication, not just "a bottle"), and criteria that map onto real consequences like safety, autonomy, and trust. The approach is closely related to disability-first dataset construction and participatory AI evaluation.