Accept or Address? Researchers' Perspectives on Response Bias in Accessibility Research
Joy Ming, Sharon Heung, Shiri Azenkot, Aditya Vashistha · 2021 · ASSETS '21: The 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3471216
Summary
Response bias — the tendency for participant responses to be skewed by factors beyond the actual research questions — is a concern in all human-subjects research, but it takes on unique dimensions in accessibility research. Ming et al. conducted semi-structured interviews with 27 accessibility researchers (doctoral students, professors, post-docs, and industry researchers) who had directly worked with participants with disabilities across populations including blind and low vision users, people with mobility disabilities, deaf and hard of hearing individuals, and neurodivergent participants. The researchers explored how response bias manifested in their studies, how it interacted with other biases, and how they decided whether to accept or address it. The study identified five key factors that influence response bias in accessibility research: participant-researcher dynamics (shared identity, prior relationships, feelings of coercion), participants' view of self (whether they see the researcher as expert or helper, desire to prove ability, the "can do" attitude shaped by navigating an ableist world), the context of accessibility technology (participants' optimism or frustration with technology, the concept of "accessibility dongles" — well-intentioned but useless solutions), researcher bias (how researchers' assumptions and question framing shape responses and interpretations), and sampling bias (who is able to participate and how recruitment language shapes the sample). Notably, the researchers found that response bias in accessibility contexts is deeply intertwined with structural ableism, the charity model of disability, and power differentials that are amplified when researchers without disabilities study people with disabilities.
Key findings
The study revealed a fundamental tension among accessibility researchers about whether to accept or address response bias. Some researchers accepted it as inherent to human research, with one noting "everything is biased because it goes through a human." Others felt strongly that unaddressed response bias could lead to "not valid" research and wasted effort on technologies that participants praised during studies but would never actually use. Several researchers reported that the charity model of disability — where participants perceived researchers as trying to "help" them — led participants to respond with gratitude rather than honest criticism. Participants with disabilities sometimes exhibited "performing ability," focusing on what they could do rather than reporting difficulties, shaped by a lifetime of proving themselves in an ableist world. The context of accessibility technology itself created bias: some participants gave overly positive feedback because any improvement over their frustrating daily technology felt like progress, while others were pessimistic after years of broken promises from "accessibility dongles." Critically, several researchers raised ethical concerns about labeling response bias at all, arguing that calling a participant's response "biased" could constitute epistemic violence — dismissing their lived experience. The paper presents practical guidelines organized around research questions (concrete vs. abstract framing), methods (piloting, triangulation, flexible communication modalities), analysis (looking for contradictions, considering who was in the room), and the importance of building reciprocal relationships with the disability community rather than extractive research encounters.
Relevance
This paper is essential reading for anyone conducting user research with people with disabilities, whether in academic, industry, or organizational settings. The findings challenge practitioners to examine their own assumptions and power dynamics when evaluating accessibility technologies. The concept that participants may give positive feedback out of gratitude, low baseline expectations, or a desire to prove their abilities has direct implications for how organizations interpret usability testing results with disabled users. The guidelines — particularly around using concrete rather than abstract questions, piloting with community partners, offering flexible communication modalities, and being transparent about researcher positionality — are immediately actionable. The paper also raises important questions about the broader accessibility field's relationship to the communities it serves, highlighting the tension between extractive research practices and genuine community partnership. For accessibility professionals, this work serves as a reminder that methodological rigor in disability research requires not just technical competence but deep engagement with disability culture, identity, and the structural forces that shape how disabled people interact with researchers and technology.
Tags: research methodology · response bias · participant-researcher dynamics · disability identity · accessibility research · power dynamics · charity model of disability · ableism