Using Think Aloud Protocol with Blind Users: A Case for Inclusive Usability Evaluation Methods
Sambhavi Chandrashekar, Tony Stockman, Deborah Fels, Rachel Benedyk · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169040
Summary
This paper examines whether the Think Aloud Protocol (TAP), one of the most widely used usability evaluation methods, can be effectively applied when testing with blind screen reader users. The study involved six vision-impaired students evaluating a website designed to test the effects of accessibility guideline conformance on user performance. Four participants were blind and used JAWS, while two had low vision and used Windows Magnifier. The authors discovered two fundamental problems with applying conventional TAP to blind users. First, the standard TAP training procedure — having participants read a short passage aloud to practice verbalising thoughts — was not feasible for blind users whose method of reading is listening to a screen reader, and no alternative training materials or methods existed in the literature. Second, during testing, blind users offered far fewer comments than the low vision users, even when prompted regularly (every 15 seconds of silence). Blind participants expressed reluctance to repeatedly pause and restart their screen reader to articulate comments, as it disrupted their task performance. The low vision users, by contrast, always responded when prompted.
Key findings
The authors offer two theoretical explanations for why TAP fails with blind screen reader users. First, there is competition for cognitive and perceptual resources between speech production (talking aloud) and audio signal processing (understanding screen reader output). Screen reader use already demands high cognitive effort: users must process content sequentially, hear repeated page elements, assemble a mental model of the page from small fragments, and parse synthetic speech that lacks prosodic cues like emphasis and rhythm. Adding concurrent verbalisation overloads these resources. Second, drawing on Wickens' multiple resource theory, humans can divide attention between visual and auditory channels more effectively than between two tasks using the same modality. Blind users using TAP must use two auditory channels simultaneously — listening to screen reader output and producing speech — causing sensory overload. The paper suggests retrospective verbal protocols (collected immediately after each task) as a potential alternative, noting that while retrospective data may be less effective at revealing decision-making processes, it avoids the dual-channel conflict. The authors also reference parallel work by Roberts and Fels who found TAP similarly unusable with deaf sign language users, leading to the development of a Gestural Think Aloud Protocol.
Relevance
This paper raises a critical methodological concern that remains highly relevant: standard usability evaluation methods were designed for able-bodied users and may not work when applied to users with disabilities. As inclusive design practices increasingly require involving disabled users in evaluation, researchers and practitioners need adapted methods. The specific insight about auditory channel competition during screen reader use has broad implications — it applies not only to TAP but to any research method or system feature that requires blind users to process additional audio while using a screen reader. The suggestion to use retrospective protocols, while pragmatic, also highlights the need for purpose-built inclusive evaluation methods rather than simply adapting existing ones. For accessibility practitioners conducting user testing with blind participants, this paper provides important guidance: avoid concurrent think-aloud, consider retrospective protocols after each task, and be aware that screen reader use imposes substantial cognitive demands that constrain what additional tasks can be performed simultaneously.
Tags: usability testing · think aloud protocol · blind users · screen readers · research methods · inclusive design · cognitive load · user research