Automatically Identifying Trouble-Indicating Speech Behaviors in Alzheimer's Disease
Frank Rudzicz, Leila Chan Currie, Andrew Danks, Tejas Mehta, Shunan Zhao · 2014 · ASSETS '14: Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2661334.2661382
Summary
This paper addresses the challenge of automatically detecting communication breakdowns in conversations with people who have Alzheimer's disease (AD). AD is a progressive neurodegenerative disease that deteriorates memory, executive capacity, visual-spatial reasoning, and linguistic ability, making it difficult for individuals to follow simple dialogues. The researchers annotated two databases of dyad conversations—the Carolina Conversations Collection (31 interviewees with AD and 41 without) and DementiaBank (196 older adults with dementia and 98 controls)—with 12 types of trouble-indicating behaviors (TIBs). These TIBs include requests for repetition, requests for confirmation, requests for specific information, correction of semantic inaccuracy, lack of uptake, hypothesis formation (guessing), metalinguistic comments like "I can't remember," and dysfluency markers. Over 200 lexical, syntactic, and acoustic features were extracted from all utterances.
Key findings
Using ANOVA-based feature ranking and machine learning classification, the system achieved up to 79.5% accuracy in classifying utterances as containing trouble-indicating behaviors or not. The most indicative features of TIB included speech rate and utterance likelihoods in a standard language model. The research identified 12 distinct types of TIBs that signal communication confusion, ranging from local requests for repetition to global requests to restart a topic. This classification capability is intended to support the development of intelligent dialogue systems that can engage people with AD in two-way communication—both to guide them through daily household tasks and to fulfil social functions, encoding in software the techniques that caregivers use to help patients recover from communication breakdowns.
Relevance
As the incidence of Alzheimer's disease continues to rise with aging populations, technology that supports communication for people with dementia becomes increasingly important. This research has direct applications for building adaptive dialogue systems—virtual assistants or social robots—that can detect when a person with AD is confused and automatically adjust their communication strategy, such as simplifying language, repeating information, or redirecting the conversation. For accessibility practitioners, the work highlights that cognitive accessibility extends beyond interface design to include how technology communicates with users. The 12-category taxonomy of trouble-indicating behaviors provides a useful framework for anyone designing conversational interfaces for people with cognitive impairments. The automated detection approach could also serve as an assessment tool for clinicians monitoring disease progression.
Tags: Alzheimer's disease · dementia · speech recognition · natural language processing · machine learning · cognitive accessibility · dialogue systems