What We Talk About: Designing a Context-Aware Communication Tool for People with Aphasia
Shaun K. Kane, Barbara Linam-Church, Kyle Althoff, Denise McCall · 2012 · Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2012) · doi:10.1145/2384916.2384926
Summary
This paper describes the design and development of TalkAbout, a context-aware augmentative and alternative communication (AAC) application for tablet devices that adapts its vocabulary to a user's current location and conversation partner. People with aphasia — a language disorder commonly caused by stroke — often struggle to recall words spontaneously but can recognize them when prompted with images, text, or audio cues. Traditional AAC systems provide browsable word lists organized by topic hierarchies, but navigating large vocabularies is cumbersome and slow. TalkAbout addresses this by using contextual signals (GPS location, face recognition of conversation partners) to automatically surface the most relevant words and phrases. The system was developed through participatory design with 5 adults with aphasia at a community aphasia center in Baltimore, over approximately 6 weeks of iterative design activities including focus groups, contextual data gathering, and prototype testing. The research team used storyboard diagrams, play-acted scenarios, and interactive prototypes to communicate design concepts to participants who had varying levels of language ability. A Wizard of Oz evaluation approach was used to simulate context detection, allowing participants to experience adaptive vocabulary without requiring fully automated sensing.
Key findings
Participants were most enthusiastic about location-based and conversation partner-based adaptation, prioritizing these over other proposed scenarios such as recognizing objects or tracking current events. During prototype testing, participants rated the system positively on a 9-point scale (7-9 for "liked software", 7-9 for "better than current technology", 5-9 for "would use"). The research identified three key benefits of context-aware AAC: reducing search effort for users with motor impairments, improving organization of large vocabulary sets, and encouraging recognition over recall — a critical advantage since people with aphasia can often recognize words they cannot independently produce. The study also produced six practical guidelines for participatory design with this population: prepare alternative activities for participants with different abilities, support rapid prototyping to iterate during field visits, balance focus groups by communication ability level, present scenarios in multiple formats, make demonstrations concrete and personalized, and clearly illustrate interface changes with audible and visual feedback.
Relevance
This research demonstrates how mobile device sensors can make AAC systems more usable by reducing the cognitive and navigational burden of finding relevant vocabulary. For accessibility practitioners, TalkAbout offers a compelling model for adaptive interfaces that respond to user context rather than requiring users to navigate complex hierarchies. The participatory design guidelines are especially valuable — they address the practical challenge of involving people with significant communication impairments in the design process, an area where standard user research methods often fall short. The finding that participants preferred recognition-based interaction over recall-based word retrieval reinforces a fundamental accessibility principle: reducing cognitive load by presenting options rather than requiring generation. Although the study is small and the prototype relied on Wizard of Oz techniques rather than fully automated sensing, it laid important groundwork for context-aware assistive technology design.
Tags: aphasia · augmentative and alternative communication · context-aware computing · participatory design · stroke · mobile devices · adaptive interfaces