Do-It-Yourself AAC: Co-Designing User-Programmable AI Communication Tools with People with Aphasia
Jong Ho Lee, Stephanie Valencia · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26) · doi:10.1145/3772318.3790321
Summary
Lee and Valencia explore how people with aphasia (PWA) can become designers of their own AI-powered augmentative and alternative communication (AAC) tools rather than users of rigid, pre-built systems. The authors note that aphasia, a language disorder typically resulting from stroke or brain injury, affects roughly 2 million people in the United States and produces highly individual communication profiles that one-size-fits-all AAC tools rarely address. Motivated by low AAC adoption rates and growing interest in generative AI for language support, they propose a tangible, block-based visual programming approach that lets PWA chain together discrete AI functions (such as Take Picture, Read Text from Picture, Suggest Sentence, Check Grammar, Create Image, Show & Read Text) into custom pipelines suited to their own conversational situations. The study used a two-part, IRB-approved protocol with eight PWA and optional conversation partners. Session one was a remote semi-structured interview probing how participants currently prepare for and manage daily conversations across four scenarios varied by formality (casual/formal) and role (informative/requestive). Session two was an in-person co-design workshop with a card-sorting activity to rate AI function usefulness, followed by a programming activity using color-coded, shape-encoded physical puzzle blocks (blue inputs, orange AI functions, green terminators; triangular edges for text, square edges for images). Data were analyzed using reflexive thematic analysis, yielding 26 codes and three overarching themes about preparation, AI perspectives, and programmable tools.
Key findings
PWA already interweave multiple mainstream technologies — notes apps, Google Maps, MyChart, Grammarly, Constant Therapy, Speak4me, ChatGPT — to prepare for conversations, retrieve words, practice speech, and draft messages, showing substantial appropriation of off-the-shelf tools. All participants who attended session two (P2, P4, P5, P6, P8) successfully assembled at least one hypothetical program with the physical code blocks, and several built multiple variants for the same scenario, demonstrating that tangible block-based programming can act as an accessible scaffold for expressing complex technology needs without language-intensive prompting. Card-sort results showed strong divergence: sentence-completion functions (Suggest Sentence, Finish Sentence, Check Grammar) were broadly rated useful, while image-based functions (Create Image, Explain Image, Read Text from Picture) produced sharply split ratings tied to personal context — P4 and P5 valued image tools for delivery and clinical scenarios, while P2 and P6 found them unhelpful or unsure. Participants also diverged on whether AI should act as an alternative voice (P8 embraced it; P1 feared it would slow rehabilitation). Shape-based compatibility cues helped users self-repair incompatible block pairings. Fatigue, hemiparesis, and language load remained significant barriers during sessions.
Relevance
For accessibility practitioners, the paper reframes AAC from a delivery problem to a personalization problem and argues that end-user programming — long used by BLV users and non-disabled ML practitioners — is viable for PWA when programming surfaces use tangible shapes, pictographic icons, and constrained function libraries instead of free-form prompts. The tangible blocks doubled as boundary objects, letting PWA communicate design intent to researchers without relying on speech, which has direct implications for inclusive participatory design methods. Practitioners designing AI assistive tools should treat customization, multimodal output, and user agency as first-class requirements and should anticipate diverse preferences even within a single disability group. Limitations include the small sample (n=8), mild-to-moderate aphasia only, US recruitment from a single metropolitan area, and hypothetical (not deployed) programs, leaving open questions about real-world use, LLM hallucinations, and training-data representation for disabled speech patterns.
Tags: aphasia · AAC · augmentative and alternative communication · end-user programming · generative AI · large language models · participatory design · co-design · visual programming · tangible interaction · cognitive accessibility · speech and language · stroke · assistive technology · personalization