Action Blocks: Making Mobile Technology Accessible for People with Cognitive Disabilities
Lia Carrari, Rain Michaels, Ajit Narayanan, Lei Shi, Xiang Xiao · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3373625.3418043
Summary
This demonstration paper presents Action Blocks, an Android application developed by Google that simplifies mobile device interactions for people with cognitive disabilities. The app allows users or their caregivers to create customizable one-tap buttons on the device home screen, each linked to a Google Assistant command such as calling a contact, playing music, getting directions, or controlling smart home devices. The core design insight is that standard mobile interactions require reading, writing, memorizing, planning, and multitasking — cognitive demands that can be challenging or impossible for the estimated 630 million people worldwide who have cognitive disabilities. Action Blocks addresses this by flattening complex multi-step interactions into single taps associated with memorable images. The design was developed iteratively with input from caregivers and professionals supporting individuals with dementia, Autism Spectrum Disorder, Down Syndrome, and mild age-related cognitive decline, as well as informal feedback from people with these conditions. Key design features include guided templates for common actions (phone calls, messages, music, navigation, smart home controls), symbol images drawn from Augmentative and Alternative Communication (AAC) research, and configurable vocalization that provides multimodal sensory feedback when an action is triggered. The app was launched in May 2020 and the paper reports on aggregated user feedback from in-app reports and Google Play Store reviews.
Key findings
Action Blocks proved beneficial beyond its target audience — both caregivers of people with cognitive disabilities and neurotypical users adopted the app. Caregivers valued the simplified interface for family members who routinely got lost navigating standard phone interfaces. Neurotypical users appreciated the efficiency of condensing multi-step voice commands and routines into single taps. User feedback revealed demand for two key enhancements: support for more Action Blocks on the home screen (since flattening interaction depth naturally increases width), and the ability to chain multiple actions into sequences. The paper demonstrates that the depth-width tradeoff from prior cognitive assistive technology research applies directly — reducing cognitive load through simpler interactions creates demand for more shortcuts. The configurable vocalization feature highlighted the tension between multimodal feedback (helpful for understanding cause-and-effect) and privacy concerns in social settings.
Relevance
This paper offers a practical, commercially deployed example of cognitive accessibility design that moved from research prototype to production Android app. For accessibility practitioners, it demonstrates several transferable principles: reducing interaction complexity to single actions, using familiar images as cognitive anchors, providing optional multimodal feedback, and involving caregivers in the configuration process. The depth-width tradeoff finding is particularly relevant for anyone designing simplified interfaces — reducing steps per task inevitably increases the number of entry points users want. The work also highlights how cognitive accessibility features can benefit all users, reinforcing the curb-cut effect. A limitation is that user feedback came primarily from tech-savvy caregivers and app store reviewers rather than from people with cognitive disabilities directly, leaving gaps in understanding end-user experience.
Tags: cognitive accessibility · mobile accessibility · assistive technology · aging in place · augmentative and alternative communication · smart home · interaction design · inclusive design