SymbolPath: A Continuous Motion Overlay Module for Icon-Based Assistive Communication
Karl Wiegand, Rupal Patel · 2012 · Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2012) · doi:10.1145/2384916.2384957
Summary
This demonstration paper introduces SymbolPath, an overlay module designed to improve the speed and ease of message formulation in icon-based augmentative and alternative communication (AAC) systems. Traditional icon-based AAC devices require users to make precise, discrete tap selections on individual icons — lifting their finger between each selection. This is particularly challenging for many AAC users who have concomitant motor impairments alongside their speech impairments, making repetitive precise movements slow, effortful, and fatiguing. SymbolPath replaces discrete tapping with continuous motion: users draw a single unbroken path through all desired icons on a touch screen without lifting their finger. The concept is analogous to Swype-style text entry on keyboards, but applied to icon-based communication grids. The system is implemented in Python as an overlay that can be integrated with existing AAC systems. Icons are arranged in a grid organized by lexical role — actors, verbs, objects, and modifiers — color-coded and laid out left to right to mirror subject-verb-object syntax. Crucially, users do not need to select icons in syntactic order; they can choose icons based on physical proximity, and the system handles reordering.
Key findings
SymbolPath solves two key technical challenges for continuous motion icon selection: superset pruning and syntactic reordering. Since a user's continuous path inevitably crosses over unintended "bystander" icons, the system must prune the set of touched icons to identify only those the user actually intended. It also must reorder the selected icons into grammatically correct sentences, since users select based on spatial convenience rather than syntactic order. The system uses a combination of semantic frames, semantic grams, and physical path features (such as pausing or slowing over intended icons) to perform semantic disambiguation — determining which subset and ordering of icons produces the most meaningful and grammatically accurate utterance. The text-to-speech synthesizer then voices the completed message. SymbolPath is compatible with any continuously varying analog input including stylus, mouse, joystick, or laser pointer, making it adaptable across different access methods and motor abilities.
Relevance
This work addresses a critical intersection in assistive technology: the co-occurrence of speech and motor impairments, which affects a large proportion of AAC users. By reducing the physical demands of message composition, SymbolPath has the potential to increase communication rate and reduce fatigue — both major barriers to effective AAC use. The approach of borrowing continuous input techniques from mainstream technology (like Swype keyboards) and adapting them for assistive contexts is a valuable design pattern for accessibility practitioners. The modular overlay design is also noteworthy, as it allows integration with existing AAC systems rather than requiring users to switch to an entirely new platform. For developers building AAC or communication tools, this paper demonstrates how natural language processing can compensate for reduced motor precision.
Tags: augmentative and alternative communication · AAC · motor impairment · speech impairment · icon-based communication · continuous motion input · text-to-speech · natural language processing · touch screen