The application of spatialization and spatial metaphor to augmentative and alternative communication
Patrick Demasco, Alan F. Newell, John L. Arnott · 1994 · Proceedings of the First Annual ACM Conference on Assistive Technologies (Assets '94) · doi:10.1145/191028.191036
Summary
This paper presents a collaborative project between the University of Delaware and the University of Dundee investigating how spatialization and spatial metaphors from mainstream HCI research can improve interfaces for augmentative and alternative communication (AAC). The authors observe that over the preceding 20 years, computing had shifted fundamentally from symbolic to spatial interaction through graphical user interfaces, yet AAC systems had not kept pace with these advances. The paper frames AAC vocabulary retrieval as a Visual Information Seeking (VIS) task — users must navigate and locate items within a visually represented information space. The authors derive four design principles from the HCI literature: the Perceptually Smooth Movement Principle (interfaces should support smooth animation through information spaces), the Spatial Contiguity Principle (the information space should be contiguous to prevent users getting lost), the Separation of Manipulation and Navigation Principle (direct manipulation of objects and navigation through the space should be conceptually distinct activities, potentially using separate input devices), and the Benefits of Space and Metaphor Principle (spatial representations and metaphors are broadly beneficial for interface design). They then describe VAL (Virtual Access to the Lexicon), a prototype that applies these principles to create a computer-based word board retaining spatial equivalence to a user's existing paper-based communication board.
Key findings
VAL addresses the fundamental problem that paper word boards with hundreds of items cannot fit on a single computer screen. Rather than using level-based systems (which cause users to get lost, similar to hypertext navigation problems) or coding systems (which impose cognitive memory burdens), VAL creates a virtual two-dimensional surface that the user navigates smoothly, seeing a portion of their word board at any time. The system separates navigation (scrolling the viewpoint across the board using joystick or trackball) from selection (choosing a specific word), supporting both single and multiple input devices. Custom screen drivers achieve 30-frames-per-second smooth scrolling on MS-DOS. Beyond replicating paper boards, VAL extends vocabulary access through an interface to the WordNet lexical database, which stores 54,000 words and their semantic relations (synonymy, taxonomy, meronymy, thematic associations). When a user selects a word like "car," associated words such as "ambulance," "BMW," "engine," or "driving" become available, dramatically expanding communicative range beyond the fixed word board. Average AAC communication rates of only 15 words per minute (versus 150 for typical speech) make this efficiency gain significant.
Relevance
This paper is notable for being one of the first to systematically apply mainstream HCI research on information visualization and spatial interfaces to the AAC domain. The design principles articulated — smooth animation, spatial contiguity, separation of navigation and manipulation — remain relevant for modern AAC app design on tablets and touchscreens. The insight that users' existing spatial knowledge of their paper boards should be preserved in digital equivalents reflects a user-centered design philosophy that respects established communication patterns. The integration of WordNet for vocabulary expansion was ahead of its time, presaging modern AAC systems that use semantic networks and word associations to extend vocabulary. For practitioners, the paper demonstrates the value of cross-pollination between HCI research and assistive technology development, and highlights that communication rate remains a fundamental challenge in AAC — a problem that persists three decades later.
Tags: augmentative and alternative communication · spatial interface · human-computer interaction · lexical access · information visualization · cerebral palsy · word prediction