Customizing Directions in an Automated Wayfinding System for Individuals with Cognitive Impairment
Alan L. Liu, Harlan Hile, Gaetano Borriello, Pat A. Brown, Mark Harniss, Henry Kautz, Kurt Johnson · 2009 · Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '09) · doi:10.1145/1639642.1639649
Summary
This paper presents the design and evaluation of a mobile wayfinding system that provides customized navigation directions to individuals with cognitive impairments such as traumatic brain injury, cerebral palsy, and intellectual disability. The central innovation is using a Markov Decision Process (MDP) framework to automatically select the most appropriate type of direction for each step of a route, adapting to individual user preferences and abilities. The system generates two core direction types: landmark-based directions that show a geo-tagged photo of a nearby building with an overlaid arrow and text (e.g., "Go along the path toward Student Union Building"), and turn-based directions that use standard street signs with arrow icons (e.g., "Take the second right and follow the sidewalk"). The MDP models each user as an agent navigating a graph network, with the state incorporating the user's position, orientation, and available options at each decision point. The system learns from observed behaviour — tracking whether users successfully follow directions or deviate — to update transition probabilities and refine its model of each individual's capabilities. The landmark-selection component automatically retrieves appropriate photos from geo-tagged image databases, using heuristics like landmark popularity and perspective matching to choose images that will be recognisable from the user's current vantage point.
Key findings
A user study with 7 participants with cognitive impairments (ages 21-49) navigating two routes on a university campus revealed several important findings. Landmark-based directions were significantly easier to follow and less error-prone than turn-based directions when heading toward a landmark, confirming the value of visual reference points. However, participants showed a wide range of preferences — some found landmark photos helpful for orientation while others found them cognitively challenging due to needing to match the photo to real-world features. Compound directions combining turns with landmarks (e.g., "Go along the sidewalk toward <landmark> and take the next left") caused confusion, as some participants interpreted the second segment as a separate instruction. Participants with visual impairments struggled with photo clarity on the Nokia N95 device, suggesting larger screens or higher contrast would help. The study also revealed that the system needed to account for fatigue, with some participants (e.g., those with multiple sclerosis) becoming less capable over time. Participants were generally positive about the system, expressing desire to use it independently, and reported feeling comfortable using the device in public without stigma.
Relevance
This research demonstrates that one-size-fits-all navigation systems are inadequate for people with cognitive impairments, and that personalisation through adaptive models can substantially improve wayfinding assistance. The MDP framework provides a principled way to learn individual preferences over time rather than requiring users to manually configure settings. The findings have broad implications for accessible navigation app design: direction types should be mixed and matched based on context and user capability, landmark images need to account for the user's actual vantage point and lighting conditions, and compound instructions should be broken into discrete steps. The study also highlights important considerations around fatigue, attention splitting between device and environment, and the social acceptability of assistive devices. While the evaluation was small-scale, the participatory design approach — incorporating feedback from prior studies with the same population — provides a model for iterative, user-centred development of assistive navigation technology.
Tags: cognitive impairment · wayfinding · navigation · mobile technology · Markov decision process · personalization · adaptive systems · independent living