Helping or Hindering: Inclusive Design of Automated Task Prompting for Workers with Cognitive Disabilities
Gavin R. Philips, Morris Huang, Cathy Bodine · 2024 · ACM Transactions on Accessible Computing · doi:10.1145/3628447
Summary
This paper addresses the significant employment barriers faced by people with cognitive disabilities (PwCDs) in the United States, where only 28.6% of the approximately 8.8 million working-age adults with cognitive disabilities are employed. Those who do find work often receive reduced hours and pay, frequently requiring job coach intervention that can impact both wages and self-esteem. The authors developed and tested a Nonlinear Context-Aware Prompting System (NCAPS) designed to support workers in warehouse order-picking tasks without requiring human supervision. The NCAPS uses a multi-device configuration mounted on a standard utility cart. A prompting device displays augmented reality navigation arrows overlaid on a live camera feed that scans QR codes placed on the warehouse floor, guiding workers to item locations via an A* pathfinding algorithm. When workers arrive at the correct location, the display switches to show a photograph of the target item, and an auditory prompt directs them to scan the item barcode. If the wrong item is scanned, the system prompts correction; if errors persist, a supervisor can be notified. A virtual job coach avatar provides additional visual and audio cues throughout the process. The system was developed through rapid ethnography in five functioning warehouse environments, interviews with managers and workers, and iterative refinement with student researchers and a consultant with cognitive disabilities. The research team identified several key requirements: the system should provide navigation and task-relevant visual cues, integrate into existing equipment like manual carts, support real-time error detection and correction, and use predetermined pick paths rather than relying on workers to memorize item locations. A pilot crossover study evaluated the NCAPS with seven working-age adults with mild to moderate intellectual and developmental disabilities, including participants with cerebral palsy, autism spectrum disorder, Down syndrome, and other diagnoses. Each participant completed two-hour simulated work sessions picking items in a mock warehouse environment, using both the NCAPS and traditional paper pick tickets in separate sessions with a two-week washout period.
Key findings
The NCAPS dramatically reduced errors compared to paper tickets. Participants averaged 14.29 errors per session with paper tickets versus 0.57 errors with the NCAPS. Six of seven participants made zero errors when using the NCAPS; only one participant (P06) produced any errors with the system. This near-elimination of picking mistakes demonstrates the potential for context-aware prompting to improve accuracy in warehouse work. However, the productivity results revealed a more complex picture. Overall, participants gathered significantly fewer items per session when using the NCAPS (mean 130.7) compared to paper tickets (mean 227.4). Critically, this effect varied by baseline ability: participants with the poorest paper-ticket performance (P06 and P07) actually improved their productivity with the NCAPS, while those with higher baseline performance experienced reduced throughput. A linear regression revealed a strong relationship (R² = 0.944) between baseline productivity and the change experienced when switching to NCAPS. System Usability Scale (SUS) scores showed no significant difference between methods, with paper tickets averaging 76.4 ("good" usability) and NCAPS averaging 70.7 (also "good"). Interestingly, participants who experienced productivity decreases with NCAPS often still rated it higher in usability than paper tickets, while the two participants whose productivity improved rated NCAPS lower in usability, potentially reflecting distrust from occasional false negative prompts. Participant feedback highlighted several concerns: distrust when the system incorrectly indicated a wrong item was selected, frustration when waiting for prompts slowed them down, and difficulty maneuvering the cart to ensure QR codes passed through the camera field of view. Multiple participants nonetheless reported enjoying the NCAPS more than paper tickets due to its prompting assistance and novelty.
Relevance
This research offers essential insights for organizations developing assistive technology for workers with cognitive disabilities. The central finding that a tool can simultaneously help some users while hindering others challenges simplistic assumptions about technological accommodation. A system that eliminates errors but reduces speed for higher-performing workers may not achieve employer acceptance or benefit all intended users. The authors propose four design principles for vocational technology supports. First, technology must not hinder any user, suggesting flexible systems that can adapt to individual ability levels rather than forcing all workers through the same workflow. Second, technology must be robust to real-world environmental conditions including varied lighting, wireless connectivity issues, and physical wear on markers. Third, technology must be intuitive, trustworthy, and ignorable, meaning users should be able to work confidently without focusing on the tool itself, and higher-performing users should be able to bypass guidance when unnecessary. Fourth, and perhaps most significantly, the authors advocate for inclusive design that targets the most challenging use cases first, then refines toward mainstream users. This inverts the typical approach of adapting mainstream technology for accessibility. The resulting tools would benefit not only people with diagnosed disabilities but also workers with temporary or situational impairments, such as those new to a job, unfamiliar with a warehouse layout, or working in a second language. This broader applicability could reduce stigma, increase employer willingness to adopt such systems, and ultimately improve employment outcomes for people with cognitive disabilities.
Tags: cognitive disability · intellectual disability · vocational support · employment · assistive technology · augmented reality · prompting systems · inclusive design · warehouse