Designing Judicious Interactions for Cognitive Assistance: The Acts of Assistance Approach
Jérémy Bauchet, Hélène Pigot, Sylvain Giroux, Dany Lussier-Desrochers, Yves Lachapelle, Mounir Mokhtari · 2009 · Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '09) · doi:10.1145/1639642.1639647
Summary
This paper presents Archipel, a cognitive orthosis developed at the DOMUS Laboratory at the University of Sherbrooke, Canada, designed to guide people with cognitive impairments through complex activities of daily living such as meal preparation. The system takes a two-pronged approach: first, an activity monitoring module detects problems the person encounters during task completion; second, an assistance generation module produces appropriate human-machine interactions — called "acts of assistance" — to help resolve those difficulties. The monitoring system uses environmental sensors (motion sensors, flow meters, electromagnetic contacts, ultra-wideband location tags) distributed throughout a smart apartment to implicitly track the person's actions without requiring explicit input. Activities are modelled as hierarchical task structures in XML, with constraints specifying ordering, timing, and repetition rules. The system classifies detected errors into four cognitive deficit types: initiation deficits (not starting an activity), planning deficits (performing steps out of order or not knowing the next step), attention deficits (being in the wrong location or distracted), and memory deficits (forgetting what to do or where objects are). The "acts of assistance" framework draws on speech acts theory from linguistics to structure the system's communications. Each act of assistance is defined by a message type (recall, indicate, guide), content, sender, receiver, temporality, and expected feedback. Assistance operates at two levels: a procedural macro act that provides step-by-step guidance through the entire activity via a multimedia touchscreen interface, and punctual micro acts that address specific difficulties as they arise.
Key findings
An evaluation with 12 people with intellectual disabilities (referred to in the paper as mental retardation) completing two similar meal preparation recipes — one with the Archipel assistant and one without — showed a decrease in the number of researcher interventions needed when the orthosis was present. The reduction was observed for all 12 participants regardless of their cognitive profile, though more autonomous participants showed smaller decreases. The study revealed that cues became more abstract over time as participants grew familiar with the system, shifting from concrete location-based prompts to more general reminders to consult the orthosis interface. Several practical challenges emerged: the activity model needed to handle unexpected user actions (e.g., participants using the tap to fill a glass of water during pasta cooking, which the flow meter misinterpreted as a cooking-related action), and the system struggled to determine the person's actual intent when actions deviated from the expected sequence. The researchers identified the need for a common-sense inference layer to handle extra actions that don't interfere with the primary activity.
Relevance
This research makes an important contribution to the design of ambient assistive living environments for people with cognitive impairments. The "acts of assistance" framework provides a principled, linguistically-grounded approach to generating contextually appropriate prompts — moving beyond simple step-by-step reminders to assistance that adapts based on the type of cognitive difficulty detected. The four-category error classification (initiation, planning, attention, memory) offers a practical taxonomy that practitioners and developers can use when designing cognitive support tools. The separation of activity monitoring from assistance generation is architecturally significant, as it allows either component to be updated independently. The study highlights a fundamental challenge in cognitive assistive technology: balancing the system's assumptions about what the user intends to do against the user's actual autonomy and agency. The finding that unexpected but legitimate actions can confuse the monitoring system underscores the importance of flexible activity models that accommodate natural human variability rather than enforcing rigid task sequences.
Tags: cognitive assistance · intellectual disability · smart home · activities of daily living · pervasive computing · activity monitoring · independent living · adaptive systems