Attention Analysis in Interactive Software for Children with Autism
A. Ould Mohamed, V. Courboulay, K. Sehaba, M. Menard · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169011
Summary
This paper presents the attention analysis component of the AutiSTIC project, a collaborative effort between computer scientists and the Child Psychiatry department of La Rochelle Hospital in France to develop interactive software that helps children with autism during rehabilitation. The system provides personalised educational games that adapt in real time based on the child's behaviour and attention level. A key challenge with autism is that children often cannot prioritise sensory stimuli, becoming saturated by too much input and responding with repetitive, self-injurious, or avoidant behaviour. The system uses a non-intrusive camera-based tracking approach — a video camera built into a small desk observes the child without requiring any wearable devices, helmets, or wires that might distract or irritate young subjects. The vision system tracks the child's face, eyes, and gaze direction to compute an attention measure based on the correlation between where the child is looking and where the relevant on-screen object is located. The system architecture consists of three subsystems: observation and behaviour analysis, a decision system that adapts game activities, and an action system that runs the games. Each child has a personalised profile containing general information, domain knowledge, preferences, and interaction history, allowing experts to define specific stimuli (sounds, images) that carry emotional and motivational significance for individual children.
Key findings
The attention measurement approach uses a robust metric based on the distance between the child's gaze direction and the relevant on-screen object over time. Rather than a simple binary attentive/inattentive classification, the system integrates the gaze-object distance over time windows with two expert-defined thresholds (p1 and p2) adapted to each child. When the distance drops below p1, the child is considered less attentive; below p2, the system intervenes by presenting stimuli to refocus attention. If the distance remains between p1 and p2 for an extended period, the system also estimates declining attention. The system classifies observable behaviours into categories including rupture (child leaves the computer), avoidance (temporal — passivity in front of the screen; visual — gaze not oriented to screen), and different attention types (selective, maintained, shared). The paper draws on a psychological framework distinguishing attention types: awakening, selective attention, maintained attention, shared attention, internal/external absent-mindedness, and vigilance. Calibration achieved a mean distance error of 13.2mm and mean angular error of 0.7 degrees. Preliminary results showed that attention naturally decreases over time during sessions, confirming expected patterns, and that the system reliably distinguishes attentive from inattentive states.
Relevance
This research represents an early and thoughtful application of computer vision and adaptive interfaces to support children with autism spectrum disorder. The non-intrusive design — using a desk-mounted camera rather than wearable sensors — is especially important for this population, as children with autism are often hypersensitive to physical contact and unfamiliar equipment. The personalisation approach, where expert clinicians define individual profiles and intervention parameters for each child, reflects an understanding that autism manifests differently in every individual and that one-size-fits-all software is inadequate. The multi-agent architecture that separates observation, decision-making, and action provides a model for adaptive educational software more broadly. While the specific technology has advanced significantly since 2006, the underlying principles — non-intrusive attention monitoring, expert-guided personalisation, adaptive stimuli delivery, and behaviour-driven interface adaptation — remain highly relevant to designing technology-assisted interventions for children with autism and other neurodevelopmental conditions.
Tags: autism · attention analysis · eye tracking · computer vision · children · adaptive interface · educational software · neurodevelopmental disability