The EasyCog Dataset: Towards Easier Cognitive Assessment with Passive Video Watching
Qingyong Hu, Yuxuan Zhou, Jinjian Wang, Yanbin Gong, Yizhen Zhang, Jingnan Sun, Jian Yao, Qijia Shao, Lili Qiu, Qian Zhang, Guihua Li · 2026 · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3789682
Summary
This paper introduces EasyCog, the first large-scale multimodal dataset designed for low-burden, low-cost cognitive assessment using passive visual engagement. The core motivation is to address well-documented limitations of standard clinical cognitive tests such as the Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE): these require active verbal participation, impose significant user burden, are susceptible to practice effects on repeated administration, and depend on clinician subjectivity. EasyCog proposes an alternative paradigm in which participants simply watch a structured 7-minute video while lightweight EEG and contactless eye-tracking sensors record neural and behavioural responses. No verbal responses, complex instructions, or active task performance are required. The video was carefully designed to passively engage nine distinct cognitive domains mapped to the MoCA scale, including attention, memory, visuospatial processing, executive function, language, and orientation, using tasks such as naturalistic scene viewing, semantic incongruity detection, moving target pursuit, paired picture recognition, and simple arithmetic displays. The EEG hardware combines low-cost forehead electrodes covering the frontal lobe with around-the-ear cEEGrid electrodes covering the temporal lobe, together approximating full-brain coverage at a hardware cost under 3 USD per session setup. The dataset spans 101 participants recruited from real-world hospital settings including outpatient rooms, wards, and nursing homes, including 21 healthy controls and 80 patients with Parkinson Disease (PD), Alzheimer Disease (AD), Vascular Dementia (VaD), and other conditions. Both MoCA and MMSE scores were collected by clinicians in these daily settings. A hierarchical deep learning benchmark model named CogAssess is provided alongside 10 baseline comparisons spanning handcrafted EEG features, EEG foundation models, and eye-tracking approaches.
Key findings
Neurophysiological and gaze data show consistent gradients across cognitive severity groups across all nine video tasks. Healthy controls exhibit broader gaze exploration, higher EEG alpha power (8-14 Hz), and stronger inter-regional functional connectivity compared to MCI and dementia groups. Dementia patients showed restricted gaze fixation, reduced attention to semantic incongruities, and impaired smooth pursuit tracking, consistent with known neural correlates of dementia. On the benchmark cognitive score regression task predicting MoCA and MMSE scores from 0-30, the full CogAssess model achieved a mean absolute error of 6.975 and Pearson correlation of 0.213 on MoCA for unseen test subjects, with acknowledged generalisation challenges across subjects. Combining both visual task EEG and resting EEG consistently outperformed using either modality alone. Eye tracking and EEG provided complementary information, with eye tracking more directly tied to visual task performance and EEG adding resting-state cognitive signals. A successive assessment case study demonstrated that EasyCog predictions were substantially more stable than raw MoCA and MMSE scores across repeat administrations, directly demonstrating reduced practice effects. Performance was strongest for PD patients due to consistent EEG patterns, and weakest for AD and VaD patients who exhibited more diffuse and variable neural signatures.
Relevance
EasyCog directly addresses cognitive accessibility: it is designed for people who cannot reliably complete traditional assessments due to attention deficits, communication difficulties, stigma, or severe impairment, populations central to digital accessibility work. A passive, video-based assessment requiring no verbal response, no complex instructions, and no active task compliance has profound implications for inclusive cognitive monitoring in community and care settings. For accessibility practitioners and researchers, the paper demonstrates that passive engagement with structured visual content can elicit measurable cognitive signals, opening pathways for low-burden screening tools that could integrate into everyday digital environments. The wearable hardware setup costs under 3 USD per session, making deployment feasible in under-resourced settings. Key limitations include the dataset being collected primarily in clinical rather than fully home settings, imbalanced disease representation favouring PD over AD and VaD, and EEG generalisation challenges across individuals that remain a known barrier to scaling brain-signal tools to diverse populations.
Tags: cognitive accessibility · cognitive assessment · EEG · eye tracking · dementia · aging · Alzheimer's disease · Parkinson's disease · wearable technology · assistive technology · machine learning · multimodal