Accessibility of 3D Game Environments for People with Aphasia: An Exploratory Study
Julia Galliers, Stephanie Wilson, Sam Muscroft, Jane Marshall, Abi Roper, Naomi Cocks, Tim Pring · 2011 · The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2049536.2049562
Summary
This paper reports on an exploratory study investigating how people with aphasia interact with 3D game environments and related technologies. Conducted as part of the GReAT (Gesture Recognition in Aphasia Therapy) project at City University London, the study involved five people with aphasia who participated in twelve participatory workshop sessions over six months. The participants varied in age (twenties to seventies), years post-stroke (2.5 to many years), and severity of language difficulties. The workshops explored four technologies: Nintendo Wii Sports (tennis and bowling), accelerometer-based gesture recognition using the Wiimote with the wiigee library, computer vision-based gesture recognition using OpenCV with a webcam and coloured glove, and custom 3D game environments built in Unity 3D. The interdisciplinary team included HCI researchers and speech and language therapy (SLT) researchers, and sessions combined observation, video recording, and adapted interview techniques using visual aids, rating scales, and pictures to accommodate participants' communication difficulties.
Key findings
The study identified two overarching categories of findings: motivation factors and interaction design issues. For motivation, four themes emerged: reliability (unreliable gesture recognition was deeply demotivating — one participant called the Wiimote approach a "stupid idea"), feedback and reassurance (participants needed explicit positive confirmation that their actions were correct, not just implied success through game progression), anger and frustration (natural side effects of communication difficulties, triggered by unexpected changes or failures), and positive effects (all participants enjoyed the games and gained confidence). Five key interaction design issues were identified: controlling the pace (users needed to speed up or slow down at different points), reminding (constant visual reminders of game state were essential due to memory difficulties), mapping and consistency (direct, intuitive mappings between input and on-screen effects were critical, while abstract or compound button sequences caused confusion), complexity and distraction (minimal visual noise, no more than two attention-demanding elements on screen), and individual differences (highly variable preferences and abilities requiring extensive customisation). The vision-based OpenCV gesture recognition with a yellow glove was strongly preferred over the Wiimote approach because it was more forgiving of gesture variation.
Relevance
This is one of the first studies to investigate how people with aphasia interact with 3D game environments, filling a significant gap in accessibility research that had focused primarily on physical rehabilitation games post-stroke rather than communication-focused applications. The design principles that emerged have broad applicability beyond gaming: keep verbal instructions minimal, provide non-textual visual reminders of system state, ensure direct and intuitive input mappings, allow user-controlled pacing, offer explicit positive feedback, maintain consistency in interface layout, and provide toggleable support that can be switched off when not needed. For game developers and accessibility practitioners, the finding that what helps one person with aphasia may distract another underscores the need for extensive customisation options. The study also demonstrates the value of participatory design with people with communication difficulties, showing that meaningful participation is possible with appropriate adapted research methods such as visual rating scales and picture-based preference ranking.
Tags: aphasia · game accessibility · virtual reality · stroke recovery · cognitive accessibility · gesture recognition · participatory design · interaction design