Video Gaming for the Vision Impaired
Manohar Swaminathan, Sujeath Pareddy, Tanuja Sunil Sawant, Shubi Agarwal · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018) · doi:10.1145/3234695.3241025
Summary
This Microsoft Research demo paper presents the Responsive Spatial Audio Cloud (ReSAC), a novel interaction toolkit designed to make mainstream video games accessible to people with visual impairments. The research is motivated by the finding that visually impaired people want to play the same video games as sighted gamers — not just purpose-built audio games — and to participate in the social conversations around gaming culture. ReSAC represents the game environment as a "spatial audio cloud" where every relevant object is an entity with attached metadata: 3D spatial coordinates, name, description, audiocons or earcons, a visibility distance threshold, and spatial extent. The toolkit provides several interaction tools: Narrator (a screen reader equivalent for games), Viewdio (a generalization of audio description), Anchor (a selected object for orientation reference), NorthHorn (a constant directional tone from the North for orientation), BodyScan (spatialized audio playout of objects intersecting the player's virtual frustum), VisorScan (a variant of BodyScan), and Select and Reach (guided navigation to a selected object via spatialized siren modulated in volume and pitch). The system uses a metadata container script attached to every game object in Unity, so when the player collides with an object, its name and description are extracted and sonified — an interaction analogous to exploring with a cane or hands in the real world.
Key findings
A preliminary user study with 6 visually impaired participants (5 fully blind, 1 with residual sensitivity, ages 21-37) tested a simple first-person 3D game built in Unity where players navigated a virtual room to find and reach 4 objects using stereo headphones and an Xbox controller. None had previously used a game controller, and only one had experience with audio games. All participants successfully completed the task, with completion times ranging from 12:23 to 36:52 minutes. Participants reported excitement and engagement. P1 stated: "I don't play PC/mobile games because I have a prejudice that games are largely inaccessible. But this game has changed my perception." P4 reported feeling immersed: "I could feel myself inside the room. I heard the voice exactly from the objects' location, so it helped me to know where I should move and take turns. I am imagining, and I am playing." One participant even deduced spatial relationships, figuring out that some objects were located behind larger objects requiring navigation around them. Orientation and sensemaking were identified as the main challenge, possibly due to the short play duration, absence of proprioceptive feedback, or limitations of the current toolset.
Relevance
ReSAC addresses a fundamental gap in game accessibility: rather than creating separate audio-only games for blind players, it provides a framework that can be integrated into mainstream games built with standard engines like Unity. The approach of attaching accessibility metadata to game objects parallels how ARIA attributes add accessibility information to web elements — suggesting a path toward standardized game accessibility. The paper argues that game engines like Unity, Unreal, and 3D modeling platforms like Blender should evolve to support accessibility metadata creation natively, minimizing the burden on game developers. For accessibility practitioners, the toolkit demonstrates that spatial audio can convey complex 3D environments to blind users, and the interaction tools (especially BodyScan and Select and Reach) provide design patterns applicable to any spatial audio navigation system. The enthusiastic participant responses confirm that the desire for mainstream gaming experiences among visually impaired people is strong and underserved.
Tags: blindness · game accessibility · spatial audio · sonification · 3D sound · video games · interaction toolkit