Exploring The Affordances of Game-Aware Streaming to Support Blind and Low Vision Viewers: A Design Probe Study
Noor Hammad, Frank Elavsky, Sanika Moharana, Jessie Chen, Seyoung Lee, Patrick Carrington, Dominik Moritz, Jessica Hammer, Erik Harpstead · 2024 · Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '24) · doi:10.1145/3663548.3675665
Summary
This paper investigates how game-aware streaming systems can be made accessible to blind and low-vision (BLV) viewers on platforms like Twitch. Live game streaming is a major social and entertainment medium, but the experience is overwhelmingly visual — viewers watch gameplay footage, read chat overlays, and interpret on-screen HUD elements — making it largely inaccessible to BLV audiences. Building on MARS (Mixed Augmented Reality Streaming), a system that extracts structured game state data in real time, the researchers developed a design probe that presents game information in personalizable visual and non-visual formats. The probe was built for the game Phasmophobia (a cooperative ghost-hunting game) and provided BLV viewers with three types of accessible game data: player status (health, sanity, items held), ghost activity (evidence types, ghost room location), and environmental information (map layout, player positions). This data was delivered through a companion Twitch extension accessible via screen reader, with options for sonification (mapping data to audio cues) and text-to-speech summaries. The researchers conducted qualitative interviews and live prototype testing sessions on Twitch with 12 BLV participants, 10 of whom were active Twitch viewers and several of whom were game streamers themselves.
Key findings
Participants identified three core motivations for watching game streams: entertainment and social connection (chat interaction, community membership), information gathering (learning about games before buying, discovering accessible games), and vicarious participation (experiencing games they cannot play themselves due to accessibility barriers). The design probe revealed several important findings. First, BLV viewers strongly valued having direct access to structured game data rather than relying solely on the streamer's verbal commentary — this reduced dependence on sighted intermediaries and gave viewers a sense of autonomy. Second, personalisation was critical: participants wanted to control what information they received, when, and through which modality (sonification, TTS, or screen reader text), because information overload was a major concern during fast-paced gameplay. Third, participants saw the system not just as a viewing aid but as a potential tool for playing games themselves — several suggested that similar data-to-audio pipelines could make games directly playable. Fourth, the social dimension of streaming was paramount: participants valued being able to contribute meaningfully to chat discussions with knowledge about game state, rather than being passive listeners. The study also found that BLV viewers had developed sophisticated existing strategies for stream accessibility, such as choosing streamers who narrate their gameplay, using Twitch chat TTS bots, and relying on sound-rich games. The concept of "game-aware" streaming — where the system has structured access to game state data — was identified as a promising architectural pattern because it decouples information from its visual presentation, enabling alternative modalities.
Relevance
This paper addresses an underexplored area of digital accessibility — the spectator and social media experience. While game accessibility research has focused primarily on making games playable, this work recognises that watching and discussing games is itself a significant form of cultural participation that BLV people are excluded from. For accessibility practitioners, the key insight is that structured data access is a powerful enabler of accessibility: when game state is available as data rather than only as pixels on screen, it can be rendered in any modality. This principle extends beyond gaming to any domain where visual information could be extracted and re-presented — dashboards, data visualisations, live events, and collaborative tools. The personalisation findings reinforce that accessible alternatives must give users control over information density and modality rather than delivering a fixed alternative experience. Limitations include the single-game focus (Phasmophobia) and small sample size, but the architectural approach and design implications are broadly transferable to other games and streaming contexts.
Tags: game accessibility · blind and low vision · live streaming · Twitch · sonification · design probe · spectator experience