Supporting the Design of Data Visualisation for the Visually Impaired through Reinforcement Learning
Dalal Aljasem · 2020 · Proceedings of the 17th International Web for All Conference (W4A) · doi:10.1145/3371300.3383354
Summary
This doctoral consortium paper presents a research programme aimed at making data visualizations more accessible to people with partial vision loss, specifically those with peripheral vision damage (tunnel vision from conditions like glaucoma) or central vision loss (from age-related macular degeneration). The author proposes using reinforcement learning to model and predict the visual search behaviour of people with these conditions when they interact with data visualizations. The research frames human visual perception as a Partially Observable Markov Decision Process (POMDP), where the viewer can only perceive part of the display at any given moment and must make strategic eye movements to gather information for decision-making. The work sits at the intersection of machine learning, cognitive science, and accessibility, drawing on WHO statistics showing 2.2 billion people worldwide affected by blindness or visual impairment — a number growing due to aging populations, increased screen time, and conditions like diabetic retinopathy. The methodology involves first replicating existing cognitive models of visual search behaviour in normally-sighted users, then adapting these models to simulate vision impairment patterns, and ultimately using the models to inform the design of visualizations optimized for users with specific types of visual field loss.
Key findings
The paper reports successful replication of a prior cognitive model of visual search, demonstrating that Q-learning (a reinforcement learning algorithm) can predict human eye movement patterns during visual search tasks. The replicated model was tested across four visualization conditions — covered text, covered colour, visible text, and visible colour — and its accuracy and average number of fixations closely matched the original human participant data. Notably, the model successfully predicted the centre-of-gravity effect, where fixations cluster not on the most informative cue directly, but on nearby positions from which multiple high-validity cues can be observed through peripheral vision. This finding is particularly relevant for vision impairment, as it demonstrates that peripheral vision plays a critical strategic role in visual search — meaning that people with peripheral vision loss (e.g., from glaucoma) face fundamentally different challenges than those with central vision loss (e.g., from macular degeneration). The paper also references prior research showing that glaucoma patients make fewer saccades and those with primary open-angle glaucoma have slower, less accurate eye movements, particularly with visual field damage.
Relevance
This research addresses a significant gap in accessible visualization design. While much accessibility work focuses on providing non-visual alternatives to charts and graphs (such as data tables or sonification), this work takes a different approach by asking how visual presentations themselves can be redesigned to work better for people who have some remaining vision. For the estimated billions of people with partial vision loss globally, this could lead to more effective data presentations in healthcare dashboards, financial tools, educational materials, and workplace applications. The reinforcement learning approach is novel because it moves beyond static design guidelines toward computationally modelling how specific vision conditions affect information gathering, potentially enabling automated or semi-automated optimization of visualization layouts for different types of visual impairment. However, as a doctoral consortium paper, this presents a research plan rather than completed results — the vision impairment-specific modelling and human participant evaluation remain as future work.
Tags: data visualization · visual impairment · reinforcement learning · visual search · machine learning · eye tracking · accessible design