Evaluating the Accessibility of Line Graphs through Textual Summaries for Visually Impaired Users
Priscilla Moraes, Gabriel Sina, Kathleen McCoy, Sandra Carberry · 2014 · Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS) · doi:10.1145/2661334.2661368
Summary
This paper presents the SIGHT (Summarizing Information GrapHics Textually) system, which automatically generates natural language summaries of line graphs found in online popular media articles such as newspapers and magazines. The system addresses a persistent accessibility gap: informational graphics in web content typically lack adequate alternative text, and even when alt text is provided, it rarely conveys the high-level knowledge that the graphic communicates to sighted readers. SIGHT works through a multi-stage pipeline. A browser plugin (Chrome) detects line graphs in web pages and sends them to a cloud server. The Visual Extraction Module converts the graphic into an XML representation of its structural components (line segments, annotations, captions, axis labels). The Intention Recognition Module then uses a Bayesian Network to identify the intended message of the graphic from ten possible categories (Rising Trend, Falling Trend, Change Trend, Big Jump, etc.). The Generation Module produces the summary through four sub-phases: Content Determination (selecting the most important propositions using a PageRank-based algorithm), Text Organization (structuring the summary in three groups — overall graph description, trend details, and computational information), Text Complexity (adapting vocabulary and sentence structure to match the reading level of the surrounding article), and Summary Generation (applying pronominalization and surface realization via FUF/SURGE). A key design principle is that summaries adapt to the reading level of the article containing the graphic, using different text plans for grade bands from 5th grade through college level.
Key findings
The system was evaluated through a three-phase task-based experiment. Four blind participants (recruited through the Delaware Association for the Blind, all screen reader users with 7+ years of internet experience) were given SIGHT-generated summaries and asked questions about the graphs. A control group of 24 sighted freshmen answered the same questions while viewing the actual graphics. Blind participants using the summaries achieved 75% correct answers, compared to 80.87% for sighted users viewing the graphics — a remarkably small gap demonstrating that the summaries effectively conveyed the high-level knowledge of the line graphs. Some graphs proved difficult for both groups: L17 and L26 had lower scores across the board, with sighted users frequently needing to re-examine the image. When blind participants answered incorrectly, it was most often on questions requiring specific numeric values (maximum, minimum, interpolation) that were not always included in the summary. Blind participants generally found the summaries clear and concise, though one participant preferred shorter summaries with the option to request more detail. Notably, in some cases blind users outperformed sighted users, who tended to guess from memory rather than re-examine the graphic. The cloud-based architecture (requiring only a browser plugin rather than a full local installation) significantly improved the system's reachability compared to earlier versions.
Relevance
This research directly addresses one of the most persistent and under-served accessibility challenges on the web: making data visualisations accessible to people who cannot see them. While WCAG requires alternative text for images, the guidelines offer limited practical guidance on how to describe complex informational graphics like line graphs, and most web content simply omits meaningful descriptions. SIGHT's approach of automatically generating summaries that capture the intended message — not just raw data — represents a fundamentally different strategy from simple alt text or data table alternatives. The reading-level adaptation is particularly valuable, as it demonstrates that accessibility and plain language can work together to serve users with both visual and cognitive access needs. For practitioners, this work highlights that effective graph descriptions should lead with the high-level message (what the trend means) rather than low-level details (individual data points), and that automated natural language generation can produce descriptions that are nearly as useful as direct visual access. The system's vision of generating alt text automatically foreshadows current AI-based image description tools, though SIGHT's structured approach to identifying communicative intent remains more sophisticated than generic image captioning.
Tags: data visualization · natural language generation · information graphics · blindness · screen readers · alternative text · reading level · graph accessibility