Providing Access to the High-Level Content of Line Graphs from Online Popular Media
Priscilla S. Moraes, Sandra Carberry, Kathleen McCoy · 2013 · Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/2461121.2461123
Summary
This paper extends the Interactive SIGHT (Summarizing Information Graphics Textually) system to generate natural language summaries of line graphs found in online popular media articles, making their high-level content accessible to visually impaired users via screen readers. Unlike simple alt text or data table alternatives, SIGHT aims to capture the intended message of a graphic — the communicative purpose the graphic designer had in creating it — and convey that along with visually prominent features. The system architecture has three main modules: a Visual Extraction Module that processes the graphic image into an XML representation, an Intention Recognition Module that uses a Bayesian Network to infer the graphic's intended message (from categories like Rising Trend, Falling Trend, Change Trend, Big Jump, etc.), and a Generation Module that produces coherent English text. This paper focuses on the Generation Module extension for line graphs, which required substantially different approaches than the earlier bar chart work due to the continuous nature of line data. The system identifies eight key features that humans notice in line graphs: overall behavior, individual trends, initial/end values, volatility, slope steepness, overall amount of change, time span, and maximum/minimum points.
Key findings
The content selection strategy uses a modified PageRank algorithm to rank propositions (facts about the graphic) by importance, selecting content in a discourse-aware fashion that favors related propositions and avoids redundant ones. Feature weights are derived from human experiments — for instance, users frequently mention trend steepness and volatility when describing line graphs, so these features receive higher initial weights. A human experiment with 21 participants established how people categorize slope steepness (flat, slightly rising/falling, rising/falling, steeply rising/falling, very steeply rising/falling), and a novel volatility metric was developed based on frequency of visual changes and their amplitude. The stopping criteria for summary length balances brevity with completeness using normalized importance scores. An evaluation with 16 participants who drew graphs from the generated summaries and then rated the summaries gave an average effectiveness rating of 7.54 out of 10. Only 49 of 201 responses identified missing information, and very few found the summaries misleading. Some participants noted redundancy where sentences could be aggregated, pointing to future work on sentence merging and pronominalization.
Relevance
This research addresses a critical accessibility gap: information graphics in news articles, magazines, and websites convey information that is almost never duplicated in the surrounding text, yet they remain largely inaccessible to screen reader users. Standard alt text practices typically provide only a caption or basic description, missing the high-level message and visual features that make a graphic informative. SIGHT's approach of automatically recognizing the graphic designer's intended message and generating rich natural language summaries goes far beyond what manual alt text typically achieves. For accessibility practitioners, this work demonstrates that meaningful chart accessibility requires conveying trends, patterns, and relationships — not just data points. The system's focus on popular media graphics (as opposed to scientific figures) makes it directly relevant to everyday web browsing. While the visual extraction module was not yet fully robust at the time of publication, the natural language generation techniques and the framework for understanding what makes a good graphic description remain valuable contributions to the field of data visualization accessibility.
Tags: data visualization · visual impairment · natural language generation · assistive technology · screen readers · image accessibility · information graphics · alt text