TAIG: textually accessible information graphics
Seniz Demir · 2008 · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '08) · doi:10.1145/1414471.1414555
Summary
This short paper presents TAIG (Textually Accessible Information Graphics), a system designed to make bar charts and other information graphics accessible to people with visual impairments by generating coherent natural language summaries. TAIG extends the SIGHT system, which is triggered by a keystroke when a user encounters a graphic while browsing the web. When a bar chart is identified, SIGHT analyzes it and infers a core message — the high-level communicative intent of the graphic (e.g., "increasing trend" or "comparison between categories"). TAIG takes the XML representation of the graphic produced by SIGHT, which includes component data such as bar heights and values along with the logical representation of the core message, and translates it into natural language using templates. The key innovation is that rather than simply describing what the graphic looks like (listing data points or describing visual features), TAIG conveys the graphic's overall message — the knowledge a sighted reader would gain from viewing it. For example, given a bar chart of jury awards over time, TAIG generates a summary explaining the increasing trend, the range of values, and notable variations, producing prose rather than raw data.
Key findings
TAIG generates coherent initial summaries organized around the core message of a graphic, constructed using heuristics derived from empirical studies with human subjects. The system goes beyond existing approaches like iGraph-Lite, which provide alternative access to what a graphic looks like through sound, touch, or textual descriptions of data. TAIG instead focuses on conveying the communicative content — what the graphic means. The paper also outlines a follow-up question mechanism with three types of queries: General follow-up (ranking unused propositions by importance and semantic relationships), Focused follow-up (classifying additional information into categories like magnitude of trend change, allowing users to select a category of interest), and Specific follow-up (letting users request information about particular data points, such as a specific year). This interactive dialogue approach addresses the finding from human subject experiments that congenitally blind participants could not identify what kinds of information they might request from an unfamiliar graphic, making structured follow-up options essential.
Relevance
This research tackles a fundamental accessibility problem: information graphics in newspapers, magazines, and web content are almost entirely inaccessible to screen reader users, and conventional alt text typically provides either no description or a superficial one. TAIG's approach of inferring and communicating the graphic's intended message rather than merely describing its visual appearance represents a significant conceptual advance. For accessibility practitioners, this work highlights the inadequacy of treating chart accessibility as a simple alt-text problem — a bar chart showing an increasing trend carries a message that cannot be captured by listing individual values. The interactive follow-up question system also addresses a real usability challenge: blind users cannot easily form a mental model of an unfamiliar graphic, so the system must guide exploration rather than expecting users to know what to ask. Although limited to bar charts at the time of publication, the underlying framework of message inference, natural language generation, and structured dialogue provides a blueprint for making all types of data visualizations meaningfully accessible.
Tags: data visualization · graph summarization · screen readers · visual impairment · natural language generation · bar charts · alternative text · information graphics