Interactive SIGHT Demo: Textual Summaries of Simple Bar Charts
Seniz Demir, David Oliver, Edward Schwartz, Stephanie Elzer, Sandra Carberry, Kathleen F. McCoy · 2010 · Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2010) · doi:10.1145/1878803.1878864
Summary
This demonstration paper presents Interactive SIGHT, a browser extension designed to make simple bar charts found in online popular media (newspapers, magazines) accessible to people with visual impairments. The system goes beyond providing raw data values — it identifies the intended message of the graphic, recognising that designers choose specific chart types and layouts to make a point. The architecture comprises four modules: a Visual Extraction Module that uses computer vision techniques to parse the graphical image into an XML representation of its elements; an Intention Recognition Module that uses a Bayesian network to infer the chart's communicative intent from visual signals (e.g., an increasing trend, a comparison between categories); an Interaction Module that coordinates the workflow; and a Generation Module that produces natural language text descriptions using discourse-aware generation algorithms. Implemented as a Browser Helper Object, the system automatically detects bar charts when a web page loads, inserts them into the page's tab order, and provides instructions accessible via screen reader (JAWS) on how to launch the description.
Key findings
The system provides a two-tier interaction model. First, it generates a brief high-level summary capturing the chart's intended message — for example, "The graphic shows an increasing trend in the percentage of college faculty employed part-time over the period from the year 1971 to the year 2003." Users can then explore further through a menu-based follow-up question facility with categories including General Information, Focused Information, and Specific Information. The generation algorithm is discourse-aware, tracking what has already been communicated across multiple follow-up turns to control redundancy and mark repetitions appropriately. The menu-based approach was deliberately chosen so users unfamiliar with the structure of bar charts could still explore them effectively. The Visual Extraction Module had limitations at the time of publication, requiring clear drawing and handling only limited fonts and text placements. Evaluations including tests with people with visual impairments showed the system was very positively received, with users able to answer questions about graphics they encountered.
Relevance
Interactive SIGHT addresses a fundamental gap in web accessibility: information graphics are overwhelmingly visual and typically receive either no alternative text or only a brief, uninformative label. The system's approach of inferring the designer's communicative intent — rather than simply listing data values — is a significant contribution to how we think about making data visualisations accessible. For accessibility practitioners, this work highlights that meaningful chart descriptions require understanding what the chart is trying to say, not just what data it contains. A trend chart should be described as showing a trend, not simply enumerated bar by bar. The interactive follow-up model also demonstrates that accessibility for complex visual content may require layered exploration rather than a single static description. While the system was limited to simple bar charts in popular media, the underlying principles — intent recognition, natural language generation, and progressive disclosure through follow-up questions — remain highly relevant as organisations work to make dashboards, reports, and data journalism accessible.
Tags: information graphics · data visualization · blindness · screen readers · natural language generation · alternative text · bar charts · web accessibility · browser extension