Evaluating a Tool for Improving Accessibility to Charts and Graphs
Leo Ferres, Gitte Lindgaard, Livia Sumegi · 2010 · Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2010) · doi:10.1145/1878803.1878820
Summary
This paper presents two formative usability studies of iGraph-Lite, a natural language-based assistive technology that enables blind and visually impaired people to interact with statistical line graphs through keyboard commands and text-to-speech output. The system comprises three subsystems: a knowledge representation system that extracts and enriches semantic information from graph object models (from applications like Excel or GNUPlot), a natural language generation system that produces textual descriptions using template-based generation, and a keyboard-driven navigation interface modelled on screen reader interaction patterns. Users can request a general graph description, navigate point-by-point through data using arrow keys, skip multiple points, query their current position, and access information at varying granularity levels (full point description, value only, or slope direction only). The system also enriches raw graph data with computed semantics such as slope qualifications (small, moderate, sharp) and trend labels (advance, double, plummets). Study 1 tested 10 legally blind and 10 matched sighted participants using an initial 9-command interface with simple (3-point) and complex (6-point) line graphs drawn from Statistics Canada publications. Study 2 tested a new cohort of 10 blind and 10 sighted participants with an expanded 18-command interface informed by Study 1 findings, using graphs with 4 and 7 data points. Participants answered questions covering global trends, local data points, and background graph information.
Key findings
In Study 1, blind participants answered 91% of questions correctly — significantly more than sighted participants (83%) — while taking the same amount of time, despite using nearly twice as many keyboard commands. Blind participants rated the system easier to use (7.5/10 vs. 6/10) and reported it was superior to their usual graph interpretation methods (tactile graphs or having others describe graphs). The "skip" command was rarely used, possibly due to confusion about its semantics or fear of losing position. In Study 2, with the expanded command set learned in advance, both groups achieved approximately 85% accuracy with no significant difference between them. Ease-of-use ratings improved for both groups (blind: 7.8/10, sighted: 7.7/10). Critically, the studies revealed distinct navigation strategies between blind and sighted participants: blind participants were three times more likely to move left point-by-point from any location, used individual data point value commands three times more often, and issued redundant direction commands at graph boundaries approximately twice as frequently — interpreted as location-confirmation behaviour consistent with spatial orientation research. Sighted participants tended to jump to the graph beginning and scan left-to-right. These behavioural differences demonstrate that testing assistive technology only with blindfolded sighted participants can miss important interaction subtleties and produce misleading usability ratings.
Relevance
This is one of the most rigorous empirical evaluations of natural language graph accessibility in the ASSETS literature, and its methodological contributions are as significant as its technical ones. The central finding — that blind participants use fundamentally different navigation strategies than sighted participants — has broad implications for assistive technology evaluation: the common practice of using blindfolded sighted participants as proxies for blind users can produce misleading results and miss critical design insights. The location-confirmation behaviour observed (blind users repeatedly checking their position at graph boundaries) directly informed design improvements and is analogous to orientation behaviours seen in physical navigation. For accessibility practitioners, the paper demonstrates that interactive exploration of graphs through natural language is both feasible and preferred by blind users over static descriptions, tactile graphics, or having others interpret graphs for them. The iterative design approach — expanding from 9 to 18 commands based on observed usage patterns — provides a model for evidence-driven interface refinement. The work also highlights that the blind community is effectively excluded from a vast repository of quantitative information published in statistical graphs, with implications for employment, education, and civic participation.
Tags: data visualization · blindness · natural language generation · information graphics · screen readers · text-to-speech · usability evaluation · graph accessibility