Making Microsoft Excel Accessible: Multimodal Presentation of Charts
Iyad Abu Doush, Enrico Pontelli, Dominic Simon, Tran Cao Son, Ou Ma · 2009 · Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '09) · doi:10.1145/1639642.1639669
Summary
This paper presents a system for making Microsoft Excel charts accessible to people with visual impairments through a combination of haptic force feedback and audio cues. The authors identify that while screen readers and specialised tools can handle tabular data in Excel, the graphing component remains largely inaccessible. The system extracts chart data from Excel via the Office XML format, parses the structure, and renders it through the Novint Falcon — a consumer-grade 3D haptic device that provides force feedback in three dimensions. The core design principle is that different chart types and different user tasks require different interaction modalities. The system supports five chart types: line charts, bar charts, pie charts, scatter charts, and surface charts. For each type, the system provides three navigation strategies. Exploratory navigation allows free movement through the haptic space, with passive force feedback guiding users along chart elements (e.g., gravity forces pulling the cursor toward data lines, attraction forces near data points). Guided navigation constrains movement along the chart structure, automatically advancing through data points with active force feedback. Summary modality provides spoken statistical overviews including minimum, maximum, average, start and end points, and trend information. The system also supports comparative navigation for multiple data series — for line charts, users can select two lines and feel vibration forces indicating distance between them at corresponding x-coordinates, while for bar charts, stiff forces push users between corresponding bars to convey relative height differences.
Key findings
Evaluation with nine sighted students (blindfolded, with no prior haptic experience) showed that guided navigation produced significantly better chart comprehension than exploratory mode — 83% of students correctly drew a line graph after guided navigation versus 33% in exploratory mode. The average navigation time was 3.85 minutes for exploratory mode versus 0.4 minutes for guided mode. All students successfully identified minimum and maximum values through the summary mode. For two-line comparison tasks, 66% of students correctly identified intersection points, with an average task time of 2.8 minutes. Students generally agreed that guided navigation was more helpful for understanding chart shape, while audio cues were important for understanding the virtual axes. For bar charts, most students found it difficult to navigate using the haptic device alone (taking less than 2 minutes per bar chart) and the introduction of the x-axis locking mechanism improved the experience. For scatter charts, the clustering-based haptic approach with gravity forces toward cluster centres and vibration at boundaries was effective for conveying data groupings. Several users disliked vibration as a signal for touching the graph, preferring stiff forces instead.
Relevance
This work addresses an important gap in workplace accessibility — charts and graphs are ubiquitous in business applications like Excel, yet remain largely inaccessible to screen reader users. The multi-strategy approach (exploratory, guided, summary) is well-grounded in how people actually use charts: sometimes browsing to get an overview, sometimes seeking specific values, sometimes needing a statistical summary. The finding that guided navigation dramatically outperformed free exploration has practical implications for all haptic and non-visual graph access systems — structured interaction with constraints produces better mental models than unconstrained exploration. The use of a consumer-grade haptic device (Novint Falcon, approximately at time of publication) rather than expensive research hardware made the approach more realistic for real-world deployment. The chart-type-specific interaction designs recognise that a one-size-fits-all approach to graph accessibility is insufficient — bar charts, line charts, scatter plots, and pie charts each require fundamentally different haptic metaphors to convey their distinct visual properties.
Tags: data visualization · haptic technology · sonification · visual impairment · charts · multimodal interaction · Microsoft Excel · accessible graphs