Understanding and Improving Drilled-Down Information Extraction from Online Data Visualizations for Screen-Reader Users
Ather Sharif, Olivia H. Wang, Alida T. Muongchan, Katharina Reinecke, Jacob O. Wobbrock · 2023 · Proceedings of the 20th International Web for All Conference (W4A) · doi:10.1145/3587281.3587284
Summary
This full research paper presents a comprehensive investigation into how screen-reader users (SRUs) extract drilled-down information from online data visualizations, and how the VoxLens tool can be improved to support this process. While prior work on VoxLens focused on holistic information extraction (summary statistics and trend understanding), this study addresses the more granular task of extracting specific data points, making comparisons within categories, and comparing across different factors in visualizations. The research follows a multi-phase methodology. First, a role-playing study with 12 SRUs taking on personas (explorer, teacher, news reporter) generated a rich set of 542 natural language queries, which the authors analyzed into a three-tier taxonomy of drilled-down information needs: Category (extraction, comparison within, comparison across) mapped to Information Types and specific Queries. Second, a 12-day longitudinal study with 7 SRUs interacting with 30 real-world visualizations validated this taxonomy in naturalistic settings, confirming that the categories captured authentic user needs. Third, the authors redesigned VoxLens to support drilled-down extraction using keyword matching and implemented a task-based evaluation comparing 10 SRUs using VoxLens against 10 non-screen-reader users without assistive tools. An extended abstract of this work (DOI: 10.1145/3587281.3587961) was also presented at this conference.
Key findings
The task-based evaluation revealed striking results for accessibility equity. SRUs using the improved VoxLens achieved 5.6% higher accuracy than non-SRUs without tools, effectively closing 62% of the performance gap identified in prior research. SRUs also completed tasks 22% faster than with the original VoxLens and 50% faster than non-SRUs without any tools. NASA-TLX workload assessments showed no significant difference between SRUs using VoxLens and non-SRUs, suggesting the tool does not impose additional cognitive burden. The three-tier taxonomy identified three main categories of drilled-down information needs: extraction (retrieving specific values), comparison within a single factor, and comparison across multiple factors. Qualitative feedback from SRUs identified five areas for further improvement: a repeat command for replaying responses, a playground environment for free exploration, more succinct response formatting, text-based response output alongside audio, and longer input time windows for voice queries. The study also revealed limitations in the keyword-matching approach, particularly with synonyms and voice recognition errors.
Relevance
This paper makes a significant contribution to accessibility by demonstrating that well-designed assistive tools can not only close but potentially reverse performance gaps between screen-reader users and sighted users for data visualization tasks. The finding that SRUs achieved higher accuracy than non-SRUs challenges deficit-based assumptions about accessibility. For practitioners, the three-tier taxonomy of information needs provides a concrete framework for designing accessible data visualization interfaces. The identified limitations with keyword matching suggest that future implementations should explore more sophisticated NLP approaches. Organizations publishing data visualizations should note that alternative text alone is insufficient for complex charts — interactive, queryable interfaces like VoxLens represent a more equitable approach to data access.
Tags: screen readers · data visualization · voice interface · information extraction · natural language processing · assistive technology · data sonification · user study
Standards referenced: WCAG 2.1 · ARIA