End-to-End Solution for Accessible Chemical Diagrams
Volker Sorge, Mark Lee, Sandy Wilkinson · 2015 · Proceedings of the 12th International Web for All Conference (W4A) · doi:10.1145/2745555.2746667
Summary
This paper presents a complete pipeline for making chemical diagrams — typically distributed as inaccessible bitmap images — fully accessible on the web. The system takes bitmap images of molecular diagrams as input and produces interactive, explorable SVG diagrams with semantic annotations. The pipeline has four stages: (1) image analysis using the MolRec system, which vectorizes the bitmap, performs OCR on chemical symbols, separates bond elements, and uses a rule-based engine with 18 rules to recognize the molecule and output it in Chemical Markup Language (CML); (2) annotated SVG generation that preserves the connection between visual elements and their chemical meaning; (3) semantic enrichment that identifies structural components (ring systems, aliphatic chains, functional groups) using the Chemistry Development Kit and creates an abstraction tree with multiple levels of detail; and (4) a browser front-end that enables interactive exploration via keyboard navigation, aural rendering through ARIA live regions, synchronized visual highlighting, and magnification. The system generates natural language descriptions at three levels of detail (basic, context, and summary) in two variants (basic and advanced), allowing it to serve audiences from secondary school students to experienced chemists.
Key findings
The image analysis component successfully handled all 63 scanned textbook images and 20 complex molecules from a US patent database, plus 100 web-harvested diagrams in various formats and quality levels — including hand-drawn images (all but 4 of which were recognized, plus 1 with a red background). The system works across platforms (Linux, Mac OS, Windows), the five most popular browsers, and common screen readers (JAWS, NVDA, VoiceOver). User testing with secondary school students with visual impairments showed they could quickly and independently engage with diagrams using only a URL, with no additional training needed. Blind students could reproduce molecular structures — even unfamiliar ones — by listening to the generated descriptions. Blind students preferred keyboard-controlled exploration, while students relying on magnification preferred menu buttons (which also minimized key binding clashes with screen readers). The interactive exploration model was reported as a potential replacement for physical models and tactile graphics. A fallback mode was implemented for older browsers lacking SVG/ARIA support, providing three levels of alternative text descriptions attached to the original bitmap image.
Relevance
This work addresses a critical gap in STEM accessibility: scientific diagrams are pervasive in education yet almost universally inaccessible when delivered as bitmap images. The approach is particularly valuable because it does not require authors to change their behavior — existing bitmap diagrams can be retroactively made accessible. The multi-layered exploration model (molecule level, block level, atom level) demonstrates best practices for making complex visual information navigable non-visually: providing overview before detail, supporting hierarchical exploration, and offering descriptions at multiple levels of expertise. The technology builds entirely on open web standards (SVG, WAI-ARIA, JavaScript), making it platform and screen-reader independent. The authors note the approach is extensible to other STEM diagrams that share similar geometric primitives, potentially serving as a foundation for broader scientific diagram accessibility — similar to how MathJax made mathematical notation accessible on the web. For web developers working with scientific content, this illustrates how semantically rich SVG combined with ARIA can create genuinely explorable alternatives to flat images.
Tags: STEM accessibility · image accessibility · visual impairment · screen readers · SVG · WAI-ARIA · chemistry · education · image recognition · low vision · diagram accessibility
Standards referenced: WAI-ARIA