GraSSML: Accessible Smart Schematic Diagrams for All
Z. Ben Fredj, D. A. Duce · 2006 · Proceedings of the 2006 International Cross-Disciplinary Workshop on Web Accessibility (W4A) · doi:10.1145/1133219.1133229
Summary
This paper addresses a persistent gap in web accessibility: making schematic diagrams (organizational charts, flowcharts, UML diagrams, etc.) accessible to people who cannot perceive visual representations. While previous approaches tried to work "bottom-up" — starting with a finished graphic and trying to extract or annotate meaning through moderator interpretation, RDF metadata overlays, or semi-automatic analysis — the authors propose a "top-down" approach called GraSSML (Graphical Structure Semantic Markup Languages). GraSSML captures the structure and semantics of a diagram at creation time rather than trying to reverse-engineer meaning from visual output. The system operates across three levels through a family of XML-based languages: the Semantic Level ("MyLanguage"), which uses domain-specific XML schemas to capture the semantic intent of the diagram — for example, Org_hierarchy for organizational charts with elements like Director, ITManager, and TeamLeader; the Structure Level ("ZineML"), a domain-independent language that represents diagram structure in terms of basic shapes (boxes, connections, hierarchies) readable by both humans and machines; and the Presentation Level, where rules transform the structural and semantic descriptions into multiple output modalities including SVG graphics, XHTML text, and potentially speech or tactile representations.
Key findings
The authors demonstrate the approach with an organizational chart example, showing how the same diagram can be expressed at all three levels with rule-based transformations between them. The MyLanguage document captures that "Director John manages IT Manager Peter and Finance Manager Sue" — semantic information that is only implicit in the visual representation. ZineML captures the structural relationships (which boxes connect to which) without visual specifics. From these, the system generates both an SVG graphical representation and an XHTML textual verbalization describing both structure ("the box labelled Director John is connected to...") and semantics ("The director named John is at the top of the organizational chart. He manages..."). The approach also enables a query system where users can ask questions about the diagram content ("What is the total number of employees reporting to Peter?"), making diagrams truly "smart" rather than static images. Implementation used XSLT transformations for generating SVG and textual representations from ZineML and MyLanguage documents, with Java for presentation algorithms. The key innovation is that by making structure and semantics explicit at creation time, the diagram inherently supports multiple accessible representations without requiring after-the-fact annotation or interpretation by third parties.
Relevance
The problem this paper addresses — making complex visual information accessible — remains one of the hardest unsolved challenges in web accessibility. While the specific XML/Semantic Web technologies proposed (RDF, OWL, SPARQL for diagram semantics) did not gain widespread adoption for this purpose, the core insight is increasingly relevant: accessibility of complex graphics requires capturing semantic meaning at the authoring stage, not bolting on descriptions afterward. Today's approaches to accessible data visualization, chart description, and diagram tools echo this principle — libraries like Highcharts and D3.js can generate structured textual alternatives from the same data used to create the visual, rather than requiring separate alt text. The idea of queryable diagrams anticipates modern AI-driven approaches where users can ask questions about chart content. For practitioners, the key takeaway is that treating graphics accessibility as an alt-text problem fundamentally undersells the potential: when the information behind a diagram is structured and machine-readable, it can support text, speech, tactile, and interactive exploration — truly accessible representations rather than mere descriptions.
Tags: accessible graphics · SVG · semantic web · diagram accessibility · alternative representations · multimodal · visual impairment · data visualization
Standards referenced: SVG · RDF · OWL · XHTML