AudioFunctions.web: Multimodal Exploration of Mathematical Function Graphs
Dragan Ahmetovic, Cristian Bernareggi, João Guerreiro, Sergio Mascetti, Anna Capietto · 2019 · Proceedings of the 16th International Web for All Conference (W4A) · doi:10.1145/3315002.3317560
Summary
This full research paper presents AudioFunctions.web, a web application that enables blind and visually impaired people to explore mathematical function graphs through a combination of sonification, earcons, and speech synthesis. The system is the platform-independent successor to AudioFunctions, an earlier iPad-only prototype. AudioFunctions.web introduces several design innovations: a dual sonification approach that maps function values to sound frequency (200-2000 Hz range) and the distance between the user's explored point and the function graph to sound intensity, allowing users to track the graph shape by following the loudest sound. Spatial audio through stereo channels reinforces left-right position awareness. The system provides verbal or musical earcons at points of interest (local minima/maxima, axis intersections, origin passage), and users can request spoken information about exact coordinates, first and second derivatives at any point. A key design goal is personalized access through multiple interfaces — touchscreen, keyboard, touchpad, and mouse — on both mobile and desktop devices. Each interface maps a consistent set of actions (exploration, fast exploration, return to center, request/cancel information, complete sonification, toggle edit mode) to interface-appropriate interactions. The system is implemented in JavaScript using Function-Plot for graph rendering, Tone.js for Web Audio API sonification, and Web Speech API for voice generation. It can be linked directly from teaching materials via URL parameters specifying the function, viewport, and earcon preferences, and can even be embedded as a self-contained HTML file within PDFs for offline use.
Key findings
An experimental evaluation with 12 blind participants and 1 low-vision participant assessed usability across three interaction modalities (touchscreen, keyboard, touchpad/mouse) using the System Usability Scale (SUS) and custom questions. All three modalities received positive SUS scores: touchscreen ranked highest at 86.7 (grade A+), keyboard at 82.7 (grade A), and touchpad/mouse at 77.7 (grade B+). However, touchpad/mouse scores were significantly lower than both touchscreen (p < 0.05) and keyboard (p < 0.01). Participants were divided in their preferences: 6 preferred touchscreen for its proprioceptive, intuitive exploration similar to embossed paper; 5 preferred keyboard for its simplicity and precision; 1 preferred touchpad/mouse. A critical finding was that mathematical education level significantly impacted usability scores more than interface expertise. Participants with university-level maths education rated the touchpad/mouse modality significantly higher (86.88 vs 73.13, p < 0.05), found it easier to use, and perceived less learning effort. This suggests that domain knowledge — knowing what to expect from a function graph — matters more than interface familiarity for this type of specialized tool. Participants with higher mathematical expertise could identify exact functions (e.g., "it is a gaussian curve"), while less experienced users described general shapes ("it goes up, then down"). The dual sonification using both frequency and volume was favorably perceived compared to the prior mono-dimensional approach, enabling exploration without dividing the viewport into separate areas.
Relevance
This paper provides strong empirical evidence for several principles important to accessibility practice. First, it demonstrates that offering multiple interaction modalities is not merely a nice-to-have but essential — users with different abilities and preferences genuinely prefer different interfaces, and no single modality is universally best. This validates the principle of providing multiple means of interaction rather than designing a one-size-fits-all solution. Second, the finding that domain knowledge impacts perceived usability more than interface expertise has important implications for designing specialized accessible tools: the interface should accommodate varying levels of domain expertise, not just varying levels of technical skill. For STEM accessibility specifically, this work shows that web-based sonification tools can achieve high usability scores with blind users across multiple platforms, making mathematical graph exploration practical without requiring expensive haptic hardware or static tactile materials. The system's availability as a linkable web service under a Creative Commons license, accessible directly from educational documents, provides a practical model for integrating accessibility solutions into existing educational workflows rather than requiring separate specialized software.
Tags: sonification · mathematics accessibility · visual impairment · blindness · STEM accessibility · multimodal interaction · data visualization · assistive technology · web accessibility · user study · touchscreen · keyboard accessibility
Standards referenced: Web Audio API · Web Speech API · SVG · HTML5 · CSS