Nonvisual Access to an Interactive Electric Field Simulation: Work in Progress
Clayton Lewis, Derek Riemer · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2700648.2811341
Summary
This demonstration paper presents work-in-progress on making the PhET Project's Radiating Charge physics simulation accessible to blind users through audio representation. The PhET Project offers over 130 interactive science simulations downloaded more than 75 million times annually, but these simulations are completely inaccessible to blind learners—both because content is presented as pictorial animations and because control requires seeing the mouse cursor. The authors draw on blind computer scientist T.V. Raman's conceptual insight that the visual system functions as a means of querying a spatial database; if alternative means of asking questions and receiving answers exist, the visual system becomes unnecessary. The Radiating Charge simulation displays the dynamic electric field produced by a moving charge, showing complex patterns like spiral disturbances emanating outward when the charge moves in a circle. The authors created a wholly nonvisual representation using audio encoding. Field strength maps naturally to loudness—stronger fields produce louder sounds. Field orientation maps to pitch, with low tones representing alignment with the positive X-axis and higher pitches indicating larger angles. However, this simple mapping creates a problem: when the field oscillates through the 0/360-degree boundary, pitch jumps discontinuously from lowest to highest values, falsely suggesting discontinuous change in the field itself. To solve this, the authors employ Shepard tones—a psychoacoustic phenomenon where a sequence of tones creates the paradoxical impression of continuously rising (or falling) pitch indefinitely, with the first tone sounding higher than the last. This allows oscillations and even full rotations to be represented without discontinuous pitch jumps.
Key findings
The authors solved the challenge of nonvisual probe control using the W3C Pointer Lock API, which allows the simulation to detect relative mouse motion independent of cursor position. This enables blind users to move a probe through the field space using natural mouse movements, without needing to see the cursor. Probe movement allows users to explore how field strength decreases with distance from the center and how orientation varies across space. For navigation, the authors adopted a spatial (but invisible) command layout suggested by accessibility consultant Sina Bahram, based on the TWBlue Twitter client pattern: a horizontal row of tabs containing vertically-arranged commands, navigable via arrow keys with space bar for selection. This spatial convention could potentially apply across many applications. A significant limitation remains: visual displays convey simultaneous information across space—sighted users can see spiral patterns emerge across the entire field at once. The probe technique cannot replicate this, as it samples only one point at a time. The authors suggest tactile displays or stereo audio with multiple probes as potential solutions but prioritize working within standard browser and commodity hardware constraints. Notably, the authors discovered that inclusive design can benefit all users: oscillation in the electric field is actually clearer in the audio representation than in the sparse visual lines-of-force display, which requires close reasoning to interpret.
Relevance
This work addresses a critical gap in STEM education accessibility. Interactive simulations are increasingly central to science education, yet remain largely inaccessible to blind students. The conceptual framework—treating visual display as a queryable database that can be accessed through alternative modalities—provides a generalizable approach for making dynamic visual content accessible. The use of Shepard tones to represent continuous angular change is an elegant solution to the problem of mapping circular quantities to linear pitch scales, applicable beyond physics simulations to any context requiring nonvisual representation of orientation or rotation. The Pointer Lock API technique for nonvisual mouse control opens possibilities for other applications requiring spatial input from blind users. The finding that audio representation can actually clarify certain phenomena better than visual display reinforces the universal design principle that accessibility improvements often benefit all users.
Tags: blindness · sonification · STEM education · interactive simulation · inclusive design · auditory display · physics education · PhET
Standards referenced: W3C Pointer Lock API