Refreshable tactile graphics applied to schoolbook illustrations for students with visual impairment
Grégory Petit, Aude Dufresne, Vincent Levesque, Vincent Hayward, Nicole Trudeau · 2008 · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '08) · doi:10.1145/1414471.1414489
Summary
This paper presents research on making schoolbook illustrations accessible to students with visual impairments using a refreshable tactile display system. The system combines the STReSS2 (Stimulator of Tactile Receptors by Skin Stretch), a device that produces tactile feedback by laterally stretching the skin of the finger through an 8x8 matrix of piezoelectric bending motors, with the Pantograph, a haptic device enabling two-dimensional movement over a 11.3x6 cm exploration surface. The MaskGen software was developed to interactively transpose schoolbook illustrations into tactile graphics: images are first simplified in Photoshop (converting to greyscale, simplifying rendering, thickening borders, enlarging edges), then MaskGen automatically extracts different parts as masks, allowing each part to be assigned a specific tactile feedback (dots, waves, or vibration) and audio feedback (speech synthesis or voice recording). Three scientific illustrations from a general history schoolbook were adapted: a gothic cathedral floor plan (diagram), a bar chart of Athens' population distribution, and a world map showing early civilizations. The system supports multiple levels of information — for the map, two tactile graphic layers (civilizations and continents) could be toggled via keyboard, allowing gradual exploration from overview to detail with no equivalent in traditional tactile materials.
Key findings
Forty participants were tested in two groups: 20 sighted adults (blindfolded) and 20 participants with visual impairment (10 adults, 10 children, average age 15.4 years). No significant difference was found between blindfolded sighted participants and those with visual impairment for total time or exploration time, challenging the assumption that people with visual impairment have inherently better tactile sensation. However, participants with visual impairment reported finding the tactile graphics easier to understand (mean rating 4.0 vs 2.9 on a 1-5 scale, p = 0.001). Children with visual impairment completed tasks faster than adults with visual impairment (5 minutes less on average, p = 0.011) across all three illustrations. Content comprehension scores were good across all groups (69-89% correct), with participants correctly answering questions about the illustrations' pedagogical content. Video analysis identified five distinct exploration strategies: "following outlines" (most common, 215 tags during exploration), "saccade" (jumping between zones), "missed zone" (passing over areas unknowingly), "systematic scan" (top-to-bottom exploration), and "random scan" (ignoring tactile/audio feedback). Voice recordings were slightly preferred over speech synthesis for audio feedback (87% vs 79% comprehension, p = 0.046). Adding undulation texture inside shapes helped users identify global forms faster (p = 0.014) but had limited benefit overall.
Relevance
This study demonstrates that refreshable tactile displays can effectively convey the pedagogical content of schoolbook illustrations, addressing a critical gap in educational accessibility. While text can be transposed to Braille and digital text accessed via screen readers, the numerous illustrations in schoolbooks remain largely inaccessible — current methods using embossed or thermoform paper are slow to produce, static, and deteriorate with use. The refreshable approach offers significant advantages: multiple levels of detail can be layered and toggled, audio descriptions complement tactile exploration, and graphics can be updated dynamically. For accessibility practitioners, the identified exploration strategies provide valuable design guidance for tactile interfaces. The finding that children explored faster than adults suggests that younger users may adapt more readily to novel tactile technologies. The MaskGen tool's relatively quick preparation time (5-15 minutes in Photoshop plus 5 minutes in MaskGen) makes the approach potentially scalable for educational settings. The research also highlights the importance of multimodal feedback — combining tactile rendering with audio descriptions — as participants found audio feedback essential even when tactile rendering alone conveyed shape information.
Tags: tactile graphics · visual impairment · education · haptic technology · refreshable displays · children · multimodal interaction · STEM accessibility · inclusive education