PlatMult: A Multisensory Platform with Web Accessibility Features for Low Vision Users
Marcio Oyamada, Jorge Bidarra, Clodis Boscarioli · 2013 · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2513383.2513405
Summary
This poster paper presents PlatMult, an integrated hardware and software platform that combines three sensory modalities — visual, auditory, and tactile — to improve web accessibility for low vision users. The authors note that screen magnifiers, the most common assistive technology for low vision, simply amplify the desktop without modifying content, leaving significant limitations for web access. PlatMult addresses this by integrating three components: the xLupa screen magnifier (which allows configuration of brightness, font size, colour, and spacing), a screen reader using text-to-speech, and a custom mouse with motor (vibration) feedback. These components work together through the AT-SPI (Assistive Technology Service Provider Interface) accessibility API on Linux — when a user moves the mouse over links and buttons on a webpage, the system simultaneously activates the screen reader to announce the element and triggers the mouse motor feedback to provide tactile confirmation. The platform is built entirely on free software: Ubuntu, GNOME, Xorg graphics server, and the Firefox browser.
Key findings
The key technical contribution is the integration architecture that coordinates three feedback channels in response to web interaction events. The system uses AT-SPI to capture accessible events from the Firefox browser — when a user interacts with components like menus, buttons, or text boxes, these events are distributed to the screen magnifier, screen reader, and feedback mouse simultaneously. The xLupa screen magnifier provides configurable visual enhancement, while the screen reader adds auditory information, and the adapted mouse provides tactile confirmation through vibration when hovering over interactive elements. This multimodal approach means that low vision users receive redundant cues across multiple senses, reducing reliance on any single modality. The platform targets low-cost deployment, making it suitable for interactive kiosks and public computers as well as personal use. The entire implementation uses open-source components, lowering barriers to adoption particularly in developing countries.
Relevance
PlatMult addresses an important gap in low vision assistive technology: most solutions provide only one modality of feedback (visual magnification or audio), forcing users to rely entirely on that single channel. By combining visual, auditory, and tactile feedback in an integrated system, PlatMult offers a more robust interaction experience where multiple senses reinforce each other. The haptic mouse feedback is particularly notable — vibration when hovering over interactive elements provides spatial awareness that neither magnification nor screen reading alone can offer efficiently. For accessibility practitioners, this work illustrates the value of multimodal design and the potential of the AT-SPI accessibility API for building cross-application assistive tools on Linux. The paper is a brief poster with no formal user evaluation reported, which limits assessment of its practical effectiveness, but the architectural approach of coordinating multiple assistive technologies through a shared accessibility API is sound.
Tags: low vision · screen magnifier · screen reader · haptic feedback · multisensory · assistive technology · web accessibility · open source