Feel-It: Personalized Audio-Tactile Web Browsing
Anatoliy Borodin, Yevgen Borodin, Andrii Soviak, Vikas Ashok, Shirin Disfani, I. V. Ramakrishnan · 2019 · Proceedings of the 16th International Web for All Conference (W4A) · doi:10.1145/3315002.3332441
Summary
This demonstration paper presents Feel-It, a hardware-software system for audio-tactile web browsing that enables blind users to explore the two-dimensional spatial layout of web pages through touch. The core problem addressed is that screen readers reduce web pages to a one-dimensional linear sequence of text, losing the semantic information encoded in 2D visual layout — section positioning, relative sizing, spatial grouping — that sighted users instantly perceive. Even touchscreen exploration only reveals content one element at a time with no spatial context. Feel-It uses a custom "mouse" shaped like a human hand with four adjustable finger extensions, each fitted with a Braille cell (2x4 pin matrix with piezoelectric actuators) aligned with the fingertip's most sensitive area. Infrared LEDs on each finger are tracked by an overhead IR camera, and OpenCV resolves finger positions to map them onto a segmented representation of the web page. As the user moves the mouse across a working area (approximately 80x60cm with the camera 80cm above the table), the Braille cells dynamically render tactile representations of webpage structure — horizontal and vertical lines for section borders, angled lines, and curves. The system uses established web segmentation algorithms (like VIPS) to decompose pages into a hierarchy of semantic segments similar to the HTML DOM tree structure. This provides up to 320x240 pin resolution within the working area, far exceeding dedicated pin-matrix displays like BrailleDis (120x60) while being more portable and affordable.
Key findings
Feel-It provides extensive personalization options across hardware and software. Hardware personalization includes adjustable finger extension lengths and a sliding hand stabilizer to accommodate different hand sizes. The working area dimensions can be customized by adjusting camera distance or programmatically, and its orientation can be changed. Software personalization covers three feedback modalities that can be combined: earcons (customizable for events like entering/exiting page segments, page navigation, or leaving the working area), haptic feedback (configurable for all four fingers, two fingers, or one finger, with adjustable segment density and tactile border thickness), and text-to-speech (triggered on demand via button press or automatically when entering new segments or hovering over text). The combination allows users to quickly scan a page's spatial structure through touch, then selectively listen to content they find interesting — reversing the screen reader paradigm where users must listen to everything sequentially to find what matters. The system can connect to any device, providing a consistent personalized interface across different form factors. Future plans include dual-mouse support for both hands and virtual keyboard input capabilities.
Relevance
Feel-It represents an ambitious attempt to fundamentally change how blind people experience the web by restoring access to spatial layout information that screen readers discard. The approach recognizes that the 2D structure of web pages carries meaningful semantic information — navigation bars are at the top, related content is grouped together, important elements are larger and more prominently placed — and that losing this information makes web browsing significantly more difficult and time-consuming for blind users. For accessibility practitioners, the system demonstrates that the screen reader paradigm of linearizing content, while functional, represents a significant loss of information that haptic technology could help recover. The personalization features are particularly thoughtful, acknowledging that different users have different tactile sensitivity, preferences, and strategies for exploring content. While Feel-It requires specialized hardware that limits immediate adoption, it points toward a future where multimodal (audio + tactile) web browsing could give blind users a richer, more efficient browsing experience that better parallels the visual scanning that sighted users take for granted.
Tags: blindness · haptic feedback · tactile display · web browsing · screen reader · assistive technology · personalization · computer vision · Braille · web segmentation