Interactive Tactile Display System: A Support System for the Visually Disabled to Recognize 3D Objects
Yoshihiro Kawai, Fumiaki Tomita · 1996 · Proceedings of the Second Annual ACM Conference on Assistive Technologies (Assets '96) · doi:10.1145/228347.228356
Summary
This paper from the Electrotechnical Laboratory in Tsukuba, Japan describes an interactive tactile display system that combines stereo computer vision with a pin-based tactile display and voice synthesis to help blind users recognize three-dimensional objects and environments. The system has three components: a stereo camera that captures 3D data from real-world scenes using two pan-tilt 3CCD cameras, an image processing pipeline that performs object recognition by matching against a database of known objects, and a 16x16 pin tactile display with variable pin heights (0-6mm in multiple levels) driven by stepping motors. Each pin also contains a tact switch at its base, allowing the user to press individual pins to query the system about what is at that position. A voice synthesizer provides supplementary auditory information about identified objects, including their name, size, colour, and position. The system uses two stereo methods — edge-based reconstruction for 3D boundary data and isoluminance-contour-based reconstruction for surface data — to extract both shape outlines and surface geometry from the visual scene.
Key findings
The system provides three interactive display modes that allow progressive exploration of a scene. In position mode, each recognized object is represented by a single raised pin at its relative position on the display, giving an overview of the spatial layout; pressing a pin triggers a voice description of that object. In boundary shape mode, the wireframe outline of a selected object is displayed across all 16x16 pins, allowing the user to feel its 2D boundary shape. In surface shape mode, pin heights represent the 3D surface depth of the object, enabling the user to feel convex and concave shapes. The user can switch between modes using three selection keys. The variable pin height is a key innovation over conventional two-level (raised/lowered) tactile displays, as it enables representation of actual 3D surface topology rather than just 2D outlines. The authors note that while the 16x16 resolution is limited, the multi-level display modes and interactive querying compensate by allowing users to explore different aspects of objects on demand. The system was still at the prototype stage, with plans for user experiments with visually impaired participants.
Relevance
This paper represents an early integration of computer vision and tactile output for environmental awareness — a concept that has evolved into modern camera-based assistive technologies like smart glasses and AI-powered scene description apps. The multi-mode interaction design (overview, boundary, surface) demonstrates a principle that remains central to accessible information design: providing information at multiple levels of detail and letting the user control which level they explore. The variable-height pin array foreshadowed modern refreshable tactile displays with higher resolutions now being developed for braille and graphics. For practitioners, the system illustrates how combining multiple output modalities (tactile and auditory) and multiple representation modes provides richer access than any single modality alone. The interactive pin-pressing input mechanism is an early example of bidirectional tactile interfaces where the same surface serves for both input and output, a concept relevant to current work on accessible touchscreens and haptic interfaces.
Tags: blindness and low vision · tactile display · stereo vision · computer vision · 3D object recognition · haptic technology · assistive technology · refreshable display