Interactive Tracking of Movable Objects for the Blind on the Basis of Environment Models and Perception-Oriented Object Recognition Methods
Andreas Hub, Tim Hartter, Thomas Ertl · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169007
Summary
This paper from the University of Stuttgart presents advances to an indoor navigation and object identification system for blind and deafblind users. The system combines a stereo camera and 3D inertial sensor mounted on a bicycle helmet with detailed 3D environment models of buildings to provide self-localization, object identification, and navigation assistance. Building on previous work that handled only static objects stored in pre-built 3D models, this paper introduces methods for tracking three types of movable objects: partly-fixed objects (like doors), free movable objects (like chairs), and people. Door state (open/closed) is determined by comparing stereo camera distance measurements against the 3D model. Free movable objects are recognized using a perception-inspired algorithm that matches shape outlines and color histograms against a trained object database, using the CamShift algorithm from OpenCV. People are detected using the Viola-Jones face detection algorithm, with virtual human models inserted into the 3D environment to enable spatial tracking. All information — object names, distances, and positions — is conveyed acoustically or through tactile means.
Key findings
The system successfully demonstrated real-time tracking of movable objects in indoor environments. Door position recognition was statistically unambiguous when doors were fully open or fully closed, with approximately 50% accuracy during the opening process. Free movable object recognition (tested with chairs) typically completed within one to two seconds under good lighting conditions, operating at 10-30 frames per second depending on scene complexity. The stereo camera achieved distance measurement accuracy of about 10 cm at one meter and 20 cm at distances under six meters — sufficient for typical indoor navigation. The system could distinguish objects by shape and color but could not differentiate identical objects of the same type (e.g., two matching chairs). Face detection and tracking worked when faces were oriented roughly toward the camera, with position errors typically under 0.5 meters. The researchers noted that users who tested the prototype found the bicycle helmet form factor acceptable, reporting that gains in functional mobility and independence outweighed concerns about the system's appearance. Current limitations include a four-hour battery life and restriction to single-object tracking due to laptop computing constraints.
Relevance
This research addresses a fundamental challenge in orientation and mobility for blind people: while blind individuals can navigate familiar environments with remarkable skill, the position of movable objects like chairs, doors, and people creates unpredictable hazards that traditional mobility aids (white canes, guide dogs) handle only at close range. The system's approach of combining pre-built 3D environment models with real-time computer vision to detect changes from the expected environment is architecturally significant — it mirrors how sighted people navigate by noticing what has changed rather than processing everything from scratch. The work also pioneered the concept of providing blind users with spatial awareness of other people's locations, a social accessibility feature that goes beyond obstacle avoidance. Although the hardware was bulky by modern standards, the underlying approach of model-based indoor navigation with movable object tracking anticipated many capabilities now being explored with smartphones and AR glasses for blind navigation assistance.
Tags: indoor navigation · object recognition · blind users · computer vision · assistive technology · self-localization · 3D modeling · face detection · orientation and mobility