Providing Haptic Feedback Using the Kinect
Brandon T. Shrewsbury · 2011 · The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2049536.2049628
Summary
This poster paper presents a system that combines the Microsoft Kinect depth camera with a custom haptic glove to provide spatial awareness to people who are blind or visually impaired. The research addresses a gap in existing mobility aids: while canes detect obstacles within a small radius and guide dogs navigate diverse environments, neither provides information about the distance, height, and width of objects at moderate distances. The system repurposes the Kinect — an inexpensive, widely available gaming peripheral — as a depth-sensing camera that works even in low-light conditions. The architecture has three components: the Kinect captures depth data via its RGBD (Red Green Blue plus Depth) camera, middleware processes the depth image by converting it to grayscale and resizing it to match the haptic array dimensions using a nearest-neighbor algorithm, and an Arduino-controlled haptic glove delivers tactile feedback to the user. The haptic device uses a 4x4 matrix of actuators spanning the ventral side of the fingers, chosen because this area has a dense concentration of mechanoreceptors for fine tactile discrimination.
Key findings
The system successfully translates environmental depth information into haptic signals that can be felt through the fingertips. The middleware converts Kinect depth data to signal levels recognizable by the haptic device, downscaling to 1 byte per pixel to reduce transmission times and applying low and high limits to filter unwanted depth data. The processed information is transmitted wirelessly to the haptic glove. The design draws on two prior approaches: tactile displays developed for the lower back and haptic gloves developed for wayfinding, combining them into a finger-mounted array that leverages the hand's superior tactile sensitivity. By using a low-cost gaming peripheral rather than specialized stereoscopic vision hardware, the system aims to be affordable and accessible to a broad population of visually impaired users.
Relevance
This research demonstrates the potential of repurposing consumer gaming technology for accessibility applications, an approach that has become increasingly relevant as depth-sensing cameras have proliferated in smartphones and other devices. The concept of translating visual depth information into tactile feedback represents a practical application of sensory substitution — replacing visual perception with touch to convey spatial relationships. For accessibility practitioners and assistive technology developers, the work highlights that effective aids do not require expensive specialized hardware; creative use of mainstream consumer devices can produce functional prototypes at a fraction of the cost. The finger-based haptic array design is notable for exploiting the hand's natural tactile acuity, though questions remain about real-world usability, user training requirements, and how the system performs in dynamic outdoor environments.
Tags: haptic technology · sensory substitution · blindness and low vision · assistive technology · depth camera · navigation · wearable technology