ForeSee: A Customizable Head-Mounted Vision Enhancement System for People with Low Vision
Yuhang Zhao, Sarit Szpiro, Shiri Azenkot · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2700648.2809865
Summary
This paper presents ForeSee, a video see-through augmented reality system that enhances the visual experience for people with low vision. Unlike assistive technologies that bypass vision entirely (like screen readers), ForeSee leverages users' remaining functional vision—which the vast majority of the estimated 19 million Americans with visual impairments possess. The system captures real-time video through a webcam, applies image processing enhancements, and displays the modified view through an Oculus Rift head-mounted display. ForeSee offers five enhancement methods: Magnification (1-35x), Contrast Enhancement (increased luminance and color contrast), Edge Enhancement (using Canny edge detection to emphasize contours), Black/White Reversal (Otsu thresholding for high-contrast binary display), and Text Extraction (OCR using Tesseract to display recognized text). These can be applied in two display modes: Full Display (enhancement across entire field of view) or Window Display (enhancement in an adjustable rectangular region, preserving peripheral context). Users control the system through natural speech commands via a Wizard of Oz implementation. The researchers evaluated ForeSee with 19 participants representing remarkable diversity in low vision conditions including nystagmus, optic atrophy, Stargardt's disease, retinitis pigmentosa, Leber's congenital amaurosis, tunnel vision, macular degeneration, and more. Participants performed near-distance (reading a printed page) and far-distance (reading signs at 3 meters) tasks with various enhancement combinations.
Key findings
The central finding is that customization is essential—no two participants chose the same enhancement combination for both tasks, and preferences varied not only between people with different conditions but also among people with similar conditions. Magnification was the most popular method (used by 14-15 of 19 participants), but its effectiveness varied: it helped users see details but narrowed field of view and slowed target identification, particularly impacting those with tunnel vision. Window Display Mode proved especially valuable for far-distance tasks, with 12 of 17 eligible participants preferring it because it helped them concentrate on targets while maintaining environmental context. Participants described using it like a "flashlight" to highlight specific areas. However, some found switching gaze between the enhanced window and surrounding area disorienting. Edge enhancement generated polarized responses: some found it helpful for defining letter boundaries, while others (especially those with more functional vision) felt it created a "crowding effect" making text harder to read and obscured color information. Black/white reversal was popular for near-distance reading (reducing glare) but largely rejected for far-distance viewing where it distorted environmental details. Text extraction was preferred for far-distance signs (12/17) but not for reading continuous text (6/19) due to the impracticality of word-by-word extraction. Most participants (15/19 for near-distance, 14/19 for far-distance) felt that combining multiple enhancement methods produced the best results—for example, magnification alone was insufficient, but combining it with edge enhancement helped distinguish numbers and letters.
Relevance
This research challenges the one-size-fits-all approach that has characterized many low vision aids. The finding that people with similar diagnosed conditions often preferred different enhancement methods suggests that clinical diagnosis alone cannot predict optimal assistive technology configuration. This has implications for how low vision devices should be designed and prescribed—extensive customization options and the ability to quickly adjust settings for different tasks are not luxury features but essential functionality. The Window Display Mode represents a significant design innovation over existing commercial systems like eSight, which only offer full-field enhancement. The ability to enhance a specific region while preserving peripheral awareness supports both focused tasks (reading) and environmental awareness (navigation). Participants' creative uses—like using the window for multitasking or to provide context during magnified viewing—demonstrate how flexible tools enable users to develop personalized strategies. For practitioners, the detailed participant feedback provides actionable guidance: edge enhancement helps some users define letters but creates crowding for others; contrast enhancement is beneficial in windowed mode but can cause eye strain in full mode; magnification level should auto-scale with window size. The study also validates that consumer AR hardware (Oculus Rift, webcams) can serve as platforms for low vision aids, potentially making advanced enhancement technology more affordable than specialized medical devices like the ,000 eSight.
Tags: low vision · augmented reality · head-mounted display · vision enhancement · magnification · assistive technology · customization