Exploring Shared Augmented Reality for Low-Vision Training of Activities of Daily Living
Yong-Joon Thoo, Karim Aebischer, Nicolas Ruffieux, Denis Lalanne · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746371
Summary
This paper investigates the feasibility of using co-located shared augmented reality (S-AR) to support low-vision therapist (LVT)-guided training for people with low vision. While virtual reality has shown promise for practicing orientation and mobility skills, it isolates users from their physical environment and limits natural interaction with therapists. AR bridges this gap by overlaying virtual content onto real-world settings, and shared AR allows multiple users — therapist and client — to see and interact with the same virtual objects simultaneously. The researchers developed a custom platform for the Microsoft HoloLens 2 using Unity, Microsoft's Mixed Reality Toolkit (MRTK), and Photon Unity Networking for synchronization. Working closely with two certified low-vision therapists from a Swiss blind and low-vision association over eight months, they conducted two feasibility studies with two LV participants to evaluate gesture-based AR interactions, then iteratively designed three AR training tasks targeting foundational skills underlying activities of daily living: lateral object placement (hand-eye coordination), depth-based placement (visual-motor coordination with distance variations), and object identification and placement (feature-based matching under time constraints). An exploratory evaluation with eight additional LV participants (diverse conditions including macular degeneration, nystagmus, Stargardt's disease, diabetic retinopathy, and optic neuropathy) assessed accessibility, usability, and engagement.
Key findings
Participants generally found the gesture-based interactions intuitive and the AR tasks engaging, with high usability ratings (ease of use μ=4.75, recognition accuracy μ=4.25 on 5-point scale) and moderate-to-high enjoyment scores (Interest/Enjoyment μ=5.57 on 7-point IMI scale). Gesture controls were rated positively, though some participants experienced frustration when the system failed to register grab attempts, particularly in visually complex scenes. The HoloLens 2's limited field of view (FoV) was a notable constraint — for some participants it encouraged beneficial head-scanning behaviors aligned with their residual vision strategies, while for others it hindered natural coordination between vision and hand movements. Visual feedback mechanisms, particularly white outlines during object selection, significantly improved interaction clarity and confidence. The LVTs valued the platform's ability to observe participant visual strategies in real-time, noting that AR tasks made search patterns more visible than conventional pen-and-paper exercises. They saw the system as a motivating complement to existing training — particularly for clients with recent vision loss — rather than a standalone tool. Key design recommendations included customizable object properties (shape, color, size, contrast), progressive difficulty levels controlled by therapists, tactile anchors (coasters, placemats) to bridge virtual and physical interactions, and therapist-facing tools like a virtual "3D pen" for illustrating strategies.
Relevance
This research opens a promising new direction for low-vision rehabilitation by demonstrating that shared AR can support the collaborative, therapist-guided nature of vision training while adding interactive and motivating elements. The ability-based design approach — leveraging residual vision rather than compensating for loss — aligns with best practices in rehabilitation. For practitioners, the findings highlight important trade-offs between accessibility, task difficulty, and skill development: visual cues that make AR tasks easier may reduce opportunities for developing independent strategies. The shared AR paradigm is particularly significant because it preserves the essential therapist-client relationship while enabling real-time observation of visual strategies that are often invisible in conventional training. The detailed design considerations for diverse visual conditions provide practical guidance for developing AR rehabilitation tools, and the emphasis on progressive, customizable training parameters offers a model for personalized intervention that could extend beyond low vision to other rehabilitation contexts.
Tags: low vision · augmented reality · vision rehabilitation · activities of daily living · shared AR · HoloLens · mixed reality · gesture interaction · assistive technology · ability-based design