NavCog3: An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment
Daisuke Sato, Uran Oh, Kakuya Naito, Hironobu Takagi, Kris Kitani, Chieko Asakawa · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17) · doi:10.1145/3132525.3132535
Summary
This paper presents NavCog3, a smartphone-based indoor navigation assistant for people with visual impairments, evaluated at scale in a 21,000 m² shopping mall in Japan. Unlike many prior indoor navigation prototypes that were tested only in constrained lab environments, NavCog3 was deployed with approximately 220 Bluetooth Low Energy (BLE) beacons across three connected buildings and a subway station, enabling navigation to about 100 shops. The system uses a hybrid localization approach combining BLE beacon fingerprinting with pedestrian dead reckoning (PDR) via the phone's inertial sensors, achieving average localization accuracy of 1.65 m — comparable to the best smartphone-based indoor systems. NavCog3 distinguishes itself from prior systems by providing not just turn-by-turn directions but also semantic features about the environment: landmarks (tactile paving, doorways, obstacles), floor transition information (elevator button locations, Braille availability, escalator directions, stair shapes), and points of interest (shops, restaurants with cuisine type, restrooms with accessibility info, ATMs). A LIDAR-based fingerprinting machine reduced map creation time by 1/20th compared to manual methods. The system also includes a conversation-based cognitive assistant using IBM Watson that allows users to search for destinations by natural language (e.g., "I want to eat Italian food with my young son"). Two studies were conducted: a controlled study with 10 visually impaired participants navigating three fixed routes, and a free-navigation study with 43 visually impaired participants choosing their own destinations.
Key findings
In Study 1 (fixed routes), all 10 participants completed all three routes successfully, with average total task time of 990.5 seconds and total distance of 450 m. Turn success rate was 85.0% without correction and 93.5% with the system's fail-safe guidance or self-correction. Regular 90° turns had significantly higher success rates (89.4%) than slight 45° turns (78.0%), χ²=6.245, p=.012. All participants found semantic features useful, particularly tactile paving information, obstacle alerts, and elevator details (button locations, Braille availability). Seven participants reported that semantic features helped them build spatial maps of the environment. However, seven participants also noted that the amount of guidance could increase cognitive load and decrease awareness of surroundings, raising safety concerns. In Study 2 (free navigation), 43 participants navigated to destinations of their choice, selecting 52 unique POIs across 188 trips. Average trip distance was 152 m and average duration was about 5 minutes. Participants reacted positively to landmark and POI information, though preferences varied by mobility aid — guide dog users found tactile paving and obstacle information less useful since their dog handles these. Twenty-five of 43 participants said NavCog3 would enable them to walk independently in unfamiliar places. Participants suggested an exploration mode (rich POI info) vs. direction-only mode, distance in steps rather than metres, and integration with external services like Google Maps.
Relevance
NavCog3 represents one of the most comprehensive real-world evaluations of indoor navigation for blind users, tested with 53 total participants in a genuine commercial environment rather than a controlled lab. For accessibility practitioners and urban planners, the key findings are: (1) BLE beacon-based localization at 1.65 m accuracy is sufficient for turn-by-turn indoor navigation when combined with fail-safe guidance; (2) semantic features about surroundings are valued by users for both navigation and spatial awareness, but the type of useful information varies by mobility aid (white cane vs. guide dog); (3) cognitive load from continuous audio guidance is a real concern that must be balanced against spatial awareness and safety; and (4) users want the system to complement rather than replace their existing O&M skills. The infrastructure cost (~220 beacons at each plus LIDAR fingerprinting) demonstrates feasibility for large venues. Limitations include that the study was conducted in Japan with Japanese-speaking participants, potential exposure effects from prior NavCog versions, and the single-session design that doesn't capture long-term learning.
Tags: indoor navigation · blindness · visual impairment · wayfinding · Bluetooth beacons · orientation and mobility · smartphone · assistive technology · semantic features