All the Way There and Back: Inertial-Based, Phone-in-Pocket Indoor Wayfinding and Backtracking Apps for Blind Travelers
Chia Hsuan Tsai, Fatemeh Elyasi, Peng Ren, Roberto Manduchi · 2024 · ACM Transactions on Accessible Computing · doi:10.1145/3696005
Summary
This paper presents two iOS apps designed to help blind travelers navigate indoor building environments without requiring external infrastructure like Bluetooth beacons or visual markers. The Wayfinding app uses a known floor plan to compute and guide users along the shortest route to a destination, while the Backtracking app records the path taken during an outbound journey and provides guidance to retrace it in reverse—useful when no map is available. The key innovation is that both apps rely solely on the smartphone's inertial sensors (accelerometer and magnetometer) for localization, allowing users to keep their phone in a pocket rather than holding it to maintain camera visibility. This "phone-in-pocket" design is crucial for blind travelers who typically use both hands for a long cane or dog guide. Users interact with the apps through an Apple Watch, receiving speech notifications via bone conduction headset and controlling the apps through crown rotation and swipe gestures. The system employs two Pedestrian Dead Reckoning (PDR) algorithms running in parallel: A/S (Azimuth/Steps), which counts steps and tracks heading using an LSTM neural network, and RoNIN, a machine learning approach that produces velocity vectors independent of phone orientation. A Particle Filter constrains the estimated trajectory to physically plausible paths within the building layout. For backtracking, the app uses magnetic field signatures and graph-based sequence alignment to match the return path with the recorded outbound route.
Key findings
The apps were tested with seven blind participants (ages 53-76, median 72) who navigated three routes totaling 292 meters with 13 turns in a campus building. Key results: Wayfinding Performance: All participants successfully completed the prescribed routes, though the first three participants required a system restart on route R1W due to incorrect step length measurements (the app was subsequently modified to adaptively track step length). Average walking speed was 0.50 m/s. The 90th percentile localization error was 3.4m for A/S and 3.75m for RoNIN. Backtracking Performance: The Backtracking app worked in most cases, but failed in 6 of 21 trials (29%), primarily due to magnetic field variability across corridor widths or participants taking incorrect turns and walking too far from the recorded path before the error could be corrected. User Experience: The System Usability Scale score was 80.36 (90th percentile). All participants found the notifications understandable and "just right" in quantity. The early advance notification of upcoming turns (issued 7m before waypoints) was generally well-received, though some participants missed turns when walking fast or distracted. Participants with dog guides faced unique challenges as dogs would sometimes refuse to turn or walk past junctions before processing the notification. Participant feedback indicated the apps could make them feel "safer and more confident" when traveling alone. Several described practical use cases like navigating medical buildings or conference venues.
Relevance
This research addresses a significant gap in accessible indoor navigation. While GPS-based outdoor navigation is mature, indoor spaces remain challenging because GPS signals are unavailable and existing solutions require either expensive infrastructure (BLE beacons) or camera-based systems that force users to hold their phones in view. The inertial-only approach enables true hands-free navigation. For practitioners and O&M specialists, the study validates that speech-based turn-by-turn directions with early advance notice are acceptable to blind travelers, even when localization accuracy is limited (2-3m errors). The UI design principles—consistency between apps, robustness to localization inaccuracy, minimal disruption through short notifications, and Watch-based control—offer a template for accessible navigation interfaces. Limitations include the requirement for a known starting position and orientation, challenges in very large open spaces where the Particle Filter cannot constrain drift, and the 29% failure rate for backtracking. The participant sample skewed older (median 72 years) and included only two dog guide users. Future work should explore hybrid systems combining inertial sensing with sporadic camera-based landmark recognition for automatic position resets.
Tags: indoor navigation · wayfinding · blind and low vision · inertial sensors · dead reckoning · mobile apps · orientation and mobility · smartwatch