Creating Accessible Online Floor Plans for Visually Impaired Readers
Anuradha Madugalla, Kim Marriott, Simone Marinai, Samuele Capobianco, Cagatay Goncu · 2020 · ACM Transactions on Accessible Computing · doi:10.1145/3410446
Summary
This paper presents a generic model for converting online graphics—specifically house floor plans—into accessible formats for blind and severely vision-impaired users. The model supports both fully automatic and semi-automatic transcription, allowing sighted users to correct recognition errors without needing transcription expertise. The system outputs three presentation formats: text-only descriptions suitable for screen readers or braille displays, embossed tactile graphics printed with braille labels, and touch-controlled audio presentations using the GraVVITAS app on touchscreen devices. The researchers built a floor plan recognition system using OpenCV and Python (21,500 lines of code) that processes raster images through nine stages: preprocessing, text identification via Tesseract OCR, wall/door/window/stair recognition, external wall closing, closet detection, room identification, open plan partitioning, and high-level JSON description generation. A formative user study with five blind participants informed the design, revealing that users wanted information about building shape, size, orientation, room count and types, spatial arrangement, entrances, and room connectivity. Two datasets were used for evaluation: the UAB-CVC corpus (90 architectural floor plans) and a new RSVG dataset (100 diverse real-world floor plans).
Key findings
The recognition system achieved high accuracy on standardized floor plans (UAB-CVC: 91% precision for doors, 86% for windows, 96% for walls) but struggled with the diverse RSVG dataset due to varying graphical conventions. A crucial finding was that current graphics recognition cannot achieve 100% accuracy because floor plans use widely varying notational conventions—the system cannot anticipate every drawing style. To address this, the researchers implemented confidence levels (high, medium, low) for recognized elements, with uncertain items labeled "maybe" or "unknown element." In a user study with eight blind participants, all three presentation formats proved useful for route-finding and general layout questions. Participants preferred graphical representations (tactile or GraVVITAS) over text for complex spatial tasks, though text was more accurate for room connectivity questions. Critically, participants appreciated being informed about uncertainty—they could use contextual reasoning to deduce what unknown elements might be (e.g., inferring an unknown room was a bathroom because "there is no other bathroom in the house"). Users suggested adding room dimensions to help identify unknown rooms.
Relevance
This research addresses a significant accessibility gap: floor plans on real estate websites are almost entirely inaccessible to blind users, yet essential for home searching, navigation planning, and understanding building layouts. The multi-format output approach is exemplary—different situations call for different modalities, and users have individual preferences. The finding about trust and uncertainty is particularly important for the growing field of AI-powered accessibility tools: users would rather know when information is uncertain than receive confident but incorrect data. The semi-automatic approach, where non-expert sighted helpers can correct recognition errors, offers a practical middle ground between fully manual transcription (expensive, slow) and fully automatic recognition (error-prone). For practitioners developing accessible graphics tools, this paper provides a solid template for structuring text descriptions of spatial information and demonstrates that blind users can build effective mental models from multiple presentation formats.
Tags: visual impairment · blindness · floor plans · tactile graphics · image recognition · accessible graphics · navigation · automatic transcription
Standards referenced: North American Braille Authority Guidelines for Tactile Graphics · American Council for the Blind Audio Description Guidelines