A11yShape: AI-Assisted Accessible 3-D Modeling for Blind and Low-Vision Programmers
Zhuohao (Jerry) Zhang, Haichang Li, Chun Meng Yu, Faraz Faruqi, Junan Xie, Gene S-H Kim, Mingming Fan, Angus Forbes, Jacob O. Wobbrock, Anhong Guo, Liang He · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746362
Summary
This paper presents A11yShape, a novel system that makes 3-D modeling accessible to blind and low-vision (BLV) programmers through code-based modeling with AI assistance. The system integrates with OpenSCAD, a programmatic 3-D modeling environment where users write code to define geometric shapes and operations. A11yShape addresses a fundamental challenge: while code-based modeling removes the need for visual direct manipulation, BLV users still struggle to understand and verify the spatial output of their code without visual feedback. The system architecture consists of three interconnected panels: a Code Editor with standard editing features, an AI Assistance Panel that generates textual descriptions of 3-D models using GPT-4o from multiple camera angles, and a Model Panel displaying a semantic hierarchy of model components. A key innovation is cross-representation highlighting, which synchronizes selections across code segments, semantic hierarchy nodes, AI-generated descriptions, and 3-D renderings, allowing users to maintain correspondence between what they write and what the model looks like. The system also includes version control for tracking incremental changes and an AI verification loop where users can query the AI about specific aspects of their model. The research was conducted through participatory design with a BLV co-author who uses OpenSCAD, ensuring authentic representation of user needs throughout the development process.
Key findings
A validation study with 15 sighted participants rated AI-generated descriptions highly across accuracy (4.11/5), completeness (4.19/5), spatial relationships (4.33/5), and helpfulness (4.52/5). A multi-session user study with four BLV programmers (three sessions each, approximately 7.5 hours total per participant) demonstrated that participants could independently create 12 distinct 3-D models ranging from simple geometric compositions to complex real-world objects like a snowman or chess piece. The system achieved a mean System Usability Scale score of 80.6, indicating good usability. Participants developed three distinct workflows with varying levels of AI reliance: code-first with AI verification, AI-guided iterative refinement, and hybrid approaches. Key strategies included incremental building through AI-verification loops, using semantic hierarchy for error correction, and leveraging real-world metaphors to construct mental models. However, significant challenges emerged including high cognitive load from managing multiple dynamic representations simultaneously, difficulty understanding complex spatial relationships through text alone, challenges constructing and maintaining accurate mental models without tactile feedback, and persistent uncertainty about whether operations succeeded as intended.
Relevance
A11yShape demonstrates that AI can serve as an effective bridge between code and spatial understanding for BLV users, opening up creative domains that have been largely inaccessible. The cross-representation highlighting approach offers a generalizable design pattern for any tool where users need to understand relationships between abstract representations and concrete outputs. For accessibility practitioners, the finding that text descriptions alone are insufficient for full spatial comprehension reinforces the need for multimodal feedback, potentially including tactile or haptic approaches. The participatory design methodology, with a BLV co-author embedded in the research team, exemplifies inclusive research practice. The identified challenge of cognitive overload from managing multiple representations simultaneously has implications for designing any complex accessible interface and suggests that progressive disclosure and simplified views should be standard accessibility considerations.
Tags: 3-D modeling · blind and low-vision · accessible programming · AI assistance · OpenSCAD · cross-representation · spatial understanding · participatory design · code-based modeling
Standards referenced: WCAG 2.1