Tangibles + Programming + Audio Stories = Fun
Varsha Koushik, Shaun K. Kane · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17) · doi:10.1145/3132525.3134769
Summary
This poster introduces Story Blocks, a tangible programming toolkit designed to make block-based programming accessible to blind and visually impaired users. Block-based programming languages like Scratch and Blockly are popular tools for teaching children to code, but they present two accessibility barriers: first, they rely on visual drag-and-drop interaction that screen readers cannot easily translate; second, their output is typically visual animations or games that are inaccessible to blind learners. Story Blocks addresses both problems by using physical blocks with tactile features that represent story characters and actions — users assemble a program by stacking blocks in sequence, then scan the arrangement with a mobile device camera. The program is executed on the phone as an interactive audio story. For example, blocks representing a cat, a mouse, and actions like "hisses," "runs," and "eats" can be assembled to create a narrative that plays as spoken audio with sound effects. The blocks use distinct tactile shapes (raised icons of animals, actions) so that both blind and sighted users can identify them by touch. The system requires no specialised hardware beyond the blocks themselves and a standard smartphone, making it potentially scalable for classroom use.
Key findings
Story Blocks demonstrates a novel approach to accessible programming education that addresses both input and output accessibility simultaneously. By using tangible blocks as input instead of screen-based drag-and-drop, the system removes the visual interaction barrier. By producing audio stories rather than visual animations as output, it creates engaging program results that are inherently accessible. The system builds on prior work in tangible programming (Strawbies, Project Bloks, Project Torino) but differentiates itself by using tactile features readable by blind users and by not requiring sophisticated embedded electronics in each block — instead using computer vision to recognise block arrangements. This makes it more scalable and affordable than systems like Project Torino, which requires hardware in each block. The paper is a short poster presenting the concept and prototype rather than a formal evaluation with users.
Relevance
Story Blocks addresses a critical gap in STEM education accessibility. Programming literacy is increasingly important, yet most introductory programming tools exclude blind and visually impaired children. For accessibility practitioners and educators, the key insight is that making programming accessible requires rethinking both the input modality (how programs are created) and the output modality (what programs produce). The tangible-input, audio-output approach creates an inclusive environment where blind and sighted children can program together using the same tool. While this is an early-stage poster without formal user evaluation, the concept points toward important design principles for accessible educational technology: use multiple modalities, leverage existing consumer hardware (smartphones) rather than specialised equipment, and create output formats that are engaging for all users rather than retrofitting visual output with accessibility features.
Tags: programming education · blindness · visual impairment · tangible interaction · children · inclusive design · STEM accessibility · block-based programming