TIP-Toy: a tactile, open-source computational toolkit to support learning across visual abilities
Giulia Barbareschi, Enrico Costanza, Catherine Holloway · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020) · doi:10.1145/3373625.3417005
Summary
This paper presents TIP-Toy (Tactile Inclusive Programming Toy), an open-source educational programming toolkit using physical blocks and music output, designed for children ages 6-11 with mixed visual abilities — from full sight to total blindness. The fundamental problem is that mainstream computational education toolkits (Scratch, Blockly, Snap) rely heavily on visual GUIs, making them inaccessible to children with visual impairments. Even existing accessible alternatives like Torino (Microsoft Research) and StoryBlocks have limitations: Torino uses active electronic beads that are costly and not open-source, while StoryBlocks is not publicly available. TIP-Toy uses passive 3D-printed blocks with QR-code markers read by an overhead webcam (Logitech C920 HD), connected to a Raspberry Pi running custom Python software. Blocks represent programming concepts: Play (plays a note), Pause, Repeat/Start Loop, End Loop, and Variable blocks (containing data values like note assignments or loop counts). Variable blocks slot inside larger function blocks, creating an intuitive physical metaphor for programming concepts. The workspace accommodates up to 10 blocks on a magnetic A3 sheet with fixed reference markers. Two buttons provide key interactions: "Run" executes the program as music, and "Read Aloud" narrates the code as pseudocode — serving as a debugging tool and a bridge toward screen reader literacy. The design was informed by interviews with 3 visually impaired adult programmers who also mentor VI children, followed by testing with 9 children (ages 6-10; 2 blind, 2 partially sighted, 5 sighted) across 5 sessions.
Key findings
All 9 children, regardless of visual ability, successfully interacted with TIP-Toy — recognizing blocks by touch or sight, combining them into sequences, and listening to musical outputs. Children adopted different strategies: some methodically examined blocks before building (Bob and Harry tactilely inspected all pieces first), others used trial-and-error discovery learning (Peter and Thomas placed blocks and used Run/Read Aloud to understand the system). Through four structured activities (piano scales, completing "Twinkle Twinkle Little Star," recreating melodies, building "Row Your Boat" with loops), children learned sequences, events, loops, data variables, and the computational practices of experimenting/iterating and testing/debugging. Collaboration between children of different visual abilities emerged naturally: in one pair, Harry (partially sighted) handled spatial tasks like arranging blocks on the workspace while Donald (blind) identified blocks tactilely and recognized musical pitches, creating a complementary partnership. The "Read Aloud" button proved particularly valuable for debugging, helping children identify misaligned or incorrectly oriented blocks. Children displayed creativity beyond the structured activities — Wayne and Ben discovered they could create repeat effects by pressing Run immediately after a melody finished, and Martin mixed Run and Read Aloud outputs for a "DJ" effect. Challenges included VI children struggling with workspace boundaries and locating the buttons box when it was not fixed to the workspace edge. The paper frames findings using Resnick and Silverman's "low floors, high ceilings, wide walls" framework and Alper et al.'s accessibility elaboration (ramps, ladders, frames of interest, reinforced corners).
Relevance
TIP-Toy demonstrates that inclusive computational education — where children with and without visual impairments learn together using the same tool — is achievable through thoughtful tangible interface design. For accessibility practitioners and educators, the key design principles are transferable: use passive blocks with camera-based recognition (cheaper, more robust than active electronics), provide multi-modal feedback (music output + verbal pseudocode), design physical affordances that work for both touch and sight (embossed symbols, color coding, orientation markers), create nesting/slotting relationships that are physically intuitive, and use magnets to stabilize workspace layouts. The open-source, 3D-printable nature (with alternative salt-dough fabrication explored) makes TIP-Toy reproducible in low-resource settings. The observation that mixed-ability pairs naturally developed complementary roles challenges the assumption that VI children need separate tools. Limitations include all-male participants, small sample size, single-session format, and the inherent complexity ceiling of TUI-based programming — the VI expert consultants recommended TIP-Toy be positioned as a stepping stone toward text-based accessible programming.
Tags: visual accessibility · STEM education · tangible interaction · blindness and low vision · inclusive design · education accessibility · music accessibility · computer science · digital fabrication