SoundLines: Exploration of Line Segments through Sonification and Multi-touch Interaction
Dragan Ahmetovic, Cristian Bernareggi, Sergio Mascetti · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020) · doi:10.1145/3373625.3418041
Summary
This paper presents SoundLines, a mobile application designed to help children with visual impairments explore and understand line segments on a touchscreen using sonification and multi-touch interaction. The system addresses a fundamental challenge in early education for blind children: understanding spatial relationships and geometric concepts that are typically taught through visual media. SoundLines maps audio feedback to finger position on the screen, allowing users to discover and trace line segments connecting pairs of endpoints without requiring any visual information. The app uses a playful framing where children guide a kitten (one endpoint) to its mother cat (another endpoint) by finding and following the connecting line. The interaction model relies on three distinct sonification cues that guide the user through the exploration process. A proximity cue uses rising pitch as the finger approaches a line segment, helping users locate lines on the screen. Once the finger is on the line, a tracing cue provides continuous pitch feedback mapped to the vertical position of the finger, giving real-time spatial awareness during line following. An endpoint cue plays a distinctive meow sound when the user successfully reaches a target endpoint. The system supports multi-touch input, allowing users to place one finger on a starting endpoint while using another to explore and trace the connecting line. The authors conducted a preliminary evaluation with four sighted children aged three to six and two blind adults to assess the basic usability of the interaction paradigm.
Key findings
The preliminary evaluation revealed several important insights about using sonification for spatial exploration with young users. All four sighted children (aged 3-6) were able to understand and use the sonification cues to locate and trace line segments, suggesting the interaction paradigm is intuitive even for very young users. The two blind adult participants also successfully completed the tasks, validating the approach for users without visual feedback. The proximity-based sonification proved effective as a search mechanism, allowing users to sweep the screen and hone in on line locations through audio pitch changes. The multi-touch approach — anchoring one finger on a known endpoint while exploring with another — provided a stable reference frame that helped users maintain spatial orientation. Children engaged positively with the gamified kitten-and-cat framing, which motivated continued exploration. The study identified areas for refinement, including the need to calibrate pitch mapping sensitivity and to explore additional sonification strategies for more complex geometric shapes beyond simple line segments. While the sample size was small, the results demonstrate the feasibility of sonification-based touchscreen interaction for teaching spatial concepts to young children with visual impairments.
Relevance
SoundLines contributes to the growing body of work on making touchscreen devices accessible for spatial and geometric understanding, particularly for children who are blind or have low vision. The approach is significant because it repurposes mainstream mobile devices rather than requiring specialized hardware, potentially making spatial education more widely available. For accessibility practitioners, the three-tier sonification model (proximity, tracing, endpoint) offers a reusable design pattern for any application that needs to convey spatial relationships through audio. The work also highlights the importance of designing accessible educational tools specifically for young children, a population often underserved by assistive technology research. The gamification strategy demonstrates how playful design can make accessible interactions engaging rather than clinical. Practitioners working on accessible education, tactile graphics alternatives, or audio-based spatial interfaces will find valuable design insights in this work.
Tags: sonification · visual impairments · children · touchscreen accessibility · multi-touch interaction · audio feedback · tactile graphics · education