MathMelodies 2: a Mobile Assistive Application for People with Visual Impairments Developed with React Native
Niccolò Cantù, Mattia Ducci, Dragan Ahmetovic, Cristian Bernareggi, Sergio Mascetti · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018) · doi:10.1145/3234695.3241006
Summary
This demo paper from the University of Milan and University of Turin reports on MathMelodies 2, a cross-platform math education app for blind and visually impaired (BVI) children in grades 1-5, developed using React Native. The original MathMelodies was iPad-only, developed in native code via a crowdfunding campaign in 2013, and downloaded over 50,000 times worldwide. Users consistently requested availability on other platforms (iPhone and Android), prompting the team to rebuild using React Native for cross-platform deployment. The app presents a fantasy story narrative where children solve 13 types of math exercises at varying difficulty levels to advance the plot. It is designed to be accessible and entertaining for both sighted and BVI children. The paper establishes five key design principles derived from the original app and user feedback: simple gestures (compatible with VoiceOver and TalkBack), no scrolling (all elements visible on screen), reference points (important interactions in consistent locations, preferably bottom-right), homogeneity (same interface for all users, letting system accessibility tools adapt), and readability (small text portions, uniform neutral backgrounds). A notable design evolution was shifting from fixed-size icons (convenient for blind users learning screen dimensions but causing overflow on small screens) to max-size icons that fill available space, optimizing screen real estate while maintaining the no-scrolling principle.
Key findings
The paper provides a practical assessment of React Native's accessibility support for assistive application development. Basic accessibility features work well — React Native maps components to native graphical objects, supporting alternative text for images and compatibility with screen readers and magnifiers with effort comparable to native development. However, advanced accessibility features are not supported. Specifically, React Native cannot capture screen reader focus change events — critical for MathMelodies 2, where touching a dog icon during a counting exercise should play the dog's sound. In native code, this is implemented by detecting when VoiceOver/TalkBack focus moves to the icon, but React Native provides no access to this system API. The solution was to develop native plug-in components (for both Android and iOS) that look and behave like React Native Image components but forward accessibility events to JavaScript callback functions. This hybrid approach allowed the team to achieve full accessibility while keeping most code cross-platform. Preliminary testing with two adult BVI users (one blind, one low vision) on both Android and iOS found no accessibility problems, though more extensive testing with children is planned.
Relevance
This paper addresses two underserved areas in accessibility: math education for blind children and cross-platform accessible app development. For developers building accessible applications, the practical finding that React Native supports basic but not advanced accessibility features is directly actionable — teams can plan native plug-in development for specific accessibility needs while keeping the majority of code cross-platform. The design principles articulated (no scrolling, consistent reference points, max-size icons, simple gestures) are broadly applicable to any accessible mobile app for BVI users. The paper also highlights an important equity issue: the original MathMelodies was iPad-only, meaning children who could only afford Android devices were excluded — cross-platform development directly addresses this access gap. The app's approach of presenting the same interface to all children (sighted and BVI), with system accessibility tools providing the adaptations, aligns with inclusive design principles and avoids stigmatizing BVI users with a visually different application.
Tags: blindness · low vision · mathematical accessibility · education · children · mobile accessibility · React Native · cross-platform development · screen reader · VoiceOver · TalkBack