ChattyBooks and ChattyBook Service
Masakazu Suzuki, Katsuhito Yamaguchi · 2017 · Proceedings of the 14th International Web for All Conference (W4A) · doi:10.1145/3058555.3060619
Summary
This demonstration paper presents ChattyBooks, a Windows application that converts STEM content in DAISY/accessible EPUB3 format into audio-embedded HTML5 with JavaScript, enabling playback in any web browser on any platform without requiring specialized DAISY/EPUB3 players. The work emerges from over 20 years of research by the Infty Project, which began in 1996 when author Suzuki counseled a blind mathematics student and recognized the profound difficulty of accessing mathematical content without sight. The project produced two key tools: InftyReader, the only math OCR system in practical use worldwide, which recognizes scientific documents including mathematical expressions, tables, figures, and graphs from print or PDF; and ChattyInfty, an accessible STEM document editor that enables visually disabled people to both read and author scientific content with speech output. The motivation for ChattyBooks arose from observing that ordinary DAISY content and players are insufficient for dyslexic students, who typically do not use screen readers and may not have text-to-speech engines installed. Furthermore, the diversity of devices, operating systems, and environments makes it impractical to customize DAISY content for every user's setup. ChattyBooks solves this by converting to universal HTML5/JavaScript that embeds mp3 audio files for all content including math expressions, with synchronized text/math highlighting during playback.
Key findings
The ChattyBooks system consists of a converter and file manager. Converted content retains the same functionality and operability as original DAISY/accessible EPUB3 while being readable in any web browser. Mathematical expressions are represented in either MathML or SVG (user-selectable during conversion), and audio files of aloud reading are embedded for each sentence and math expression. The ChattyInfty editor uses Microsoft Speech API (SAPI5) as its text-to-speech engine for the English version. A companion web service (ChattyBook Service) enables server-side conversion: users upload DAISY/EPUB3 content and download the converted ChattyBook format, eliminating the need for a Windows PC. A dedicated iPad app (ChattyBooks for iPad) serves as a file manager for downloading and playing back content. The full pipeline can convert e-born PDF (electronically produced, not scanned) through InftyReader into DAISY/accessible EPUB3, then via ChattyBooks into browser-readable content — effectively making PDF STEM documents accessible through a chain of conversions. Since 2008, the Japanese Society for Rehabilitation of Persons with Disabilities has been using this ecosystem to create accessible elementary and junior high school textbooks in multimedia DAISY for students with learning disabilities including dyslexia.
Relevance
This work addresses two critical gaps in accessible publishing: the inaccessibility of STEM content (mathematical formulas, scientific notation) and the platform dependency of DAISY/EPUB3 players. By converting to HTML5/JavaScript, the authors make accessible STEM content truly portable — playable on any device with a browser, which is particularly important in educational settings where students use diverse hardware. The InftyReader math OCR system fills a unique niche as the only practical system for recognizing mathematical notation from print/PDF, which is essential given the vast amount of existing STEM literature locked in inaccessible formats. The pipeline from PDF to browser-readable accessible content represents a practical end-to-end solution for institutions seeking to make STEM materials accessible. The inclusion of dyslexic students as a target audience — not just blind users — broadens the impact significantly, as dyslexia affects 5-15% of the population. The Japanese nationwide deployment for school textbooks demonstrates real-world scalability. Limitations include the Windows-only nature of the converter application (partially addressed by the web service) and dependence on e-born PDF quality for the conversion pipeline.
Tags: STEM accessibility · DAISY · EPUB · MathML · mathematics accessibility · visual impairment · blindness · dyslexia · text-to-speech · OCR · accessible publishing · e-textbooks · HTML5 · JavaScript · PDF accessibility · SVG · Japan · education accessibility
Standards referenced: DAISY · EPUB 3 · MathML