Improving Programming Interfaces for People with Limited Mobility Using Voice Recognition
Xiomara Figueroa Fontánez, Patricia Ordóñez · 2014 · Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS) · doi:10.1145/2661334.2661417
Summary
This paper describes an effort to make programming more accessible to people with motor impairments by integrating voice recognition into an Integrated Development Environment (IDE). The work is motivated by the specific case of a computer scientist with spinal muscular atrophy (SMA) who can no longer physically attend classes and can only type with one finger. Programming presents particular challenges for voice control because existing speech recognition software is designed for standard spoken or written language vocabulary, not for the specialised syntax of programming languages — code contains symbols, brackets, indentation, and keyword patterns that do not map naturally to speech. Many programmers with physical disabilities already bootstrap voice recognition into their programming workflows, but this requires time-consuming custom configuration and the available tools are not designed for coding. The project uses Simon, an open-source speech recognition platform that differs from commercial tools by allowing users to create custom language and acoustic models from scratch rather than relying on pre-trained models. Simon uses "scenarios" — configurations that define what words and phrases to listen for and what actions to perform when recognised. Prior work had integrated Simon with the Gedit text editor via Gnome's Assistive Technology Service Provider Interface, and with the Vi editor for voice-controlled programming. The current project extends this by creating scenarios specifically for IDE interaction, aiming to provide a bimodal interface where the programmer can use both voice commands and limited single-finger typing as appropriate. Programming's structured format is noted as potentially advantageous — its syntax could be leveraged to reduce keystrokes and speed navigation, if properly incorporated into the IDE.
Key findings
The paper identifies a significant gap in assistive technology for programming: while voice recognition tools exist for general computer use, and while individual programmers with disabilities have created personal workarounds, there is no well-integrated voice-controlled programming environment. The choice of Simon as an open-source platform is deliberate — it allows creation of custom language models tailored to programming vocabulary (keywords, syntax, IDE commands) rather than forcing code through a natural language recogniser that will misinterpret programming terms. The participatory design approach, with the computer scientist with SMA as a co-designer providing "invaluable input into the design of the interface," exemplifies user-centred assistive technology development. The paper notes that word prediction — commonly used in text entry assistive technology — could be particularly effective for programming, where the structured syntax makes next-token prediction more accurate than in natural language. The bimodal approach (voice plus minimal physical typing) recognises that pure voice coding may not be efficient for all programming tasks, and that users with residual motor function benefit from flexible input options.
Relevance
This work highlights an underexplored accessibility barrier: the difficulty of professional software development for people with motor impairments. Programming requires extensive typing of syntactically precise text including special characters, specific indentation, and rapid navigation through large codebases — all tasks that are difficult with voice recognition designed for prose. For accessibility practitioners, the paper surfaces an important insight: many knowledge workers with disabilities have already developed personal assistive technology solutions that remain undocumented and unshared, representing a missed opportunity for the broader community. The participatory design model — where the end user with SMA is a collaborating computer scientist, not just a test subject — represents best practice in assistive technology research. Though this is an early-stage project without evaluation results, the problem it addresses has become increasingly relevant as software development is one of the most accessible career paths for people with physical disabilities who have strong cognitive abilities, and as tools like GitHub Copilot and AI-assisted coding have since opened new possibilities for reducing the physical demands of programming.
Tags: programming accessibility · voice interface · speech recognition · motor disability · spinal muscular atrophy · IDE · open source · participatory design