VocalIDE: An IDE for Programming via Speech Recognition
Lucas Rosenblatt · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17) · doi:10.1145/3132525.3134824
Summary
This student research paper addresses the underrepresentation of people with upper-limb physical impairments in the developer community — while 6.7% of Americans have upper-limb impairments, less than 4% of developers report any physical disability. The author argues that current coding practice implicitly requires keyboard use, and existing alternatives like soft keyboards or specialised trackballs are inefficient, difficult to learn, or expensive. VocalIDE is a prototype voice-based integrated development environment (IDE) designed to make programming accessible without a keyboard. The research began with a Wizard of Oz study with 10 participants (all with coding experience, none with motor impairments) who were asked to correct code by giving vocal instructions to a "human computer" (researcher acting as the text editor). The study aimed to understand what commands programmers naturally use when directing code editing verbally, the major friction points, and the most difficult tasks to complete vocally.
Key findings
The Wizard of Oz study revealed that participants shared a common vocabulary for vocal code editing, frequently using words like "right," "backspace," "jump-to," "space," "the," "after," "and," "then," "to," "line," and "type." However, all participants exhibited low lexical density (16.33-35.81, compared to 45+ for normal speech), indicating significant inefficiency — participants used many filler words and non-command speech relative to productive instructions. Common challenges included navigating to specific positions in code, referencing particular characters or symbols, and expressing precise editing operations. Based on these findings, the author developed VocalIDE with specialised editing mechanisms including a level-based error correction system where the IDE presents numbered errors that users can select and correct vocally. The prototype demonstrates that publicly available speech recognition APIs can provide a foundation for keyboard-free programming, though the system requires some user education on non-natural voice commands.
Relevance
This work highlights an important but often overlooked accessibility gap: the tools used to build software are themselves inaccessible to many people with physical disabilities. For the accessibility community, this is both a practical concern (limiting workforce diversity in tech) and a meta-concern (the people building accessible software may not include those who most understand accessibility barriers). While this is a short student research paper presenting early-stage work with a prototype rather than a fully validated system, it opens an important research direction. The Wizard of Oz methodology provides useful insights into how people naturally think about vocal code editing, which could inform the design of future voice-based development tools. Key limitations include that the WOz study participants did not have motor impairments, the prototype was early-stage, and the paper does not report formal usability evaluation of VocalIDE itself.
Tags: speech recognition · motor disability · programming accessibility · voice interface · upper-limb impairment · software development · assistive technology