Handsfree for Web: A Google Chrome extension to browse the web via voice commands
Javier Pérez · 2019 · Proceedings of the 16th International Web for All Conference (W4A) · doi:10.1145/3315002.3332443
Summary
This demonstration paper presents Handsfree for Web, a free Google Chrome browser extension that enables users to browse the web entirely through voice commands. The tool addresses a fundamental accessibility barrier: nearly all websites require manual interaction via keyboard, mouse, or touchscreen, excluding people with motor disabilities who cannot handle these devices. While existing speech recognition tools exist (such as Dragon NaturallySpeaking), they tend to focus on specific subsystems (desktop apps, mobile, or home assistants), offer limited web interaction capabilities, are platform-specific, and require commercial licenses. Handsfree for Web was developed after an extensive analysis of all actions a user can perform in a web environment, mapping each manual interaction to a voice command alternative. The extension implements a voice command model with two structural types: atomic commands that contain all information needed to execute an action (e.g., "Open wikipedia.com") and segmented commands that form a chain describing a multi-step behavior (e.g., "Select video ~ Number 2 ~ Mute"). Commands are also classified by origin: generic commands work on any website (e.g., "Click link", "Scroll down", "Zoom in", "New tab") and custom commands are site-specific (e.g., "Submit email" for a webmail platform). The extension uses the Web Speech API for speech-to-text conversion, with phonetic analysis and user context considered to improve command matching accuracy.
Key findings
The extension supports hundreds of built-in generic commands and provides two speech recognition modes: continuous (always listening, suitable for quiet environments) and interrupted (activated by a key press, suitable for noisy environments but not fully hands-free). Custom voice commands can be added by users or temporarily loaded by the active website, enabling extensibility through community contributions. The tool supports English and Spanish for navigation commands and all Web Speech API languages for text dictation. When the user invokes a "Click" command, the extension visually labels all clickable elements on the page with numbers, allowing the user to say the number of the desired element. The extension is freely available on the Chrome Web Store and has a growing community of users and developers who create and share custom voice commands. Future work includes a companion library called "Handsfree for Websites" that website developers can embed to provide voice command support directly within their sites, plus additional language support for Italian and Dutch.
Relevance
Handsfree for Web addresses a real accessibility gap for people with motor impairments who need hands-free web browsing. While screen readers serve blind users and magnifiers serve low-vision users, the options for people with motor disabilities who cannot use standard input devices are more limited and often expensive. This free, easily installable Chrome extension provides an immediate, zero-cost solution. The voice command model is thoughtfully designed — the distinction between atomic and segmented commands allows both simple actions and complex multi-step workflows, while the generic/custom split enables the tool to work on any website while also supporting site-specific interactions. The numbered element labeling approach for clicking is practical, solving the fundamental problem of how to identify specific clickable elements through voice alone. For accessibility practitioners and web developers, the concept of websites being able to load their own custom voice commands points toward a future where voice interaction is designed into web applications rather than being bolted on externally. The two recognition modes (continuous and interrupted) reflect a practical understanding that voice control must adapt to different environmental conditions.
Tags: voice control · speech recognition · motor disability · browser extension · web accessibility · hands-free interaction · Chrome extension · Web Speech API · assistive technology
Standards referenced: Web Speech API