Breaking Boundaries with Live Transcribe: Expanding Use Cases Beyond Standard Captioning Scenarios
Fernando Loizides, Sara Basson, Dimitri Kanevsky, Olga Prilepova, Sagar Savla, Susanna Zaraysky · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3373625.3417300
Summary
This short paper catalogs non-traditional, serendipitous uses of Google's Live Transcribe, a free Android application that provides real-time speech-to-text transcription in over 80 languages. The authors — a mix of Google developers, researchers, and DHH users (co-creator Dimitri Kanevsky is deaf) — document first-hand accounts and user-reported scenarios where Live Transcribe was adopted in ways beyond its original design intent of captioning face-to-face conversations. The app originated from a personal experience: during a dinner in a poorly lit restaurant, Kanevsky could not lip-read, so Basson's son typed speech into Google Docs, sparking the idea for a real-time transcription app. The paper is organized around several categories of expanded use: community acceptance and adoption scenarios (employment, spontaneous communication, parent-child dynamics, changing hearing people's communication habits), emergency communication during COVID-19 (communicating through masks, through glass doors at nursing homes, at medical facilities where PPE blocked lip reading), captioning as a speech improvement tool (DHH users using the transcription as real-time feedback to refine their pronunciation), and hands-free interaction innovations (wearable mounts on hats, chests, and wrists to free DHH users from holding a phone).
Key findings
The paper reveals several important patterns in how ASR technology transforms DHH communication beyond simple captioning. First, Live Transcribe enables spontaneous interactions that were previously impossible — a deaf usher at a sports event could communicate with hearing attendees, and two deaf travelers could help a lost hearing woman find her hotel. Second, the technology shifts communication responsibility: hearing colleagues installed Live Transcribe on their own phones so a deaf coworker could communicate even when her phone broke, distributing the accessibility burden across the group. Third, COVID-19 dramatically expanded use cases, as masks eliminated lip reading — a lifeline for many DHH people. One user communicated with her father through the glass doors of his nursing home using a tablet with Live Transcribe, first in person and later via video call as quarantine restrictions tightened, switching between English and Russian. Fourth, DHH users discovered the app as a speech training tool: because Live Transcribe has near-zero latency compared to CART (human stenographer) services, cochlear implant users could use it to train their auditory processing in real time. Fifth, the physical form factor of phone-based captioning creates social barriers — staring at a phone signals disengagement — prompting users to improvise wearable solutions using magnets, hat mounts, and fabric bracelets.
Relevance
This paper demonstrates how accessibility technology, once deployed at scale, evolves through user innovation in ways designers cannot fully anticipate. For accessibility practitioners, the key lesson is that real-world use cases often exceed designed-for scenarios, and that observing serendipitous adoption patterns can inform future development. The COVID-19 examples are particularly striking: the pandemic created universal communication barriers (masks, distancing, glass barriers) that mirrored everyday challenges DHH people already face, making ASR technology suddenly relevant to a much wider population. The paper also highlights important design considerations: hands-free display options matter because holding a phone disrupts social interaction; multilingual support enables cross-cultural family communication; and near-zero latency is critical for speech training applications. The shift in communication responsibility — hearing people installing the app on their devices — aligns with disability studies principles about distributing access work rather than placing it entirely on disabled individuals.
Tags: automatic speech recognition · deaf and hard of hearing · captioning · speech to text · COVID-19 · communication accessibility · mobile accessibility