Reviews

Search results

• Deaf and Hard of Hearing Access to Intelligent Personal Assistants: Comparison of Voice-Based Options with an LLM-Powered Touch Interface

  Paige S DeVries, Michaela Okosi, Ming Li, Nora Dunphy, Gidey Gezae, Dante Conway, Abraham Glasser, Raja Kushalnagar, Christian Vogler · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26)

  This mixed-methods study compares three input methods for Deaf and Hard of Hearing (DHH) people who use their voice to interact with an Amazon Echo Show: (1) natural deaf-accented speech via Alexa's built-in ASR, (2) Wizard-of-Oz 'facilitated English' where a trained human…

  deaf and hard of hearing · voice assistant · intelligent personal assistant · automatic speech recognition · deaf-accented speech

• Like, Comment & Caption: A Decade of Social Media Video Caption Research (2015-2025)

  Huong Nguyen, Emma J. McDonnell, Lloyd May, Alexander Druzenko, Zoobia Saifullah Syeda, Mark Cartwright, Sooyeon Lee · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26)

  This CHI 2026 paper is a systematic literature review of 36 peer-reviewed studies on Social Media Video Captions (SMVC) published between 2015 and 2025, spanning HCI, accessibility, media studies, education, and language learning. The authors use 'SMVC' as an umbrella for…

  captioning · captions · video accessibility · social media accessibility · Deaf and hard of hearing

• Challenges in Automatic Speech Recognition for Adults with Cognitive Impairment

  Michelle Cohn, Alyssa Lanzi, Yui Ishihara, Chen-Nee Chuah, Georgia Zellou, Alyssa Weakley · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26)

  This CHI 2026 paper quantifies how well state-of-the-art automatic speech recognition (ASR) handles voice commands produced by older adults with cognitive impairment, and asks which acoustic features actually predict transcription accuracy. The authors draw on the Voice…

  automatic speech recognition · ASR · dementia · Alzheimer's disease · mild cognitive impairment

• Speech AI for All: The What, How, and Who of Measurement

  Kimi Wenzel, Alisha Pradhan, Maria Teleki, Tobias M. Weinberg, Robin Netzorg, Alyssa Hillary Zisk, Anna Seo Gyeong Choi, Jingjin Li, Raja Kushalnagar, Colin Lea, Abraham Glasser, Christian Vogler, Ly Xinzhen M. Zhangsun Brown, Nan Bernstein Ratner, Allison Koenecke, Karen Nakamura, Shaomei Wu · 2026 · Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA '26) — Workshop

  This CHI 2026 workshop proposal — the second in the organisers' 'Speech AI for All' series — assembles 17 researchers, practitioners, and community advocates to tackle a specific downstream problem in fair and accessible speech AI: measurement. The motivating claim is that…

  speech AI · automatic speech recognition · speech diversity · augmentative and alternative communication · disfluency

• Silence is a Feature, Not a Bug: A Deaf Developer’s Autoethnography on Agency and Local AI

  Chenyang Gong · 2026 · Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26)

  This CHI 2026 Extended Abstract is a three-page autoethnographic provocation by a Deaf computer science graduate student who uses a MED-EL cochlear implant. The author refuses the medical-model framing of deafness as deficit and instead argues that the ability to remove the…

  autoethnography · deaf and hard of hearing · cochlear implant · automatic speech recognition · captioning

• Speaker-Aware Affective Captioning for Multi-Speaker STEM Talk in Inclusive Classrooms

  Sunday David Ubur, Denis Gracanin, Stephanie P DeHart, Enoch Katey Akli, Fatemeh Sarshartehrani, Sikiru Adewale · 2026 · Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA '26)

  Ubur and colleagues at Virginia Tech address a specific failure mode of live captioning in classroom and meeting settings: collapsing multi-speaker discourse into a single text stream that obscures who said what and how it was said. They argue this is especially consequential…

  captioning · deaf and hard of hearing · speaker diarization · speech emotion recognition · STEM education

• CARTGPT: Real-Time Correction of CART Captions Using Large Language Models

  Liang-Yuan Wu, Andrea Kleiver, Dhruv Jain · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility

  This paper introduces CARTGPT, a real-time system that enhances Communication Access Realtime Translation (CART) captions by combining human-generated CART transcripts with automatic speech recognition (ASR) output and using GPT-4 to detect and correct transcription errors. CART…

  deaf and hard of hearing · real-time captioning · CART · large language models · automatic speech recognition

• Notification Designs for Influencing Hearing Speakers' Behaviors During Captioned Conversations Among Mixed DHH-Hearing Groups

  Matthew Seita, Sarah Andrew, Matt Huenerfauth · 2025 · Proceedings of the 22nd International Web for All Conference (W4A 2025)

  This paper investigates notification system designs that prompt hearing speakers to adjust their speech behaviors (speaking slower, louder, or more clearly) during ASR-captioned videoconference conversations with Deaf and Hard of Hearing (DHH) participants. The researchers…

  deaf and hard of hearing · automatic speech recognition · captioning · notification design · videoconferencing

• Measuring the Accuracy of Automatic Speech Recognition Solutions

  Korbinian Kuhn, Verena Kersken, Benedikt Reuter, Niklas Egger, Gottfried Zimmermann · 2024 · ACM Transactions on Accessible Computing

  This study provides independent, comprehensive benchmarking of 11 common automatic speech recognition (ASR) services to assess their real-world accuracy for accessibility purposes. The research addresses a critical gap: while vendors claim "state-of-the-art accuracy" and…

  automatic speech recognition · ASR · captions · deaf and hard of hearing · transcription

• Modeling Word Importance in Conversational Transcripts: Toward improved live captioning for Deaf and hard of hearing viewers

  Akhter Al Amin, Saad Hassan, Matt Huenerfauth, Cecilia O. Alm · 2023 · Proceedings of the 20th International Web for All Conference (W4A)

  This paper investigates how to model word importance in conversational transcripts to improve live captioning quality for Deaf and hard of hearing (DHH) viewers. Live captions generated by automatic speech recognition (ASR) systems inevitably contain errors, but not all errors…

  live captioning · deaf and hard of hearing · automatic speech recognition · word importance · natural language processing

• From User Perceptions to Technical Improvement: Enabling People Who Stutter to Better Use Speech Recognition

  Colin Lea, Zifang Huang, Jaya Narain, Lauren Tooley, Dianna Yee, Dung Tien Tran, Panayiotis Georgiou, Jeffrey P. Bigham, Leah Findlater · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)

  This paper investigates how people who stutter (PWS) experience consumer speech recognition systems and demonstrates technical improvements that can significantly reduce errors. The work combines user research with engineering interventions across the speech recognition…

  stuttering · speech recognition · voice assistants · dictation · speech accessibility

• Visualization of Speech Prosody and Emotion in Captions: Accessibility for Deaf and Hard-of-Hearing Users

  Caluã de Lacerda Pataca, Matthew Watkins, Roshan Peiris, Sooyeon Lee, Matt Huenerfauth · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)

  This CHI 2023 paper tackles a dimension of captioning that has gone largely unaddressed for four decades: captions depict words but strip out the prosody and emotion carried by a speaker's voice. The authors argue that while automatic speech recognition (ASR) has reduced word…

  captioning · deaf and hard of hearing · prosody · affective computing · videoconferencing accessibility

• Access on Demand: Real-time, Multi-modal Accessibility for the Deaf and Hard-of-Hearing based on Augmented Reality

  Roshan Mathew, Brian Mak, Wendy Dannels · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This experience report documents two deaf researchers' hands-on evaluation of Access on Demand (AoD), an augmented reality application developed at Rochester Institute of Technology that delivers real-time captioning and American Sign Language (ASL) interpretation through Vuzix…

  augmented reality · deaf and hard of hearing · smart glasses · captioning · sign language interpretation

• Understanding Social and Environmental Factors to Enable Collective Access Approaches to the Design of Captioning Technology

  Emma McDonnell · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This doctoral consortium paper presents a dissertation research program that reimagines how captioning technology should be designed by applying the disability justice principle of collective access — the idea that accessibility is a shared responsibility of all group members,…

  captioning · collective access · disability justice · deaf and hard of hearing · co-design

• Remotely Co-Designing Features for Communication Applications using Automatic Captioning with Deaf and Hearing Pairs

  Matthew Seita, Sooyeon Lee, Sarah Andrew, Kristen Shinohara, Matt Huenerfauth · 2022 · Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper addresses two intertwined problems. First, methodologically, how can co-design research involving both Deaf/Hard-of-Hearing (DHH) and hearing participants be conducted remotely during and beyond COVID-19, when in-person sessions are not possible and masks…

  automatic speech recognition · deaf and hard of hearing · participatory design · co-design · videoconferencing

• Deaf and Hard-of-Hearing Users' Preferences for Hearing Speakers' Behavior during Technology-Mediated In-Person and Remote Conversations

  Matthew Seita, Sarah Andrew, Matt Huenerfauth · 2021 · Proceedings of the 18th International Web for All Conference (W4A)

  This paper presents the first quantitative evidence of Deaf and hard-of-hearing (DHH) individuals' preferences for specific speech and non-verbal behaviors from hearing conversational partners during technology-mediated communication. The researchers conducted two experimental…

  deaf and hard of hearing · automatic speech recognition · videoconferencing · communication accessibility · speechreading

• A Longitudinal Evaluation of Tablet-Based Child Speech Therapy with Apraxia World

  Adam Hair, Kirrie J. Ballard, Constantina Markoulli, Penelope Monroe, Jacqueline McKechnie, Beena Ahmed, Ricardo Gutierrez-Osuna · 2021 · ACM Transactions on Accessible Computing

  This paper presents Apraxia World, a tablet-based speech therapy game designed for long-term home practice by children with speech sound disorders (SSDs), particularly childhood apraxia of speech (CAS). Unlike many therapy games that use simple arcade mechanics and quickly…

  speech therapy · childhood apraxia of speech · speech sound disorders · serious games · games for health

• Artificial Intelligence Fairness in the Context of Accessibility Research on Intelligent Systems for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Abraham Glasser, Sedeeq Al-khazraji, Larwan Berke, Matthew Seita, Matt Huenerfauth · 2020 · SIGACCESS Accessibility and Computing

  This paper from RIT's Center for Accessibility and Inclusion Research discusses AI fairness issues specifically through the lens of the authors' extensive research on intelligent systems for people who are Deaf or Hard of Hearing (DHH). The authors identify five interconnected…

  AI fairness · deaf and hard of hearing · automatic speech recognition · captioning · evaluation metrics

• Deaf and hard-of-hearing users’ prioritization of genres of online video content requiring accurate captions

  Larwan Berke, Matthew Seita, Matt Huenerfauth · 2020 · Proceedings of the 17th International Web for All Conference (W4A)

  This paper investigates which genres of online video content Deaf and Hard-of-Hearing (DHH) users consider most important to have accurately captioned. With over 400 hours of video uploaded to YouTube every minute and no U.S. legal mandate to caption all online video (especially…

  deaf and hard of hearing · captioning · video accessibility · automatic speech recognition · user research

• Breaking Boundaries with Live Transcribe: Expanding Use Cases Beyond Standard Captioning Scenarios

  Fernando Loizides, Sara Basson, Dimitri Kanevsky, Olga Prilepova, Sagar Savla, Susanna Zaraysky · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This short paper catalogs non-traditional, serendipitous uses of Google's Live Transcribe, a free Android application that provides real-time speech-to-text transcription in over 80 languages. The authors — a mix of Google developers, researchers, and DHH users (co-creator…

  automatic speech recognition · deaf and hard of hearing · captioning · speech to text · COVID-19

• Deaf Individuals' Views on Speaking Behaviors of Hearing Peers when Using an Automatic Captioning App

  Matthew Seita, Matt Huenerfauth · 2020 · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20)

  This CHI 2020 Late-Breaking Work paper investigates what behaviors hearing speakers should ideally exhibit when holding in-person conversations with Deaf or deaf people using an Automatic Speech Recognition (ASR) captioning app on a mobile device. The authors position the study…

  automatic speech recognition · deaf and hard of hearing · captioning · captions · speaking behavior

• Accessibility for Deaf and Hard of Hearing Users: Sign Language Conversational User Interfaces

  Abraham Glasser, Vaishnavi Mande, Matt Huenerfauth · 2020 · Proceedings of the 2nd Conference on Conversational User Interfaces (CUI '20)

  This short CUI 2020 position paper (3 pages, presented at the CUI@CHI workshop) lays out the research agenda for making voice-based conversational user interfaces (CUIs) — Alexa, Google Assistant, and similar personal assistant devices — accessible to Deaf and Hard-of-Hearing…

  deaf and hard of hearing · sign language · conversational user interfaces · personal assistants · american sign language

• Predicting the Understandability of Imperfect English Captions for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2019 · ACM Transactions on Accessible Computing (TACCESS)

  This paper tackles a fundamental measurement problem in ASR-based captioning for Deaf and Hard-of-Hearing (DHH) users: the standard Word Error Rate (WER) metric has little correlation with how DHH users actually perceive caption quality. WER treats all word errors as equally…

  automatic speech recognition · captioning · deaf and hard of hearing · evaluation metrics · word error rate

• Preferred Appearance of Captions Generated by Automatic Speech Recognition for Deaf and Hard-of-Hearing Viewers

  Larwan Berke, Khaled Albusays, Matthew Seita, Matt Huenerfauth · 2019 · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19)

  This CHI 2019 Late-Breaking Work (6 pages) investigates a practical question that has received surprisingly little research: when Automatic Speech Recognition (ASR) is used to caption small-group meetings for Deaf and Hard-of-Hearing (DHH) viewers, how should those captions…

  captioning · deaf and hard of hearing · automatic speech recognition · user interface design · typography

• Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers

  Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson, Matt Huenerfauth · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This study from Rochester Institute of Technology investigates a largely unexplored question: how does using an ASR-based captioning tool in meetings with deaf or hard of hearing (DHH) colleagues change the speaking behavior of hearing participants? While prior work has focused…

  deaf and hard of hearing · automatic speech recognition · captioning · communication accessibility · speech behavior