Reviews

Search results

• Artificial Intelligence Fairness in the Context of Accessibility Research on Intelligent Systems for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Abraham Glasser, Sedeeq Al-khazraji, Larwan Berke, Matthew Seita, Matt Huenerfauth · 2020 · SIGACCESS Accessibility and Computing

  This paper from RIT's Center for Accessibility and Inclusion Research discusses AI fairness issues specifically through the lens of the authors' extensive research on intelligent systems for people who are Deaf or Hard of Hearing (DHH). The authors identify five interconnected…

  AI fairness · deaf and hard of hearing · automatic speech recognition · captioning · evaluation metrics

• Deaf and hard-of-hearing users’ prioritization of genres of online video content requiring accurate captions

  Larwan Berke, Matthew Seita, Matt Huenerfauth · 2020 · Proceedings of the 17th International Web for All Conference (W4A)

  This paper investigates which genres of online video content Deaf and Hard-of-Hearing (DHH) users consider most important to have accurately captioned. With over 400 hours of video uploaded to YouTube every minute and no U.S. legal mandate to caption all online video (especially…

  deaf and hard of hearing · captioning · video accessibility · automatic speech recognition · user research

• Breaking Boundaries with Live Transcribe: Expanding Use Cases Beyond Standard Captioning Scenarios

  Fernando Loizides, Sara Basson, Dimitri Kanevsky, Olga Prilepova, Sagar Savla, Susanna Zaraysky · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This short paper catalogs non-traditional, serendipitous uses of Google's Live Transcribe, a free Android application that provides real-time speech-to-text transcription in over 80 languages. The authors — a mix of Google developers, researchers, and DHH users (co-creator…

  automatic speech recognition · deaf and hard of hearing · captioning · speech to text · COVID-19

• Deaf Individuals' Views on Speaking Behaviors of Hearing Peers when Using an Automatic Captioning App

  Matthew Seita, Matt Huenerfauth · 2020 · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20)

  This CHI 2020 Late-Breaking Work paper investigates what behaviors hearing speakers should ideally exhibit when holding in-person conversations with Deaf or deaf people using an Automatic Speech Recognition (ASR) captioning app on a mobile device. The authors position the study…

  automatic speech recognition · deaf and hard of hearing · captioning · captions · speaking behavior

• Predicting the Understandability of Imperfect English Captions for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2019 · ACM Transactions on Accessible Computing (TACCESS)

  This paper tackles a fundamental measurement problem in ASR-based captioning for Deaf and Hard-of-Hearing (DHH) users: the standard Word Error Rate (WER) metric has little correlation with how DHH users actually perceive caption quality. WER treats all word errors as equally…

  automatic speech recognition · captioning · deaf and hard of hearing · evaluation metrics · word error rate

• Evaluating the Benefit of Highlighting Key Words in Captions for People who are Deaf or Hard of Hearing

  Sushant Kafle, Peter Yeung, Matt Huenerfauth · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This paper investigates whether visually highlighting important words in video captions benefits Deaf and Hard of Hearing (DHH) users, through formative studies and a larger evaluation study. DHH users face a unique visual-attention challenge when watching captioned video: they…

  Deaf and hard of hearing · captioning · text highlighting · educational accessibility · video accessibility

• Exploration of Automatic Speech Recognition for Deaf and Hard of Hearing Students in Higher Education Classes

  Janine Butler, Brian Trager, Byron Behm · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019)

  This paper presents a qualitative study of how deaf and hard of hearing (DHH) students at the National Technical Institute for the Deaf (Rochester Institute of Technology) experienced automatic speech recognition (ASR) as a supplemental access service in mainstream higher…

  speech recognition · Deaf and hard of hearing · captioning · higher education · real-time captions

• How to Develop Accessible Web Interfaces for Deaf People?

  Gênesis Medeiros do Carmo, Débora Maria Barroso Paiva, Maria Istela Cagnin · 2019 · Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (IHC)

  This paper presents a systematic mapping study examining how web interfaces are being designed and implemented for deaf and hard of hearing users. The authors searched six databases and 17 conferences from 2005-2018, screening 469 publications to select 29 primary studies. The…

  deaf accessibility · hard of hearing · web accessibility · systematic mapping · sign language

• Preferred Appearance of Captions Generated by Automatic Speech Recognition for Deaf and Hard-of-Hearing Viewers

  Larwan Berke, Khaled Albusays, Matthew Seita, Matt Huenerfauth · 2019 · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19)

  This CHI 2019 Late-Breaking Work (6 pages) investigates a practical question that has received surprisingly little research: when Automatic Speech Recognition (ASR) is used to caption small-group meetings for Deaf and Hard-of-Hearing (DHH) viewers, how should those captions…

  captioning · deaf and hard of hearing · automatic speech recognition · user interface design · typography

• Internet of Things (IoT) as Assistive Technology: Potential Applications in Tertiary Education

  Scott Hollier, Shadi Abou-Zahra · 2018 · Proceedings of the 15th International Web for All Conference (W4A 2018)

  This short paper explores how consumer Internet of Things (IoT) devices could serve as assistive technology for students with disabilities in tertiary education, based on qualitative interviews with five students representing hearing, mobility, print, low vision, and…

  Internet of Things · assistive technology · higher education · W3C · voice assistant

• Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers

  Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson, Matt Huenerfauth · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This study from Rochester Institute of Technology investigates a largely unexplored question: how does using an ASR-based captioning tool in meetings with deaf or hard of hearing (DHH) colleagues change the speaking behavior of hearing participants? While prior work has focused…

  deaf and hard of hearing · automatic speech recognition · captioning · communication accessibility · speech behavior

• Usability Evaluation of Captions for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2018 · SIGACCESS Accessibility and Computing Newsletter (Issue 122)

  This is a SIGACCESS Newsletter article summarizing a line of research by Kafle and Huenerfauth on building a caption-quality evaluation metric that actually reflects the experience of Deaf and Hard-of-Hearing (DHH) readers — rather than simply counting speech-recognition errors.…

  automatic speech recognition · captioning · captions · caption quality · accessibility metrics

• Methods for Evaluation of Imperfect Captioning Tools by Deaf or Hard-of-Hearing Users at Different Reading Literacy Levels

  Larwan Berke, Sushant Kafle, Matt Huenerfauth · 2018 · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18)

  This CHI 2018 paper (awarded an Honourable Mention) is the originating methodological study behind the group’s later Alonzo et al. work on Automatic Text Simplification evaluation. It asks: when Deaf and Hard-of-Hearing (DHH) participants evaluate imperfect captions produced by…

  captioning · deaf and hard of hearing · automatic speech recognition · research methodology · literacy

• Scopist: Building a Skill Ladder into Crowd Transcription

  Jeffrey P. Bigham, Kristin Williams, Nila Banerjee, John Zimmerman · 2017 · Proceedings of the 14th International Web for All Conference (W4A)

  This paper introduces Scopist, a JavaScript application designed to teach crowd workers stenotype — a chording-based text entry method used by professional real-time captioners — while they perform audio transcription microtasks. The research addresses a fundamental problem in…

  crowdsourcing · captioning · stenography · deaf accessibility · transcription

• Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper addresses a fundamental problem in automatic captioning for Deaf and Hard of Hearing (DHH) users: the standard metric used to evaluate automatic speech recognition (ASR) systems — Word Error Rate (WER) — poorly predicts how usable the resulting captions actually are…

  captioning · automatic speech recognition · deaf and hard of hearing · evaluation methods · natural language processing

• Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings

  Larwan Berke, Christopher Caulfield, Matt Huenerfauth · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper investigates whether and how to display word-level confidence information from Automatic Speech Recognition (ASR) systems in real-time captions for Deaf and Hard-of-Hearing (DHH) users during one-on-one meetings with hearing people. ASR engines assign confidence…

  deaf accessibility · automatic speech recognition · captioning · communication · user research

• Leveraging Complementary Contributions of Different Workers for Efficient Crowdsourcing of Video Captions

  Yun Huang, Yifeng Huang, Na Xue, Jeffrey P. Bigham · 2017 · CHI Conference on Human Factors in Computing Systems

  This paper presents BandCaption, a crowdsourcing system that combines automatic speech recognition (ASR) with input from diverse crowd workers to efficiently correct video captions. The key insight is that different groups of people — hearing-impaired users, second-language…

  captioning · crowdsourcing · video accessibility · speech recognition · deaf and hard of hearing

• Scribe: Deep Integration of Human and Machine Intelligence to Caption Speech in Real Time

  Walter S. Lasecki, Christopher D. Miller, Iftekhar Naim, Raja Kushalnagar, Adam Sadilek, Daniel Gildea, Jeffrey P. Bigham · 2017 · Communications of the ACM

  Scribe is a system that provides on-demand, real-time captioning of live speech for deaf and hard of hearing (DHH) people by combining groups of non-expert human captionists with machine intelligence. The system addresses a critical accessibility gap: professional CART…

  real-time captioning · deaf and hard of hearing · crowdsourcing · human computation · speech recognition

• The Effects of Automatic Speech Recognition Quality on Human Transcription Latency

  Yashesh Gaur, Walter S. Lasecki, Florian Metze, Jeffrey P. Bigham · 2016 · Proceedings of the 13th International Web for All Conference (W4A)

  This paper from Carnegie Mellon University and the University of Michigan empirically investigates when automatic speech recognition (ASR) output helps or hinders human transcriptionists producing captions for deaf and hard of hearing people. Manual transcription remains…

  speech recognition · captioning · deaf and hard of hearing · crowdsourcing · human computation

• Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students

  Saba Kawas, George Karalis, Tzu Wen, Richard E. Ladner · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16)

  This paper takes a holistic, qualitative approach to understanding deaf and hard of hearing (DHH) university students' experiences with real-time captioning in mainstream classrooms, examining both human-based captioning (CART — Communication Access Realtime Translation) and…

  deaf and hard of hearing · real-time captioning · CART · automatic speech recognition · education

• Evaluation of Real-time Captioning by Machine Recognition with Human Support

  Hironobu Takagi, Takashi Itoh, Kaoru Shinkawa · 2015 · Proceedings of the 12th International Web for All Conference (W4A)

  This paper from IBM Research Tokyo investigates a hybrid approach to real-time captioning that combines Automated Speech Recognition (ASR) with human correction to make workplace meetings accessible for deaf and hard of hearing (DHH) employees. Professional stenography services…

  real-time captioning · deaf and hard of hearing · automated speech recognition · workplace accessibility · Japanese

• The Effects of Automatic Speech Recognition Quality on Human Transcription Latency

  Yashesh Gaur · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility

  This paper investigates a practical question for accessibility: when does providing automatic speech recognition (ASR) output help human captionists work faster, and when does it slow them down? Converting speech to text is fundamental for making audio content accessible to deaf…

  deaf · hard of hearing · automatic speech recognition · ASR · captioning

• Tracked Speech-To-Text Display: Enhancing Accessibility and Readability of Real-Time Speech-To-Text

  Raja S. Kushalnagar, Gary W. Behm, Aaron W. Kelstone, Shareef Ali · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility

  This research addresses a subtle but significant barrier facing deaf and hard of hearing (DHH) students in educational settings: visual dispersion. While hearing students can simultaneously watch lecture visuals (slides, demonstrations, whiteboard) and listen to the speaker's…

  deaf and hard of hearing · speech-to-text · CART · captioning · education

• Introducing Game Elements in Crowdsourced Video Captioning by Non-Experts

  Hernisa Kacorri, Kaoru Shinkawa, Shin Saito · 2014 · Proceedings of the 11th Web for All Conference (W4A)

  This paper from CUNY Graduate Center and IBM Research Tokyo presents a gamified crowdsourcing platform for video captioning that combines ASR output with non-expert human transcription to improve caption accuracy without monetary rewards. The system builds on the Collaborative…

  captioning · crowdsourcing · deaf and hard of hearing · gamification · automatic speech recognition

• Helping students keep up with real-time captions by pausing and highlighting

  Walter S. Lasecki, Raja Kushalnagar, Jeffrey P. Bigham · 2014 · Proceedings of the 11th Web for All Conference (W4A)

  This paper addresses a fundamental problem with real-time captioning for deaf and hard of hearing (DHH) students: the mismatch between speaking rates (approximately 170 words per minute) and reading rates, which causes students to fall progressively behind the live content. The…

  deaf and hard of hearing · captioning · real-time captioning · education · inclusive classrooms