Reviews

Search results

• Preferred Appearance of Captions Generated by Automatic Speech Recognition for Deaf and Hard-of-Hearing Viewers

  Larwan Berke, Khaled Albusays, Matthew Seita, Matt Huenerfauth · 2019 · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19)

  This CHI 2019 Late-Breaking Work (6 pages) investigates a practical question that has received surprisingly little research: when Automatic Speech Recognition (ASR) is used to caption small-group meetings for Deaf and Hard-of-Hearing (DHH) viewers, how should those captions…

  captioning · deaf and hard of hearing · automatic speech recognition · user interface design · typography

• Multi-view Mouth Renderization for Assisting Lip-reading

  Andrea Britto Mattos, Dario Augusto Borges Oliveira · 2018 · Proceedings of the 15th International Web for All Conference (W4A)

  This paper presents an assistive tool that uses Generative Adversarial Networks (GANs) to enhance video for people who rely on lip-reading. The core problem is that lip-readers generally prefer a frontal view of a speaker's face, but in real-world video the speaker may be…

  lip-reading · hearing impairment · Deaf and hard of hearing · deep learning · generative adversarial networks

• Towards Accessible Conversations in a Mobile Context for People who are Deaf and Hard of Hearing

  Dhruv Jain, Rachel Franz, Leah Findlater, Jackson Cannon, Raja Kushalnagar, Jon Froehlich · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '18)

  This paper presents two studies examining the communication needs of deaf and hard of hearing (DHH) people in mobile contexts (walking, transit, recreational activities) and the potential for head-mounted display (HMD) captions to address those needs. Prior research on DHH…

  deaf and hard of hearing · real-time captioning · augmented reality · head-mounted display · mobile accessibility

• Behavioral Changes in Speakers who are Automatically Captioned in Meetings with Deaf or Hard-of-Hearing Peers

  Matthew Seita, Khaled Albusays, Sushant Kafle, Michael Stinson, Matt Huenerfauth · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This study from Rochester Institute of Technology investigates a largely unexplored question: how does using an ASR-based captioning tool in meetings with deaf or hard of hearing (DHH) colleagues change the speaking behavior of hearing participants? While prior work has focused…

  deaf and hard of hearing · automatic speech recognition · captioning · communication accessibility · speech behavior

• Towards More Robust Speech Interactions for Deaf and Hard of Hearing Users

  Raymond Fok, Harmanpreet Kaur, Skanda Palani, Martez E. Mott, Walter S. Lasecki · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This University of Michigan study addresses a largely overlooked accessibility gap: while much research has focused on providing deaf users access to spoken output (via captioning or sign language), almost no work has addressed improving deaf users' ability to provide speech…

  deaf and hard of hearing · automatic speech recognition · deaf speech · crowdsourcing · speech intelligibility

• Exploring the Performance of Facial Expression Recognition Technologies on Deaf Adults and Their Children

  Irene Rogan Shaffer · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This Boston University student research paper investigates how commercial facial expression recognition services perform on Deaf ASL signers and Children of Deaf Adults (CODAs) compared to hearing non-signers. The study is motivated by a critical problem: in ASL and other sign…

  deaf and hard of hearing · sign language · facial expression recognition · emotion recognition · AI fairness

• Usability Evaluation of Captions for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2018 · SIGACCESS Accessibility and Computing Newsletter (Issue 122)

  This is a SIGACCESS Newsletter article summarizing a line of research by Kafle and Huenerfauth on building a caption-quality evaluation metric that actually reflects the experience of Deaf and Hard-of-Hearing (DHH) readers — rather than simply counting speech-recognition errors.…

  automatic speech recognition · captioning · captions · caption quality · accessibility metrics

• Methods for Evaluation of Imperfect Captioning Tools by Deaf or Hard-of-Hearing Users at Different Reading Literacy Levels

  Larwan Berke, Sushant Kafle, Matt Huenerfauth · 2018 · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18)

  This CHI 2018 paper (awarded an Honourable Mention) is the originating methodological study behind the group’s later Alonzo et al. work on Automatic Text Simplification evaluation. It asks: when Deaf and Hard-of-Hearing (DHH) participants evaluate imperfect captions produced by…

  captioning · deaf and hard of hearing · automatic speech recognition · research methodology · literacy

• Evaluation of Language Feedback Methods for Student Videos of American Sign Language

  Matt Huenerfauth, Elaine Gale, Brian Penly, Sree Pillutla, Mackenzie Willard, Dhananjai Hariharan · 2017 · ACM Transactions on Accessible Computing (TACCESS)

  This paper investigates how to best present video-based feedback to students learning American Sign Language (ASL), as part of a long-term project to build an automatic system that analyses student signing videos and provides immediate corrective feedback. The motivation is…

  sign language · deaf and hard of hearing · ASL education · video feedback · language learning

• Regression Analysis of Demographic and Technology-Experience Factors Influencing Acceptance of Sign Language Animation

  Hernisa Kacorri, Matt Huenerfauth, Sarah Ebling, Kasmira Patel, Kellie Menzies, Mackenzie Willard · 2017 · ACM Transactions on Accessible Computing (TACCESS)

  This paper investigates how deaf participants' demographic backgrounds and technology experience influence their evaluation scores when assessing sign language animation systems, revealing that participant characteristics — not just animation quality — significantly affect study…

  sign language avatar · deaf and hard of hearing · evaluation methodology · regression analysis · ASL animation

• Closed ASL Interpreting for Online Videos

  Raja Kushalnagar, Matthew Seita, Abraham Glasser · 2017 · Proceedings of the 14th International Web for All Conference (W4A)

  This paper introduces "closed interpreting," a concept analogous to closed captioning but for sign language interpretation of online videos. While many deaf viewers prefer ASL interpreters over captions (as verbatim captioning speed often exceeds reading abilities, and deaf…

  American Sign Language · ASL · Deaf and hard of hearing · sign language interpreting · video accessibility

• Subjective Evaluation of Website Accessibility and Usability: A Survey for People with Sensory Disabilities

  Tahani Alahmadi, Steve Drew · 2017 · Proceedings of the 14th International Web for All Conference (W4A)

  This paper presents a novel subjective evaluation model for assessing web accessibility and usability from the perspective of students with sensory disabilities, applied to Australian university websites. The model integrates accessibility criteria from WCAG 2.0 and Section 508…

  accessibility evaluation · usability · university websites · education accessibility · sensory disabilities

• Personal Perspectives on Using Automatic Speech Recognition to Facilitate Communication between Deaf Students and Hearing Customers

  James R. Mallory, Michael Stinson, Lisa Elliot, Donna Easton · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This experience report examines the use of Automatic Speech Recognition (ASR) via the WhatsApp smartphone app to facilitate communication between deaf and hard-of-hearing (D/HH) students and hearing business customers in real workplace settings. The study took place at the…

  automatic speech recognition · deaf and hard of hearing · workplace accessibility · deaf education · speech recognition

• Real-Time Depth-Camera Based Hand Tracking for ASL Recognition

  Brandon Taylor, Anind Dey, Daniel Siewiorek, Asim Smailagic · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This demonstration paper validates the use of a publicly available real-time hand tracking algorithm (Sphere-Mesh) for recognizing American Sign Language (ASL) handshapes using a depth camera. Sign Language Recognition (SLR) has long been a motivating goal for high-precision…

  sign language recognition · hand tracking · computer vision · depth camera · machine learning

• Sign Language Support System for Viewing Sports Programs

  Tsubasa Uchida, Taro Miyazaki, Makiko Azuma, Shuichi Umeda, Naoto Kato, Hideki Sumiyoshi, Yuko Yamanouchi, Nobuyuki Hiruma · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This demonstration paper from NHK (Japan Broadcasting Corporation) presents a prototype system that provides Japanese Sign Language (JSL) support for deaf and hard-of-hearing viewers watching sports broadcasts. The system was developed in response to strong demand from Japan's…

  sign language · signing avatar · Japanese Sign Language · deaf and hard of hearing · media accessibility

• Speed-Accuracy Tradeoffs for Detecting Sign Language Content in Video Sharing Sites

  Frank M. Shipman, Satyakiran Duggina, Caio D.D. Monteiro, Ricardo Gutierrez-Osuna · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This paper addresses the problem of automatically detecting sign language content in videos on sharing platforms like YouTube and Vimeo. For many deaf and hard-of-hearing people, sign language is their primary communication medium, and they rely on online video content to stay…

  sign language · computer vision · video classification · information retrieval · deaf and hard of hearing

• Deaf, Hard of Hearing, and Hearing Perspectives on Using Automatic Speech Recognition in Conversation

  Abraham Glasser, Kesavan Kushalnagar, Raja Kushalnagar · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This experience report describes the real-world accessibility challenges encountered by five participants — two deaf, one hard of hearing, and two hearing — including the authors, when using the top seven most popular ASR applications (DEAFCOM, Dragon Dictation, Siri, Virtual…

  automatic speech recognition · deaf and hard of hearing · speech recognition · communication accessibility · voice interface

• Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper addresses a fundamental problem in automatic captioning for Deaf and Hard of Hearing (DHH) users: the standard metric used to evaluate automatic speech recognition (ASR) systems — Word Error Rate (WER) — poorly predicts how usable the resulting captions actually are…

  captioning · automatic speech recognition · deaf and hard of hearing · evaluation methods · natural language processing

• Deaf and Hard-of-Hearing Perspectives on Imperfect Automatic Speech Recognition for Captioning One-on-One Meetings

  Larwan Berke, Christopher Caulfield, Matt Huenerfauth · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper investigates whether and how to display word-level confidence information from Automatic Speech Recognition (ASR) systems in real-time captions for Deaf and Hard-of-Hearing (DHH) users during one-on-one meetings with hearing people. ASR engines assign confidence…

  deaf accessibility · automatic speech recognition · captioning · communication · user research

• Leveraging Complementary Contributions of Different Workers for Efficient Crowdsourcing of Video Captions

  Yun Huang, Yifeng Huang, Na Xue, Jeffrey P. Bigham · 2017 · CHI Conference on Human Factors in Computing Systems

  This paper presents BandCaption, a crowdsourcing system that combines automatic speech recognition (ASR) with input from diverse crowd workers to efficiently correct video captions. The key insight is that different groups of people — hearing-impaired users, second-language…

  captioning · crowdsourcing · video accessibility · speech recognition · deaf and hard of hearing

• Scribe: Deep Integration of Human and Machine Intelligence to Caption Speech in Real Time

  Walter S. Lasecki, Christopher D. Miller, Iftekhar Naim, Raja Kushalnagar, Adam Sadilek, Daniel Gildea, Jeffrey P. Bigham · 2017 · Communications of the ACM

  Scribe is a system that provides on-demand, real-time captioning of live speech for deaf and hard of hearing (DHH) people by combining groups of non-expert human captionists with machine intelligence. The system addresses a critical accessibility gap: professional CART…

  real-time captioning · deaf and hard of hearing · crowdsourcing · human computation · speech recognition

• The Effects of Automatic Speech Recognition Quality on Human Transcription Latency

  Yashesh Gaur, Walter S. Lasecki, Florian Metze, Jeffrey P. Bigham · 2016 · Proceedings of the 13th International Web for All Conference (W4A)

  This paper from Carnegie Mellon University and the University of Michigan empirically investigates when automatic speech recognition (ASR) output helps or hinders human transcriptionists producing captions for deaf and hard of hearing people. Manual transcription remains…

  speech recognition · captioning · deaf and hard of hearing · crowdsourcing · human computation

• Nothing to Hide: Aesthetic Customization of Hearing Aids and Cochlear Implants in an Online Community

  Halley P. Profita, Abigale Stangl, Laura Matuszewska, Sigrunn Sky, Shaun K. Kane · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16)

  This paper analyzes aesthetic customization practices within a Facebook community of over 4,800 members dedicated to decorating and personalizing hearing aids (HAs) and cochlear implants (CIs). Approximately 48 million people in the United States (20% of the population) have…

  hearing aid · cochlear implant · DIY assistive technology · social accessibility · disability identity

• Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students

  Saba Kawas, George Karalis, Tzu Wen, Richard E. Ladner · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16)

  This paper takes a holistic, qualitative approach to understanding deaf and hard of hearing (DHH) university students' experiences with real-time captioning in mainstream classrooms, examining both human-based captioning (CART — Communication Access Realtime Translation) and…

  deaf and hard of hearing · real-time captioning · CART · automatic speech recognition · education

• Ad-Hoc Access to Musical Sound for Deaf Individuals

  Benjamin Petry, Thavishi Illandara, Juan Pablo Forero, Suranga Nanayakkara · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16)

  This poster paper presents MuSS-Bits (Music Sensory Substitution Bits), a wearable sensor-display system that enables deaf individuals to explore musical sound from various audio sources with real-time vibrotactile feedback. While existing sensory substitution systems for music…

  deaf and hard of hearing · music accessibility · sensory substitution · haptic technology · wearable technology