Reviews

Search results

• Deep Learning Methods for Sign Language Translation

  Tejaswini Ananthanarayana, Priyanshu Srivastava, Akash Chintha, Akhil Santha, Brian Landy, Joseph Panaro, Andre Webster, Nikunj Kotecha, Shagan Sah, Thomastine Sarchet, Raymond Ptucha, Ifeoma Nwogu · 2021 · ACM Transactions on Accessible Computing

  This comprehensive study evaluates deep learning methods for translating sign language video directly to spoken/written text—critically, without requiring the intermediate step of gloss-based recognition (manual sign-for-sign transcription). The researchers systematically…

  sign language · machine translation · deep learning · transformer · neural network

• The FATE Landscape of Sign Language AI Datasets: An Interdisciplinary Perspective

  Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, Meredith Ringel Morris · 2021 · ACM Transactions on Accessible Computing

  This interdisciplinary paper examines the ethical landscape of AI datasets used for sign language recognition, generation, and translation technologies. Drawing on expertise from deaf community members, sign language linguists, and AI researchers, the authors apply the FATE…

  sign language · AI datasets · deaf community · FATE framework · machine learning

• Deaf Users' Preferences Among Wake-Up Approaches during Sign-Language Interaction with Personal Assistant Devices

  Vaishnavi Mande, Abraham Glasser, Becca Dingman, Matt Huenerfauth · 2021 · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21)

  This CHI 2021 Extended Abstract investigates a narrow but previously unexplored question: if future personal-assistant devices (Alexa, Google Assistant, etc.) could recognise sign language, how should Deaf users wake them up? Current wake-up mechanisms — speaking a wake-word…

  deaf and hard of hearing · sign language · personal assistants · voice interface · conversational user interfaces

• Exploring Collection of Sign Language Datasets: Privacy, Participation, and Model Performance

  Danielle Bragg, Oscar Koller, Naomi Caselli, William Thies · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020)

  This paper tackles a fundamental tension in building machine learning systems for marginalized communities: the need for large training datasets versus the privacy risks of collecting data from small, identifiable populations. The authors focus on sign language video collection,…

  sign language · privacy · machine learning · data collection · Deaf culture

• Teaching ASL Signs using Signing Avatars and Immersive Learning in Virtual Reality

  Lorna C. Quandt, Jason Lamberton, Athena S. Willis, Jianye Wang, Kaitlyn Weeks, Emily Kubicek, Melissa Malzkuhn · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020)

  This demonstration paper presents SAIL (Signing Avatars & Immersive Learning), the first ASL instructional system built for immersive virtual reality. Developed at Gallaudet University by a majority-Deaf team, SAIL uses a signing avatar — a computer-animated virtual human…

  sign language · American Sign Language · virtual reality · signing avatars · motion capture

• Teleconference Accessibility and Guidelines for Deaf and Hard of Hearing Users

  Raja S. Kushalnagar, Christian Vogler · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020)

  This experience report from two Gallaudet University researchers examines the accessibility challenges that deaf and hard of hearing (DHH) users face in teleconference environments, drawing on nearly a decade of first-hand participation across platforms including Zoom, Microsoft…

  deaf · hard of hearing · teleconferencing · remote participation · sign language

• Student and Teacher Perspectives of Learning ASL in an Online Setting

  Garreth W. Tigwell, Roshan L Peiris, Stacey Watson, Gerald M. Garavuso, Heather Miller · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020)

  This experience report examines the challenges and opportunities of teaching and learning American Sign Language (ASL) online during the COVID-19 pandemic, from the perspectives of both students and a Deaf instructor at Rochester Institute of Technology (RIT), home of the…

  deaf and hard of hearing · sign language · education accessibility · videoconferencing · remote learning

• Chat in the Hat: A Portable Interpreter for Sign Language Users

  Larwan Berke, William Thies, Danielle Bragg · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This paper addresses a fundamental limitation of Video Remote Interpreting (VRI) for Deaf and Hard-of-Hearing (DHH) sign language users: existing smartphone-based VRI requires the DHH person to either hold the phone with one hand (reducing signing to one-handed and causing…

  deaf and hard of hearing · sign language · video remote interpreting · augmented reality · workplace accessibility

• Accessibility for Deaf and Hard of Hearing Users: Sign Language Conversational User Interfaces

  Abraham Glasser, Vaishnavi Mande, Matt Huenerfauth · 2020 · Proceedings of the 2nd Conference on Conversational User Interfaces (CUI '20)

  This short CUI 2020 position paper (3 pages, presented at the CUI@CHI workshop) lays out the research agenda for making voice-based conversational user interfaces (CUIs) — Alexa, Google Assistant, and similar personal assistant devices — accessible to Deaf and Hard-of-Hearing…

  deaf and hard of hearing · sign language · conversational user interfaces · personal assistants · american sign language

• Sign Language Interfaces: Discussing the Field's Biggest Challenges

  Danielle Bragg, Meredith Ringel Morris, Christian Vogler, Raja Kushalnagar, Matt Huenerfauth, Hernisa Kacorri · 2020 · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20)

  This CHI 2020 Special Interest Group (SIG) meeting paper (5 pages) is an organising document rather than a conventional research paper. It convenes HCI and accessibility researchers working on sign-language interfaces around the five calls-to-action laid out in Bragg et al.’s…

  sign language · deaf and hard of hearing · deaf culture · american sign language · research methodology

• Tradeoffs in the Efficient Detection of Sign Language Content in Video Sharing Sites

  Caio D. D. Monteiro, Frank M. Shipman, Satyakiran Duggina, Ricardo Gutierrez-Osuna · 2019 · ACM Transactions on Accessible Computing (TACCESS)

  This paper addresses the problem of finding sign language (SL) content on video sharing sites like YouTube, where such videos serve as de-facto digital libraries of Deaf community knowledge, experiences, and culture. Metadata-based search is inadequate — prior work showed only…

  sign language · video analysis · content detection · deaf and hard of hearing · computer vision

• AfricaSign -- A Crowd-sourcing Platform for the Documentation of STEM Vocabulary in African Sign Languages

  Abdelhadi Soudi, Kristof Van Laerhoven, Elmostafa Bou-Souf · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This demonstration paper presents AfricaSign, a web-based crowdsourcing platform designed to enable African Deaf communities to document their endangered sign languages, with a focus on early-grade and STEM vocabulary. Most African sign languages are severely under-studied and…

  sign language · Africa · endangered languages · crowdsourcing · signing avatar

• Delivering Sign Language in a Live Planetarium Show Using Head-Mounted Displays and Infrared Light

  Michael D. Jones, M. Jeannette Lawler · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019)

  This paper presents a system for delivering real-time American Sign Language (ASL) narration during live planetarium shows using head-mounted displays (HMDs) and infrared light. Planetariums pose unique accessibility challenges for deaf and hard-of-hearing (DHH) audiences: the…

  sign language · deaf and hard of hearing · head-mounted display · augmented reality · informal learning

• Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective

  Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, Meredith Ringel Morris · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019)

  This paper provides a comprehensive interdisciplinary overview of sign language processing — encompassing recognition, generation, and translation — produced through a two-day workshop bringing together 39 experts from computer science, linguistics, Deaf studies, and industry.…

  sign language · Deaf culture · computer vision · natural language processing · machine translation

• Effect of Automatic Sign Recognition Performance on the Usability of Video-Based Search Interfaces for Sign Language Dictionaries

  Oliver Alonzo, Abraham Glasser, Matt Huenerfauth · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019)

  This paper investigates how the performance of automatic sign recognition technology affects user satisfaction when searching for unfamiliar words in an ASL-to-English dictionary. Looking up an unknown sign in ASL is fundamentally harder than looking up an unknown written word:…

  sign language · ASL · sign language recognition · dictionary · Deaf and hard of hearing

• How to Develop Accessible Web Interfaces for Deaf People?

  Gênesis Medeiros do Carmo, Débora Maria Barroso Paiva, Maria Istela Cagnin · 2019 · Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems (IHC)

  This paper presents a systematic mapping study examining how web interfaces are being designed and implemented for deaf and hard of hearing users. The authors searched six databases and 17 conferences from 2005-2018, screening 469 publications to select 29 primary studies. The…

  deaf accessibility · hard of hearing · web accessibility · systematic mapping · sign language

• Modeling the Speed and Timing of American Sign Language to Generate Realistic Animations

  Sedeeq Al-khazraji, Larwan Berke, Sushant Kafle, Peter Yeung, Matt Huenerfauth · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '18)

  This paper addresses the challenge of generating realistic computer animations of American Sign Language (ASL) by automatically modeling three critical timing parameters: where prosodic pauses should be inserted, how long those pauses should last, and how the signing speed of…

  sign language · ASL · animation · machine learning · Deaf accessibility

• Designing an Animated Character System for American Sign Language

  Danielle Bragg, Raja Kushalnagar, Richard Ladner · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This University of Washington and Gallaudet University paper introduces the first animated character system prototype for sign language, addressing the fundamental problem that sign languages — used by approximately 70 million people worldwide as their primary language — lack a…

  sign language · deaf accessibility · ASL · writing systems · animation

• Exploring the Performance of Facial Expression Recognition Technologies on Deaf Adults and Their Children

  Irene Rogan Shaffer · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This Boston University student research paper investigates how commercial facial expression recognition services perform on Deaf ASL signers and Children of Deaf Adults (CODAs) compared to hearing non-signers. The study is motivated by a critical problem: in ASL and other sign…

  deaf and hard of hearing · sign language · facial expression recognition · emotion recognition · AI fairness

• Evaluation of Language Feedback Methods for Student Videos of American Sign Language

  Matt Huenerfauth, Elaine Gale, Brian Penly, Sree Pillutla, Mackenzie Willard, Dhananjai Hariharan · 2017 · ACM Transactions on Accessible Computing (TACCESS)

  This paper investigates how to best present video-based feedback to students learning American Sign Language (ASL), as part of a long-term project to build an automatic system that analyses student signing videos and provides immediate corrective feedback. The motivation is…

  sign language · deaf and hard of hearing · ASL education · video feedback · language learning

• Sign'Maths: An Interactive Dictionary in French Sign Language

  Camille Nadal, Christophe Collet · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  Sign'Maths is an online interactive dictionary designed to give deaf students access to mathematical definitions in French Sign Language (LSF). The project emerges from a significant gap in deaf education: while France legally recognized Sign Language as a legitimate educational…

  deaf education · sign language · e-learning · mathematical accessibility · bilingual education

• Sign Language Support System for Viewing Sports Programs

  Tsubasa Uchida, Taro Miyazaki, Makiko Azuma, Shuichi Umeda, Naoto Kato, Hideki Sumiyoshi, Yuko Yamanouchi, Nobuyuki Hiruma · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This demonstration paper from NHK (Japan Broadcasting Corporation) presents a prototype system that provides Japanese Sign Language (JSL) support for deaf and hard-of-hearing viewers watching sports broadcasts. The system was developed in response to strong demand from Japan's…

  sign language · signing avatar · Japanese Sign Language · deaf and hard of hearing · media accessibility

• Speed-Accuracy Tradeoffs for Detecting Sign Language Content in Video Sharing Sites

  Frank M. Shipman, Satyakiran Duggina, Caio D.D. Monteiro, Ricardo Gutierrez-Osuna · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This paper addresses the problem of automatically detecting sign language content in videos on sharing platforms like YouTube and Vimeo. For many deaf and hard-of-hearing people, sign language is their primary communication medium, and they rely on online video content to stay…

  sign language · computer vision · video classification · information retrieval · deaf and hard of hearing

• Design and Psychometric Evaluation of an American Sign Language Translation of the System Usability Scale

  Matt Huenerfauth, Kasmira Patel, Larwan Berke · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper addresses a significant gap in accessibility research methodology: the lack of standardized usability questionnaires available in American Sign Language (ASL) for Deaf and Hard-of-Hearing (DHH) participants. The System Usability Scale (SUS) is one of the most widely…

  sign language · deaf accessibility · usability testing · survey translation · psychometrics

• Evaluating Intelligibility and Battery Drain of Mobile Sign Language Video Transmitted at Low Frame Rates and Bit Rates

  Jessica J. Tran, Eve A. Riskin, Richard E. Ladner, Jacob O. Wobbrock · 2015 · ACM Transactions on Accessible Computing (TACCESS)

  This paper investigates the lower limits of intelligible sign language video on mobile devices, seeking to determine the minimum frame rate and bit rate that still allow deaf users to understand ASL video conversations. Mobile video communication is essential for deaf and…

  deaf and hard of hearing · sign language · video communication · mobile accessibility · bandwidth