Sign Language Interfaces: Discussing the Field's Biggest Challenges
Danielle Bragg, Meredith Ringel Morris, Christian Vogler, Raja Kushalnagar, Matt Huenerfauth, Hernisa Kacorri · 2020 · Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA '20) · doi:10.1145/3334480.3381053
Summary
This CHI 2020 Special Interest Group (SIG) meeting paper (5 pages) is an organising document rather than a conventional research paper. It convenes HCI and accessibility researchers working on sign-language interfaces around the five calls-to-action laid out in Bragg et al.’s 2019 ASSETS interdisciplinary review of sign-language recognition, generation, and translation. The authors—from Microsoft Research, Gallaudet University, RIT, and the University of Maryland—note that roughly 17.5 million people worldwide use a signed language, with about twice as many non-deaf users (CODAs, interpreters, students, family members) whose daily interactions also depend on these interfaces. Most existing digital interfaces assume a spoken or written language, which excludes users for whom a sign language is the primary language. The absence of a standard writing system for signed languages has knock-on effects: the U.S. Census Bureau does not count ASL as a non-English language because it is unwritten; search engines cannot be queried in sign; and voice assistants neither accept sign input nor render sign output. Recent advances in machine learning and deep learning (sign recognition, pose estimation, neural machine translation) have lowered the technical barriers enough that sign-language interfaces are tractable research problems, but the field still lacks agreed design guidelines, evaluation metrics, notation standards, and public datasets.
Key findings
As a SIG document the contribution is organisational rather than empirical, but the five calls-to-action it endorses are substantive. Call 1 — Partnering with the Deaf community: sign-language technology research must involve Deaf team members throughout rather than at the end, to respect community ownership over the language and surface insights only the lived Deaf experience provides. Call 2 — Real-world applications: the field should target applications that are both technically feasible and genuinely valuable to Deaf users (not proof-of-concept demos). Call 3 — User interface guidelines: general design guidelines and evaluation metrics for sign-language interfaces do not yet exist, so researchers keep re-discovering the same patterns; codifying them would accelerate progress. Call 4 — Public, representative dataset curation: existing sign-recognition datasets are small, narrow, and often not Deaf-centric; public, diverse, Deaf-centric datasets are needed for deep learning at scale. Call 5 — Notation standards: standardising a sign-language notation system would enable dataset collection, labelling, merging, and eventually routine reading and writing in a sign language. The SIG format allocated 10 minutes for introductions, 50 minutes for five small-group discussions (one per call), and 15 minutes for wrap-up, with interpreters present.
Relevance
For accessibility practitioners, this paper is most useful as a pointer to Bragg et al.’s 2019 ASSETS review, which remains the canonical research agenda for sign-language interface work. The SIG paper itself is a snapshot of a field that has since grown substantially — Mande et al.’s 2021 work on DHH wake-up approaches, Hassan et al.’s 2022 hybrid ASL dictionary search, and numerous follow-on papers from CAIR, Gallaudet, and Microsoft Research respond directly to calls 1-3. The recurring message for practitioners is that sign-language interfaces are not solved by bolting captions onto a voice assistant: they require sign-first input and output, Deaf-led research design, and community-curated datasets. Limitations are those of a SIG format: the paper reports no primary research, the references are sparse, and the five-page format precludes deeper treatment of any single call-to-action. Read as a field-organising document alongside the 2019 ASSETS review.
Tags: sign language · deaf and hard of hearing · deaf culture · american sign language · research methodology · datasets · accessibility research · participatory design
Standards referenced: Section 508 · CRPD · European Accessibility Act