A Linguistically Motivated Model for Speed and Pausing in Animations of American Sign Language
Matt Huenerfauth · 2009 · ACM Transactions on Accessible Computing · doi:10.1145/1530064.1530067
Summary
This paper tackles a critical accessibility challenge: many deaf adults in the United States have English reading levels below average 10-year-old hearing students, creating barriers to accessing written web content. Computer-generated animations of American Sign Language (ASL) offer a potential solution, but producing natural-looking animations requires more than just stringing signs together—the timing subtleties are crucial. Prior ASL animation research had focused on individual signs rather than the timing dynamics of connected discourse. The author developed two algorithms grounded in psycholinguistic research on human ASL signing. The sign-duration algorithm modifies how long each sign is displayed based on linguistic context: repeated verbs, adjectives, and adverbs are shortened by 12%, while signs appearing just before sentence or clause boundaries are lengthened by 12%. The pause-insertion algorithm determines where to place pauses and how long they should be, building on Grosjean and Lane's model that analyzed pause patterns in human signers. The algorithm considers syntactic complexity (pauses occur at major grammatical boundaries) and a "bisection tendency" (signers prefer to pause near the middle of long spans). Pauses occupy about 17% of total animation time, matching human signing patterns.
Key findings
Two evaluation studies with native ASL signers (12 participants in the first, 18 in the second) demonstrated significant benefits from the timing algorithms. In the first study, animations with linguistically motivated pauses achieved significantly higher comprehension scores than animations without pauses across all speeds tested. Participants correctly answered comprehension questions about 45% more often when viewing pause-processed animations versus unprocessed ones. The animations with pauses were also rated significantly higher for understandability. The second study used a more rigorous design where pause-processed animations were sped up to match the total duration of unprocessed animations, isolating the benefit of pause placement from simply adding time. Even under these conditions, pause-processed animations showed significantly higher comprehension scores, confirming that the location of pauses matters—not just their presence. Signers preferred animations displayed at approximately 1.1-1.2 signs per second, notably slower than the 1.5-2.37 signs/second reported for human ASL in linguistics literature. This suggests computer-generated animations may need slower speeds than human signing to achieve similar comprehensibility, likely due to limitations in facial expressions, eye gaze, and other non-manual features.
Relevance
This research provides actionable guidance for developers of ASL animation systems, including automatic English-to-ASL translation tools and content authoring systems like SignSmith. The key insight is that timing is not a single "speed" parameter—it involves the interplay of sign durations, pause locations, and pause lengths, all influenced by linguistic structure. Simply adjusting playback speed misses these nuances. For practitioners building accessible content for deaf users, the study establishes that animations should be displayed slower than typical human signing (around 1.1 signs/second) and that pauses should be inserted at syntactically appropriate boundaries. The algorithms require linguistic information (sentence boundaries, parts of speech, syntactic parse trees) that would need to be provided by content authors or generated automatically in machine translation contexts. While the study focused on ASL, the approach should be applicable to other signed languages. Limitations include the relatively small participant samples and the current inability to automatically parse ASL sentences—future work could integrate natural language processing to automate the linguistic analysis.
Tags: American Sign Language · sign language animation · deaf accessibility · natural language generation · virtual humans · machine translation