Evaluation of a psycholinguistically motivated timing model for animations of American Sign Language
Matt Huenerfauth · 2008 · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '08) · doi:10.1145/1414471.1414496
Summary
This paper investigates how psycholinguistic insights about the timing and speed of American Sign Language (ASL) can improve computer-generated sign language animations. The author developed two algorithms — a sign-duration algorithm and a pause-insertion algorithm — grounded in published research on how native ASL signers naturally vary their signing speed and where they place pauses. The sign-duration algorithm uses the Grosjean and Lane model, which accounts for three timing components: transition rate (hand movement speed between signs), number and location of pauses, and length of each pause. The algorithm calculates timing values based on syntactic structure, assigning boundary strengths between signs according to clause and sentence boundaries and using part-of-speech information. The pause-insertion algorithm identifies linguistically appropriate locations for pauses based on sentence boundaries, then allocates pause time proportionally to the highest-ranked boundary points. Twelve ASL passages were animated using the SignSmith Studio platform at three speeds (normal, fast, very fast), with and without the timing algorithms applied, producing 72 animation files. A within-subjects evaluation study was conducted with native ASL signers who viewed the animations on a 17-inch LCD screen, answered comprehension questions, and provided Likert-scale ratings for grammaticality, understandability, naturalness of movement, and speed.
Key findings
The timing algorithms led to a significant increase in comprehension task performance — this held true both across all speeds combined and specifically at "normal" speed. Animations with linguistically motivated pauses were rated significantly higher for Understandability on the Likert scales. For the "normal" speed animations (closest to natural signing rate), the Speed scores were closest to 10 ("perfect") on the 21-point scale, confirming the timing felt natural. Importantly, statistically significant differences were found only between pause and no-pause conditions at normal speed, not at fast or very fast speeds, suggesting the pauses are most beneficial when the overall animation rate allows viewers to process the linguistic structure. Comprehension scores plotted by signs-per-second showed that animations with pauses that had been sped back up to match the original duration (so they had the same total time) still outperformed animations without pauses, indicating the benefit comes from pause placement at linguistically appropriate locations rather than simply from added time. Most participant feedback focused on aspects like fingerspelling clarity and the naturalness of specific signs rather than timing, suggesting the timing felt sufficiently natural not to be a distraction.
Relevance
This research addresses a critical gap in making web and digital content accessible to deaf users who are native ASL signers. Many deaf adults have limited English reading fluency, making text-based content inaccessible — computer-generated ASL animations could provide an alternative, but only if they are comprehensible. The finding that linguistically motivated timing significantly improves both comprehension and perceived understandability has direct implications for anyone developing sign language avatar technology for websites, kiosks, or applications. The work demonstrates that it is not enough to simply play signs in sequence — the prosodic structure of the language (pauses, speed variations) must be modelled for effective communication. For accessibility practitioners, this underscores a broader principle: automated accessibility solutions must respect the linguistic properties of their target modality. The algorithms described are designed to be applicable to other sign languages beyond ASL, and the experimental framework provides a model for evaluating sign language generation systems with native signers rather than relying solely on technical metrics.
Tags: sign language · American Sign Language · animation · natural language generation · deaf accessibility · psycholinguistics · sign language synthesis