An Evaluation of Video Intelligibility for Novice American Sign Language Learners on a Mobile Device
Kimberly A. Weaver, Thad Starner, Harley Hamilton · 2010 · Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2010) · doi:10.1145/1878803.1878824
Summary
This paper investigates what video resolution is necessary for novice signers to learn American Sign Language (ASL) from video on a mobile device. The motivation is deeply practical: approximately 77% of deaf children are born to hearing parents who must learn sign language to communicate with their children, yet classes may be far away and difficult to schedule around caring for young children. Mobile-based ASL learning could provide on-demand access during free moments. The study used a vocabulary of 80 signs from the MacArthur-Bates Communicative Development Inventory — signs commonly used between parents and young children across categories including adjectives, animals, food, pronouns, verbs, and routines. Twenty participants (mean age 26.55, 15 male, 5 female, no prior signing experience) viewed sign videos on a Motorola DROID phone (854x480 native resolution) across four conditions: high resolution (640x480), medium (320x240), low (160x120), and high resolution with handshape insets. Videos were recorded by a hearing signer with 20 years of ASL experience, encoded in H.264 at 25fps with average duration of 3.18 seconds. After viewing each sign, participants attempted to reproduce it, and a sign language linguist scored their reproductions on handshape, motion, location, and orientation.
Key findings
The central finding was that video resolution had no significant effect on participants' ability to correctly reproduce signs (F(3,17) = 2.40, p = 0.08). While participants noticed and rated the low resolution condition as significantly lower quality (median 3.5 vs. 5.0 for medium, high, and inset), this perceived quality difference did not translate to worse sign production. Sign generation times were also unaffected by resolution condition. Sign difficulty, however, had a significant correlation with reproduction accuracy (r = 0.22, p = 0.049) — easier signs received higher scores. The handshape inset condition received the most negative feedback: 8 of 20 participants disliked it, finding it difficult to attend to both the insets and full-body motion simultaneously, though some found insets helpful for medium-difficulty signs. Participants were inconsistent in their hand dominance strategy when interpreting videos, with some mirroring the signer and others matching — highlighting the need for explicit instruction on dominant hand roles. Only four signs received average scores below 6 out of 7 points, demonstrating that even complex signs could be learned from mobile video with relatively high accuracy.
Relevance
This research has important implications for mobile-based sign language education and accessibility. The finding that low-resolution video does not degrade learning ability means sign language teaching apps can use smaller file sizes (approximately 129 KB vs. 340 KB per video), enabling more signs to be stored locally on devices and reducing bandwidth requirements — critical for deployment in areas with limited connectivity. For practitioners building sign language learning tools, the key design recommendations are: prioritize playback speed control and video replay over high resolution; address hand dominance explicitly (mirrored vs. matched signing); and consider that sign difficulty matters more than video quality. The work fed into the SMARTSign (Support Made Available in Real-Time) project, a comprehensive system allowing parents to reference signs by speaking words, learn vocabulary through quizzes, and practice by comparing their own signing to video models. The study validates that mobile devices are a viable platform for sign language instruction, removing a significant barrier for hearing parents of deaf children.
Tags: American Sign Language · deaf education · mobile learning · video intelligibility · computer-assisted language learning · sign language