A Web-Based Intelligibility Evaluation of Sign Language Video Transmitted at Low Frame Rates and Bitrates
Jessica J. Tran, Rafael Rodriguez, Eve A. Riskin, Jacob O. Wobbrock · 2013 · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2513383.2513432
Summary
This paper evaluates the perceived lower limits of intelligible sign language video for mobile communication, testing how much video quality can be sacrificed while maintaining comprehensibility for ASL users. The ITU-T recommends sign language video be transmitted at 25 fps at 100 kbps or higher, but this places heavy demands on mobile network bandwidth. The researchers created a national web survey where 99 respondents (77 included in analysis; 56 deaf, median age 40, median 28 years of ASL experience) watched 16 short ASL videos of a native signer performing sentences. Videos were shown in a full factorial design across four low frame rates (1, 5, 10, and 15 fps) and four low fixed bitrates (15, 30, 60, and 120 kbps), displayed at 320x240 pixels within a simulated mobile phone display. Each video could only be watched once, and participants rated intelligibility on a 7-point Likert scale and answered comprehension questions. The paper also introduces the Human Signal Intelligibility Model (HSIM), a conceptual framework that extends Shannon's communication theory to distinguish signal intelligibility (the capability of a signal to be understood) from signal comprehension (intelligibility plus the receiver's knowledge), and both from objective video quality measures like PSNR.
Key findings
The most striking finding was an "intelligibility ceiling effect": videos at 10 fps received the highest mean Likert scores for intelligibility across all bitrates, and increasing to 15 fps actually decreased perceived intelligibility significantly (F(1,1139)=77.22, p<.0001). This counterintuitive result challenges the assumption that higher frame rates always improve sign language video — at low bitrates, allocating more bits per frame (fewer frames) produces better results than spreading the same bandwidth across more frames. For bitrate, increasing from 15 to 60 kbps significantly improved intelligibility, but the difference between 60 and 120 kbps was not significant (F(1,1139)=4.62, n.s.), indicating diminishing returns. Videos at 1 fps were clearly too low for ASL conversations. Comprehension question response times showed a strong negative correlation with intelligibility ratings (R=-0.66 for frame rate, R=-0.82 for bitrate), suggesting response time could serve as a proxy measure for intelligibility. These findings suggest that the ITU-T recommended transmission rates can be relaxed to 10 fps at 60 kbps — reducing total bandwidth consumption to 25% of the current standard — while still providing intelligible ASL video.
Relevance
This research has direct practical implications for making mobile sign language video communication more accessible and affordable. By demonstrating that intelligible ASL video can be transmitted at dramatically lower bandwidth than currently recommended, the findings support extending video relay services and mobile sign language calls to users with limited data plans or in areas with poor network coverage — both of which disproportionately affect deaf and hard of hearing communities. The HSIM conceptual model provides a useful theoretical framework for any evaluation of communication signal quality, clearly separating technical video quality from human-centered intelligibility and comprehension. For accessibility practitioners and telecommunications developers, the key takeaway is that optimizing for perceived intelligibility rather than raw video quality metrics can dramatically reduce bandwidth requirements. The intelligibility ceiling effect is particularly important: blindly increasing frame rate at low bitrates can actually hurt rather than help sign language comprehension.
Tags: sign language · American Sign Language · deaf accessibility · video compression · frame rate · bitrate · telecommunications accessibility · video quality · mobile video
Standards referenced: ITU-T Q.26/16