Sign Language Support System for Viewing Sports Programs
Tsubasa Uchida, Taro Miyazaki, Makiko Azuma, Shuichi Umeda, Naoto Kato, Hideki Sumiyoshi, Yuko Yamanouchi, Nobuyuki Hiruma · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3132525.3134768
Summary
This demonstration paper from NHK (Japan Broadcasting Corporation) presents a prototype system that provides Japanese Sign Language (JSL) support for deaf and hard-of-hearing viewers watching sports broadcasts. The system was developed in response to strong demand from Japan's deaf communities for more sign language services, compounded by a shortage of human sign language interpreters. Building on prior NHK work that automatically generated JSL computer graphics (CG) animations for weather forecasts using fixed-pattern data, the researchers expanded into the more complex domain of sports, which requires larger vocabulary and real-time content generation. The system architecture consists of three components: a distribution server holding JSL CG animation videos, text, images, and content management XML; a content-generation PC that plays game coverage video and synchronizes metadata; and a smartphone/tablet application that displays the support content. The deliberate choice to deliver sign language on a second screen (mobile device) rather than overlaying it on the television broadcast avoids obscuring the game coverage. The application provides three types of JSL CG animations: game situation explanations (played automatically synchronized with the broadcast), rule commentaries (user-initiated), and game digests (user-initiated). All animations are generated as MP4 videos automatically from game metadata. Beyond sign language, the app complements audio information through device vibration for whistles and buzzers, visual icons representing crowd excitement levels, and text/image displays of scores and player information.
Key findings
The system was demonstrated using a basketball game scenario and evaluated through an opinion survey with four deaf participants (ages 20-40) and one CODA (Child of Deaf Adults) sign language interpreter. Participants watched a three-minute basketball game video with scenes where the referee interrupted play. Positive feedback indicated that JSL animations and support information were most helpful during game interruptions — moments where audio cues (whistles, announcements) carry critical information that deaf viewers miss. Rule explanations in JSL helped participants understand unfamiliar terminology, and player statistics enhanced enjoyment. However, some participants felt JSL video was unnecessary when the game action was self-explanatory visually, and others suggested overlaying JSL on the game video to reduce the eye-movement workload of switching between two screens. The researchers concluded that the ideal design would allow users to access support information on demand at their convenience rather than presenting everything automatically, recognizing that information needs vary by personal preference, sports familiarity, and event type.
Relevance
This work addresses a real gap in broadcast accessibility: live sports rely heavily on audio commentary, referee signals, and crowd reactions that are inaccessible to deaf viewers, yet human sign language interpretation for all sports programming is impractical due to interpreter shortages. The automated CG avatar approach offers a scalable alternative, particularly for structured events where game metadata can drive content generation. For accessibility practitioners in media and broadcasting, the key design insights are valuable: the second-screen approach preserves the viewing experience but creates attention-splitting concerns; on-demand access to supplementary information respects user autonomy; and multimodal support (vibration for audio events, visual icons for crowd noise) addresses information gaps beyond just language translation. The system was developed with an eye toward major international sporting events like the Olympics, Paralympics, and Deaflympics, making it relevant to the growing push for accessible broadcasting worldwide.
Tags: sign language · signing avatar · Japanese Sign Language · deaf and hard of hearing · media accessibility · sports accessibility · computer graphics · broadcast accessibility