Jod: Examining Design and Implementation of a Videoconferencing Platform for Mixed Hearing Groups
Mittal, Anant, Gupta, Meghna, Poddar, Roshni, Naik, Tarini, Kuppuraj, Seethalakshmi, Fogarty, James, Kumar, Pratyush, Jain, Mohit · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608382
Summary
This paper presents the design, implementation, and evaluation of Jod (a Hindi word meaning "link"), a web-based videoconferencing platform purpose-built for mixed hearing group communication. Current platforms like Zoom, Teams, and Google Meet present significant accessibility barriers for the 430 million d/Deaf or hard of hearing (DHH) people worldwide: limited layout customization forces DHH users to struggle to see interpreters and signers simultaneously; audio-centric designs highlight speaking participants rather than signing ones; and there are no built-in mechanisms for DHH participants to politely interject or request accommodations. Jod addresses these challenges through four key features: fully customizable visual layouts where any video tile, caption box, or screen share can be freely resized, repositioned, locked, or removed; preset feedback messages (e.g., "Please look at me," "Please speak slower," "Please use easier language") that allow DHH participants to request accommodations without interrupting conversation flow; active signer identification that highlights signing participants' video tiles (implemented via a Wizard of Oz method due to the lack of reliable automated sign detection); and accessibility indicators that display each participant's hearing ability and role (Deaf, Hearing, or Interpreter) through color-coded icons. The platform was built with React, Node.js, Microsoft Azure Communication Services, and Socket.IO, and adheres to WCAG accessibility guidelines.
Key findings
Six study sessions with 34 participants (18 DHH, 10 hearing, 6 Indian Sign Language interpreters) in Bengaluru, India revealed rich insights across approximately 10 hours of use. Participants made 485 visual layout rearrangements and sent 40 preset feedback messages. A statistically significant correlation emerged between hearing ability and layout preferences: DHH participants allocated the interpreter's video tile nearly twice the visual space (59.7% average) compared to hearing participants (33.7%), and frequently minimized or removed hearing participants' tiles entirely. Layout arrangements were dynamic — participants actively reorganized during charades games to prioritize whoever was acting. The interpreter's tile remained consistently prominent even during screen sharing, competing with the shared content for visual real estate. Preset feedback messages successfully facilitated cross-hearing-ability communication: DHH participants used them to get interpreters' attention, and hearing participants used them to directly interact with DHH participants for the first time. However, notifications were sometimes obtrusive — they overlapped video tiles, lacked acknowledgment mechanisms, and were particularly disruptive for interpreters who couldn't easily dismiss them while signing. Cultural communication norms differed significantly: DHH participants used camera flashing and expressive gestures as backchannel feedback, while hearing participants relied on audio cues, creating mismatched expectations. Interpretation lag created power imbalances, as hearing participants would react to jokes or information before DHH participants received the interpretation.
Relevance
This study provides essential design guidance for making videoconferencing accessible to mixed hearing groups — a pressing need given the post-pandemic surge in remote work and virtual meetings. The core finding that DHH users need fundamentally different visual layouts, not just captions added to standard designs, challenges the incremental accessibility approach taken by mainstream platforms. For practitioners building or procuring videoconferencing tools, the concrete recommendations are immediately actionable: provide hearing-ability-based layout templates as starting defaults, implement context-aware notification systems with priority levels and acknowledgment mechanisms, support sign name profiles for participant identification, and design automated attention-grabbing features that respect Deaf cultural practices. The study's focus on Indian Sign Language users in India also broadens the predominantly Western, ASL-focused DHH accessibility literature. The tension between user flexibility and system automation — participants wanted full control but found the manual labor burdensome — is a design challenge applicable far beyond videoconferencing.
Tags: deaf and hard of hearing · videoconferencing · sign language · mixed hearing groups · visual layout · captioning · accessible communication · Indian Sign Language · participatory design
Standards referenced: WCAG