Visualization of Speech Prosody and Emotion in Captions: Accessibility for Deaf and Hard-of-Hearing Users
Caluã de Lacerda Pataca, Matthew Watkins, Roshan Peiris, Sooyeon Lee, Matt Huenerfauth · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)
This CHI 2023 paper tackles a dimension of captioning that has gone largely unaddressed for four decades: captions depict words but strip out the prosody and emotion carried by a speaker's voice. The authors argue that while automatic speech recognition (ASR) has reduced word…
captioning · deaf and hard of hearing · prosody · affective computing · videoconferencing accessibility