Tactile Emotions: Multimodal Affective Captioning with Haptics Improves Narrative Engagement for d/Deaf and Hard-of-Hearing Viewers
Caluã de Lacerda Pataca, Saad Hassan, Lloyd May, Michelle M Olson, Toni D'aurio, Roshan L Peiris, Matt Huenerfauth · 2025 · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25)
This paper explores a multimodal approach to affective captioning — captions that convey not just the words a speaker says but the emotional dimensions of their voice — for d/Deaf and Hard-of-Hearing (dhh) viewers. Prior work has shown that typographic modulations such as…
affective captioning · haptics · deaf and hard of hearing · multimodal interaction · captions