Facetop Tablet: Note-Taking Assistance for Deaf Persons
Dorian Miller, James Culp, David Stotts · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169038
Summary
This paper presents Facetop Tablet, a system designed to assist deaf and hard of hearing people in taking notes during meetings, lectures, and group activities. The core problem is that deaf users who rely on sign language interpreters face a constant visual context-switching challenge: they must look at the interpreter to follow the discussion, then look at their notes or the presenter's slides, losing information from whichever source they are not watching. The Facetop Tablet addresses this by overlaying a translucent live video feed of the sign language interpreter directly onto the tablet PC screen where the user is taking notes. An inexpensive webcam captures the interpreter, and the video is rendered as a transparent layer over any application the user is running — a word processor, slide presentation, or web browser. Because the video is translucent, the user can see both the interpreter and their notes simultaneously without switching visual attention between sources. The video layer does not capture keyboard, mouse, or stylus input, so users write through it as if it were not there. The system was developed iteratively with feedback from members of the Deaf community, who identified the note-taking problem and suggested applying the transparent video approach from the parent Facetop remote collaboration project.
Key findings
The system evolved from the Facetop project, originally designed for distributed pair programming, where participants discovered that the visual communication channel enabled natural gesturing and lip reading. Deaf community members who tested Facetop recognised its potential for their note-taking difficulties and suggested the adaptation. Key design features include: adjustable translucency (users can switch to an opaque video window for more detailed viewing of signing), a repositionable and resizable webcam window that can be pointed at the presenter, interpreter, or audience as needed, a "keep away" feature that automatically moves the video out of the way of the mouse/stylus regardless of translucency level, and a recording capability requested by the Deaf community for reviewing lectures, completing assignments, and studying for exams. The translucent video approach retains full screen real estate for both the note-taking application and the video simultaneously, unlike side-by-side layouts that sacrifice space for each. However, the translucency can reduce the understandability of signing as details get washed out, which is why the opaque toggle option was added.
Relevance
This paper addresses a real and underexplored accessibility challenge: the information overload that deaf users experience in multi-source environments like classrooms and meetings. While much deaf accessibility research focuses on captioning or sign language recognition, this work tackles the practical problem of simultaneous visual attention — deaf users cannot watch an interpreter and take notes at the same time the way hearing users can listen and write simultaneously. The transparent video overlay is an elegant solution that demonstrates how creative interface design can reduce cognitive and attentional barriers without requiring new hardware. For practitioners working on meeting accessibility, the key insight is that providing an interpreter or captions is not sufficient if the user cannot effectively divide attention across information sources. The participatory design approach, with the Deaf community identifying the problem and shaping the solution, exemplifies best practice in assistive technology development.
Tags: deaf · hard of hearing · sign language · note-taking · meeting accommodation · transparent video · tablet PC · assistive technology