Expression: A Google Glass Based Assistive Solution for Social Signal Processing
ASM Iftekhar Anam, Shahinur Alam, Mohammed Yeasin · 2014 · Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS) · doi:10.1145/2661334.2661348
Summary
This demo paper presents Expression, a Google Glass-based assistive system that enables blind and visually impaired users to perceive non-verbal social signals during face-to-face conversations. Limited access to non-verbal cues — facial expressions, gestures, body language — is a significant barrier to social interaction for blind people, contributing to social isolation and depression. Expression uses the Google Glass camera to capture the interlocutor's face during dyadic (two-person) conversation, processes the image to predict social signals including facial appearance features, behavioural expressions, and emotions, and provides real-time speech feedback through the Glass earbuds. The system consists of three main modules: a data acquisition module that captures video frames from the Glass camera, a facial and behavioural expression module that extracts and classifies facial features, and a social signal inference module that interprets these features to identify emotions and social signals. The system was developed through participatory design with iterative refinements. Expression builds on the team's prior work with IMAPS (which predicted affective dimensions — valence, arousal, dominance — in conversations but was smartphone-based) and iFEPS (a sensory substitution system providing auditory notification of facial expression changes, also smartphone-based). The Google Glass form factor addressed limitations of smartphone-based predecessors: the head-mounted camera naturally faces the conversation partner, and the earbuds provide private audio feedback without requiring headphones or holding a device.
Key findings
Subjective evaluation using a 5-point Likert scale rated Expression as excellent (mean score 4.383). The Google Glass platform offered key advantages over prior smartphone-based systems: the camera is always oriented toward the conversation partner (eliminating the aiming problem of phone cameras for blind users), the form factor is hands-free, and the bone-conduction audio output allows the user to hear both the system's feedback and the conversation simultaneously. However, the paper notes several challenges with the Google Glass hardware: significant heating problems during extended use, short battery life, and limited bandwidth for data transmission. The system's real-time social signal processing represents a practical application of affective computing for accessibility — converting visual social information that blind people cannot perceive into an audio channel they can. The participatory design approach and iterative development suggest that the system was shaped by actual user needs rather than purely technical capabilities.
Relevance
This work addresses an often-overlooked dimension of blindness: not just the inability to see physical objects, but the loss of social information that sighted people process unconsciously during conversations. Facial expressions, smiles, frowns, eye contact, and head nods carry critical social meaning — agreement, confusion, boredom, empathy — that blind conversationalists miss entirely. For accessibility practitioners, Expression illustrates how wearable computing can function as a "social prosthesis" by translating visual social signals into audio. The concept has become more relevant as smart glasses technology has advanced since 2014, with lighter, more capable devices available. While Google Glass itself was discontinued as a consumer product, the underlying approach — real-time facial expression recognition delivered through wearable audio — has been pursued by numerous subsequent projects and commercial products. The system also raises important design questions about information overload: how much social signal feedback can a blind user process during a conversation without it becoming distracting? The paper does not address this question in depth, but it remains a key challenge for real-world deployment of social signal assistive technology.
Tags: blindness · wearable technology · facial expression recognition · social interaction · Google Glass · affective computing · computer vision · sensory substitution