Learning and Perceiving Colors Haptically
Kanav Kahol, Jamieson French, Laura Bratton, Sethuraman Panchanathan · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169017
Summary
This paper presents a novel system for enabling blind individuals to learn, perceive, and compare colours through haptic (touch-based) feedback. Colour is a purely visual feature that cannot be perceived through auditory or olfactory senses, yet it plays a significant role in spatial perception, object recognition, and social interactions. The system maps colours to textures rendered through a Phantom haptic joystick device. Each colour is represented as a combination of friction levels across four planes that the user explores sequentially: three planes correspond to the red, green, and blue (RGB) components with three quantised friction levels each (none, medium, high), and a fourth plane conveys whether white colour, black colour, or no additional colour is added (controlling lightness/darkness). This encoding allows 21 distinct colours to be represented. Users explore each plane by moving the joystick probe left to right to feel the friction level, then press a button to advance to the next plane. The system was designed in consultation with focus groups of blind individuals and assistive technology experts, who emphasised that the common temperature-based analogy for colour (yellow is hot, blue is cold) is too limited and widely used only due to a lack of alternatives. A real-time system using CyberTouch gloves with vibratory motors on five fingers and the palm was also developed, encoding colour information as patterns of dot and dash pulses across fingers.
Key findings
Two experiments were conducted with four groups: blind individuals tested blindfolded (B), sighted individuals tested blindfolded (SB), sighted individuals trained and tested sighted (S), and a crossover group (S/SB). In Experiment 1 (colour recognition), all groups achieved 100% recognition accuracy after learning trials, including congenitally blind individuals who had never experienced colour. The sighted-blindfolded group showed the slowest initial learning rate but caught up quickly. In Experiment 2 (colour similarity perception), multidimensional scaling (MDS) was applied to pairwise similarity judgements between 21 colours. The resulting scaled spaces achieved congruency values of 0.87-0.92 with the HSV colour wheel in both 2D and 3D representations — meaning participants perceived colour relationships through the haptic system in a manner highly consistent with how colours are visually organised. Critically, there was no significant difference between trained and control (untrained) groups, demonstrating that the system conveys inherent perceptual colour relationships rather than just learned labels. The real-time CyberTouch glove system achieved 98.5% colour recognition accuracy after 4 learning trials, with subjects who performed the haptic joystick experiment first showing better performance than those who learned colour through the glove system alone.
Relevance
This research challenges the assumption that colour is inherently inaccessible to blind individuals and demonstrates that meaningful colour perception — not just colour naming but understanding colour relationships and similarity — can be conveyed through haptic feedback. The finding that congenitally blind participants (who have never seen any colour) could learn to recognise and compare colours, and that their perceived colour similarity spaces matched the visual colour wheel, is remarkable and has implications for art education, fashion, interior design, and other colour-dependent activities from which blind people are typically excluded. The dual-system approach (force feedback joystick for learning and exploration; vibrotactile glove for real-time perception) addresses different use cases. The system also has potential applications for people with various types of colour blindness (achromatopsia, tritanopia, deuteranopia) who experience colour differently rather than not at all. The participatory design approach — developing the system in consultation with blind users and assistive technology experts — ensured that the chosen mapping addressed real needs rather than assumptions about what blind people want from colour information.
Tags: visual impairment · haptic interface · colour perception · sensory substitution · assistive technology · congenital blindness · tactile display · colour blindness