Rock Vibe: Rock Band® Computer Games for People with No or Limited Vision
Troy Allman, Rupinder K. Dhillon, Molly A.E. Landau, Sri H. Kurniawan · 2009 · Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '09) · doi:10.1145/1639642.1639653
Summary
This paper presents Rock Vibe, a modification of the commercial Rock Band® video game that replaces visual cues with haptic vibrations and audio feedback to enable people with no or limited vision to play the drumming component. The system uses an Arduino microcontroller to drive vibration motors placed on the player's upper and lower arms (mapping to four drumhead colors) and ankle (for the kick drum pedal). The software captures the Rock Band video feed, tracks colored pixels corresponding to each drum cue, and triggers the appropriate vibration motor when a note approaches the hit zone. Correct hits produce distinct drumming sounds (snare, hi-hat, tom-tom, crash cymbal, kick drum), while errors produce a clicking sound. The computer's speech synthesizer vocalizes menus, song titles, instructions, and scores. The project went through multiple iterative design phases: initial brainstorming with six blind and visually impaired participants, a focus group discussion, a heuristics evaluation with six legally blind participants using Nielsen's heuristics, system revision based on feedback, and a final user evaluation with seven participants ranging in age from 9 to 37. The work built on Blind Hero, an earlier project that modified Guitar Hero using haptic gloves, but extended it to the more complex five-input drumming paradigm.
Key findings
The final evaluation with seven participants (one totally blind, two detecting only light and sharp contrasts, one with central vision loss) showed that the system was quickly learnable — some participants made zero errors halfway through their first song. Participants were tested on two songs (Weezer's "Say It Ain't So" and Yeah, Yeah, Yeah's "Maps") of similar difficulty. Error analysis revealed that errors tended to cluster at the beginning of songs and decreased as players gained familiarity, with errors being a combination of late hits and hitting the wrong drumhead. The heuristics evaluation identified several design issues that were subsequently addressed: the error beep sound was changed to a less aggressive click, correct-hit sounds were differentiated by drum type, and the arm-to-drumhead mapping was swapped to be more intuitive (left wrist = yellow drum, right arm = blue, etc.). A key finding from the focus group was that haptic feedback on body parts mapped to spatial locations was surprisingly easy to learn, though participants worried it would be difficult — suggesting that body-mapped haptic cues may be more intuitive than expected. Sighted players who also tried the system noted that they actually enjoyed the non-visual mode, as the screen backdrop was visually distracting.
Relevance
Rock Vibe demonstrates that mainstream commercial games can be made accessible to blind players through sensory substitution without fundamentally altering the gameplay experience. The iterative, participatory design process — involving blind and visually impaired users at every stage from brainstorming through final evaluation — exemplifies best practice in accessible technology development. The finding that body-mapped haptic feedback is quickly learnable has implications beyond gaming for any interface that needs to convey spatial or categorical information non-visually. The observation that sighted participants also preferred the haptic mode over the visual display hints at universal design benefits. However, the system's reliance on external hardware (Arduino, vibration motors, straps) and video feed processing highlights the gap between research prototypes and commercially viable accessible gaming solutions — a gap that the game accessibility community continues to work on closing. The project also raises the important social dimension of gaming: enabling blind players to participate in the same multiplayer experiences as sighted peers.
Tags: game accessibility · visual impairment · blindness · haptic feedback · sensory substitution · inclusive design · music accessibility