A General HCI Framework of Sonification Applications
Ag Asri Ag Ibrahim, Andy Hunt · 2006 · Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '06) · doi:10.1145/1168987.1169025
Summary
This paper proposes a general HCI framework for understanding and evaluating sonification applications — software that uses non-speech sound to represent data or information. The authors define sonification as having three main elements: Goal (for understanding), Method (how to represent using non-speech sound), and Input/Output (information and non-speech sound). The framework is built on Norman's Model of HCI and comprises two complementary models. The Sonification Application (SA) Model explains what the application or designer would like the user to do and know, addressing how data is transformed through four stages: raw data (RD) is transformed into soundable data (SD), then into acoustic reality data (ARD) with acoustic attributes, and finally into the sound representation (SR) heard by the user. The User Interpretation Construction (UIC) Model addresses what the user might perceive, interpret, and understand, modelling the cognitive process from initial sensation through to useful mental representation.
Key findings
The SA Model adopts the Data State Model to describe the transformation chain in sonification: raw data undergoes data transformation to become soundable, then acoustic transformation to become acoustic reality data, and finally sound representation transformation to become the audible output. The UIC Model describes four levels of user engagement with sonification output: interaction level (first contact with sound, producing sensation), selection level (producing conditions — questions like "Can I hear the high pitch?" or "Can I identify the object?"), organisation level (producing predictions about what the sound represents), and interpretation level (producing hypotheses about the data's meaning). The framework positions sonification as particularly useful for blind or visually impaired users, and for situations where a user's eyes are occupied with other tasks such as driving, monitoring patients, or navigating environments. The authors propose using the framework as a basis for producing usability inspection materials specifically tailored to sonification applications, addressing a gap in current evaluation methods.
Relevance
This paper provides a theoretical foundation for designing and evaluating sonification applications, which is valuable for accessibility practitioners working on non-visual data representation. The clear separation between the application's transformation of data into sound (SA Model) and the user's cognitive interpretation of that sound (UIC Model) helps identify where breakdowns occur — a sonification may correctly encode data but fail if users cannot extract meaning from the sound. The four-level UIC Model (sensation, conditions, predictions, hypotheses) offers a structured way to evaluate whether a sonification is actually accessible, not just audible. For practical work, the framework's emphasis on creating usability inspection materials for sonification could help standardise evaluation of audio-based accessibility tools and data visualisation alternatives.
Tags: sonification · auditory display · HCI framework · usability inspection · parameter mapping · accessibility