An Easy to Use Data Logger for Local User Studies
Vagner Figueredo de Santana, Felipe Eduardo Ferreira Silva · 2019 · Proceedings of the 16th International Web for All Conference (W4A) · doi:10.1145/3315002.3332437
Summary
This demonstration paper introduces User Test Logger, an open-source Firefox browser add-on designed to capture detailed user interaction data during local usability and accessibility studies. The authors identify a gap in available tooling: existing interaction logging tools are either commercial (paid), have limited capabilities for capturing and downloading large datasets, or require complex setup procedures. Most also focus narrowly on click streams and mouse movements, which assumes sighted mouse-based interaction and fails to represent the diversity of how people interact with the web. User Test Logger addresses these limitations by supporting capture of any JavaScript or jQuery event type — a vocabulary of 40 events including keyboard, touch, geolocation, and device orientation events — making it capable of logging interactions from users who rely on keyboards, touchscreens, or assistive technologies rather than a mouse. The tool is built on a Model-View-Controller architecture within the Firefox web extension framework: the background component handles data storage and communication, a popup component provides controls and reporting, and content scripts capture and format UI events on each page. Four design pillars guide the tool: inclusive data capture, privacy protection (data stays local rather than being transmitted to external servers), non-disruption of participants during sessions, and inclusive reporting of captured data. Setup is straightforward — practitioners load the add-on as a temporary extension and configure which events to capture through the browser UI.
Key findings
The tool provides five types of built-in reports for analyzing interaction data. The usage graph represents interaction as a directed graph of walks showing what actions users performed, where (which UI elements), and when, using DOM paths to uniquely identify elements even without id attributes. A SAM-based heuristic automatically identifies usability problems and accessibility barrier candidates by detecting cyclic actions and deviations from expected task flows. The patterns report surfaces UI events whose occurrences exceed the 80th percentile, highlighting repeated actions along with their source and subsequent events. The incidents report shows events flagged by the heuristic as potential problems. A mouse fixation heatmap uses a dispersion algorithm (analogous to eye-tracking fixation detection) to identify areas of sustained mouse attention. A mouse plot visualizes movement paths, clicks, and double-clicks for comparing task performance across participants. All data can be exported in multiple formats (DOT, PNG, CSV) for use in external statistical and graph analysis tools. The tool is particularly notable for its inclusive approach to what constitutes interaction data — by going beyond click streams, it can represent interaction patterns of users who navigate with keyboards, screen readers, or other assistive technologies.
Relevance
This tool has practical value for accessibility practitioners conducting user studies with people with disabilities. Most web analytics and usability testing tools are biased toward mouse-based interaction, capturing clicks and cursor movements as their primary data. This approach inherently excludes or misrepresents the interaction patterns of people who use keyboards, screen readers, switch devices, or other alternative input methods. By supporting the full range of JavaScript events — including keyboard, focus, touch, and device orientation events — User Test Logger enables researchers to capture and analyze how assistive technology users actually interact with web content. The local-only data storage model also addresses privacy concerns that are particularly important when conducting research with vulnerable populations. For organizations conducting accessibility audits or user testing with people with disabilities, this tool offers a free, open-source option for gathering quantitative interaction data to complement qualitative observations and think-aloud protocols. The automatic identification of cyclic actions and deviations could help pinpoint specific accessibility barriers where users get stuck or lost.
Tags: usability testing · accessibility evaluation · interaction logging · data collection · open source tools · user studies · web accessibility