Supporting the selection of web content modality based on user interactions logs
Fabiano Marcon de Moraes, Vagner Figueredo de Santana, Juliana Cristina Braga · 2016 · Proceedings of the 13th International Web for All Conference (W4A) · doi:10.1145/2899475.2899500
Summary
This paper explores using machine learning to automatically detect whether a web user is employing assistive technology based solely on their interaction patterns during a single pageview, without requiring explicit profile configuration or customization. Grounded in Universal Design principles—particularly Flexibility of Use, Perceptible Information, Equitable Use, and Low Physical Effort—the approach aims to enable automatic selection of appropriate content modalities (text, audio, video) for individual users. The researchers collected interaction data from 261 sessions on the Todos Nós Portal (a Brazilian university accessibility support website) using the WELFIT logging tool, with 31 sessions from self-identified assistive technology users. Each session generated an "event fingerprint" consisting of counts for 23 interaction event types (click, keydown, mousemove, focus, blur, etc.) captured during the homepage visit only. The study framed the problem as a binary classification task—AT user versus non-AT user—and systematically evaluated 70 classification algorithms using the Weka data mining toolkit with 10-fold cross-validation. Class imbalance (31 AT vs 230 non-AT) was addressed using resampling to balance classes before algorithm evaluation.
Key findings
The Random Committee algorithm (an ensemble of Random Forest classifiers with different random seeds) achieved the best results with 94.8% precision and 94.3% recall. The top five algorithms were all tree-based or instance-based: Random Committee (94.8%/94.3%), Random Forest (93.5%/92.3%), IBk/k-nearest-neighbor (92.7%/91.6%), Random Tree (92.6%/92%), and NNge/nearest-neighbor with generalized exemplars (92.2%/92%). Decision tree algorithms were recommended for this problem due to their fast execution time and ability to work well with small datasets—critical requirements since classification must happen between pageviews without degrading user experience. The authors noted that while neural network approaches like MultiLayerPerceptron might achieve better accuracy with optimization, their slower execution makes them impractical for real-time web personalization. The 23 interaction events captured (mouse movements, clicks, key presses, focus changes, etc.) proved sufficient to distinguish AT from non-AT users from just one pageview, suggesting that assistive technology users produce detectably different interaction patterns.
Relevance
This research presents a practical approach to automatic accessibility personalization that avoids the common problem of requiring users to explicitly configure their needs—a process that violates Universal Design principles and creates barriers particularly for users who may not understand technical accessibility options. The ability to detect assistive technology use from a single pageview's interaction data could enable immediate content modality adaptation without user effort. For web developers and accessibility practitioners, this suggests that monitoring interaction patterns could complement WCAG compliance by dynamically optimizing the user experience. However, the study has notable limitations: the dataset is small (261 sessions, only 31 AT users), the binary AT/non-AT classification oversimplifies the diversity of assistive technologies and user needs, and the resampling technique used to address class imbalance may inflate accuracy metrics. The approach also raises privacy concerns about tracking and profiling user behavior. Future work planned to extend the method to distinguish between different types of assistive technology for more granular personalization.
Tags: universal design · machine learning · personalization · assistive technology · user interaction · adaptive interfaces · content modality · data mining
Standards referenced: WCAG · Section 508