Experiential Transcoding: An EyeTracking Approach
Yeliz Yesilada, Simon Harper, Sukru Eraslan · 2013 · Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/2461121.2461134
Summary
This paper introduces "experiential transcoding" — a user-centred approach to web page transformation that uses eye-tracking data from sighted users to understand how people actually experience web pages, then uses those patterns to guide transcoding for blind and situationally impaired users. Traditional transcoding approaches (adding skip links, generating summaries, reordering content, removing irrelevant sections) operate on content structure alone without understanding how pages are actually used. The authors propose using scanpaths — the sequences of fixations and saccades that occur as people view web pages — mapped to automatically identified visual segments of the page rather than treating pages as undifferentiated images. They extended the VIPS (Vision-based Page Segmentation) algorithm on the Eclipse Accessibility Tools Framework (ACTF) platform to automatically divide pages into semantic visual segments (header, menu, content areas, footer) that link to underlying source code. Eye-tracking data from 10 participants performing a task on a travel website was then expressed as sequences of these visual segments.
Key findings
The authors developed a novel algorithm (eMine) that identifies common scanpaths across multiple users by iteratively finding the two most similar scanpaths (using Levenshtein Distance), computing their Longest Common Subsequence, replacing the pair with the common subsequence, and repeating until a single common scanpath remains. For 10 participants's scanpaths (which varied considerably in length from 9 to 38 steps), the algorithm returned "CDED" — meaning users typically looked at the first content item, then the menu, then the latest news section, then back to the menu. This matched the task (locate latest news and click special offers) and was validated by a transition matrix showing that after viewing segment D (menu), users most likely looked at E (46.16%), and before D they most likely came from C (42.6%). The authors identified four limitations of existing scanpath comparison algorithms: they are reductionist (Dotplots reduced 10 scanpaths to just "D"), they ignore cognitive processes, they treat pages as images without connecting to source code, and they lack systematic noise elimination. The key innovation is relating scanpaths directly to DOM segments rather than pixel coordinates, enabling the common scanpath to drive actual page restructuring.
Relevance
This research proposes an elegant bridge between how sighted users visually experience web pages and how those pages should be restructured for non-visual access. For accessibility practitioners, the core insight is that sighted users' eye-tracking patterns reveal which content is most important and in what order it is naturally consumed — information that can directly inform how a screen reader should present that page or how it should be simplified for mobile. Rather than relying solely on developer-specified semantic structure (which is often poor or absent), experiential transcoding derives importance and reading order from actual user behaviour. The approach also connects to the situational impairment paradigm: mobile users with small screens face similar challenges to blind users in needing linearised, prioritised content. While this 2013 work is preliminary (small sample, single page), the principle of using aggregated user behaviour data to drive accessibility adaptations has become increasingly feasible with modern analytics and AI, and anticipates approaches where web pages could be automatically restructured based on collective usage patterns.
Tags: eye tracking · user experience · transcoding · visual impairment · situational impairment · mobile accessibility · web accessibility · user research · content adaptation