Framework for Experiential Transcoding of Web Pages with Scanpath Trend Analysis
Idil Ece Trabzon, Furkan Yagiz, Elmas Eda Karadavut, Mahmoud Elhewahey, Sukru Eraslan, Yeliz Yesilada, Simon Harper · 2022 · Proceedings of the 19th International Web for All Conference (W4A) · doi:10.1145/3493612.3520450
Summary
This paper presents EXTRA, an automated framework for experiential transcoding of web pages — a technique that restructures web content based on how sighted users actually interact with pages, rather than relying solely on source code analysis. The core problem the authors address is that web pages are designed around visual elements like menus, sidebars, and highlighted content that help sighted users navigate but become distracting clutter for screen reader users who must listen through everything sequentially. Traditional transcoding approaches reformat pages using heuristics applied to the source code, but they do not account for real user behaviour. The EXTRA framework takes a different approach by combining two key technologies: Vision-based Page Segmentation (VIPS), which breaks a web page into its visual components using both source code and visual layout, and Scanpath Trend Analysis (STA), which analyses eye-tracking data from sighted users to identify which page elements are most commonly visited and in what order. The framework uses this information to remove non-trending elements and reorder the remaining content to match natural viewing patterns. Implemented as a Chrome browser extension, EXTRA offers three transcoding modes: heuristic-based transcoding that removes common clutter like logos and ads, STA-based transcoding that restructures pages around trending elements, and a hybrid mode that provides links to removed elements rather than discarding them entirely. The framework has an open architecture allowing components to be swapped — for instance, machine learning could replace the eye-tracking component to predict trending paths without requiring collected gaze data.
Key findings
Preliminary evaluation using eye-tracking data from 10 participants across four websites showed promising results. Average access times to page elements decreased substantially on three of four tested sites: Apple dropped from 13.19 to 7.87 seconds, AVG from 31.04 to 12.79 seconds, and Yahoo from 38.82 to 12.47 seconds. Only Babylon showed a marginal increase from 35.05 to 35.89 seconds. The framework also improved perceivability, operability, understandability, and robustness scores as measured by the aDesigner accessibility simulator. Yahoo saw the most dramatic improvement, with perceivability jumping from 0 to 91 and operability from 63 to 100. AVG perceivability improved from 0 to 65 and robustness from 70 to 90. Average scores across all four metrics improved for every tested page. The authors note a slight decrease in operability and robustness for the Apple page, but attribute this to the original page already having very high scores.
Relevance
This research offers an innovative approach to a persistent accessibility challenge: screen reader users must navigate web pages linearly through content that was designed for visual scanning. By grounding transcoding decisions in actual user behaviour rather than code-level heuristics, the EXTRA framework could deliver more meaningful content prioritisation. The concept of experiential transcoding is particularly relevant as organisations struggle to retrofit accessibility into visually complex websites. The framework also has applications beyond visual impairment — it could benefit users with situational disabilities or those on limited bandwidth connections. Key limitations include the dependency on pre-collected eye-tracking data for each page (though the authors are exploring machine learning alternatives) and the small scale of the preliminary evaluation. The open architecture is a strength, allowing the approach to evolve as prediction methods improve.
Tags: screen readers · web transcoding · eye tracking · web page segmentation · content reordering · browser extension · visual impairment