Evaluating the Accessibility of Rich Internet Applications
Nádia Fernandes, Daniel Costa, Sergio Neves, Carlos Duarte, Luís Carriço · 2012 · Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/2207016.2207019
Summary
This paper addresses a critical gap in automated accessibility evaluation: most tools only assess the initial HTML served to the browser, missing the dynamically generated content that defines modern Rich Internet Applications (RIAs). Web applications using JavaScript and AJAX can change their DOM extensively in response to user interactions — clicks, keypresses, hover events — meaning that what users actually experience may be fundamentally different from what automated evaluators see. The authors present QualWeb Evaluator 3.0, an enhanced version of their accessibility evaluation framework that introduces an Interaction Simulator component to automatically trigger events on interactive elements, detect resulting DOM changes, and evaluate each new page state against WCAG 2.0. The system uses CasperJS (a scripting utility for PhantomJS, a headless WebKit browser) to simulate browser processing and crawl through application states. Crawlers attach to each clickable element with an onclick handler, trigger click actions, and an Observer monitors the DOM tree for changes. When a new state is detected, the modified DOM is sent to QualWeb for evaluation. The system tracks which interactive elements have been activated and which DOM states have already been evaluated to avoid duplication, continuing until all discoverable interactive elements have been triggered.
Key findings
Testing 50 web applications, the study found an average of 5 distinct DOM states per page — meaning 5 different versions of the page content that users could encounter through interaction, none of which would be evaluated by traditional automated tools. Across all states, 28% of evaluation results were passes, 12% were fails, and 60% were warnings (with 43% of warnings coming from CSS technique checks). Compared to their previous static evaluation experiment, they detected 14% fewer passes, 2% fewer fails, and 11% more warnings — indicating that dynamic content introduces additional potential accessibility problems that static analysis misses. When new DOM states were triggered, warnings increased by an average of 10% and fails increased by 4% per page compared to the initial state. The framework implements 26 WCAG 2.0 techniques (20 HTML, 6 CSS) and outputs results in EARL (Evaluation and Report Language) format. The experiment imposed constraints: only 5 redirections followed per page, limited to click events (not keyboard or hover), and incomplete WCAG 2.0 coverage.
Relevance
This work identified a problem that has only grown more pressing as single-page applications (SPAs) and JavaScript-heavy frameworks have come to dominate web development. In 2012, the authors demonstrated that treating web pages as static documents for accessibility evaluation was already inadequate — a finding that is dramatically more true today with React, Angular, and Vue applications where nearly all content is dynamically rendered. The average of 5 states per page, each potentially introducing new accessibility failures, makes a compelling case that automated evaluation must account for application behavior, not just initial markup. Modern tools like axe-core and Lighthouse have since incorporated some dynamic evaluation capabilities, but comprehensive state-based crawling of complex SPAs remains an unsolved challenge. The QualWeb approach of systematically triggering events and monitoring DOM changes laid important groundwork for how the field thinks about evaluating dynamic web content accessibility.
Tags: automated testing · web accessibility · rich internet applications · AJAX · dynamic content · DOM manipulation · accessibility evaluation
Standards referenced: WCAG 2.0 · WAI-ARIA · EARL