A Challenge to Web Accessibility Metrics and Guidelines: Putting People and Processes First
Martyn Cooper, David Sloan, Brian Kelly, Sarah Lewthwaite · 2012 · Proceedings of the International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/2207016.2207028
Summary
This influential paper argues that web accessibility is not an intrinsic property of a digital resource but a relational quality determined by the interaction between users, resources, and contexts — shaped by political, social, cultural, and technical factors. The authors challenge the dominant approach of treating WCAG conformance as the primary measure of accessibility, arguing that this technical focus on the artefact (the web page) divorces accessibility from the actual user experience of disabled people. They position this critique within the "Third Wave" of HCI, where system quality extends beyond task completion to encompass subjective, context-dependent user experience. The paper advocates for process-oriented standards like BS 8878 (Web Accessibility Code of Practice), which provides a 16-step model spanning the entire web product lifecycle from conception through post-launch maintenance. Notably, WCAG is referenced in only 4 of the 16 steps, demonstrating that technical guidelines are just one part of a much broader accessibility strategy. The authors also draw on Disability Studies perspectives to argue that conformance-focused approaches can be counter-productive and paternalistic, potentially undermining the importance of cultural, political, and social factors in disabled people's experiences of technology.
Key findings
The paper presents several user-focused alternatives to pure guideline conformance. The Open University's proposed "Social Software Approach to Accessibility" would allow disabled users to rate resources on a 3-point accessibility scale (accessible, problematic, inaccessible), with ratings aggregated by access profile to help similar users find accessible resources. A learning analytics case study from the Open University (200,000+ students, 12,000+ declaring disabilities) analysed completion and pass rates across 164 undergraduate modules. The difference between disabled and non-disabled student outcomes ranged from -33% to +29%, with disabled students underperforming on 67% of modules. The authors argue this data can identify where accessibility barriers are most impactful, then web accessibility metrics can be applied in a targeted way to investigate whether technical accessibility is a contributing factor — rather than applying metrics uniformly across all resources. This pragmatic approach addresses the reality that organisations have insufficient resources to evaluate all web assets, so analytics can focus effort where it has the most impact. The authors emphasise that poor outcomes for disabled students could stem from WCAG failures, usability issues, insufficient training, or poor inclusive learning design — the analytics approach initiates investigation rather than assuming the cause.
Relevance
This paper presents a foundational critique that remains highly relevant: WCAG conformance is necessary but insufficient for genuine accessibility. For organisations, the key insight is that accessibility should be embedded as a process across the entire product lifecycle (as BS 8878 articulates) rather than treated as a testing checkpoint near the end. The relational definition of accessibility — as a property of the interaction between user, resource, and context rather than a property of the resource alone — has significant practical implications: the same resource may be accessible on one platform but inaccessible on another, making universal conformance statements meaningless without qualification. The learning analytics approach offers a powerful model for any organisation with user data: rather than auditing everything equally, use outcome data to identify where accessibility barriers cause the most real-world harm, then investigate and remediate those high-impact areas first. This pragmatic, evidence-driven approach to prioritising accessibility work is especially relevant for organisations with large content estates and limited accessibility expertise.
Tags: accessibility theory · accessibility policy · WCAG compliance · user experience · disability studies · BS 8878 · learning analytics · inclusive design · accessibility metrics · organisational accessibility
Standards referenced: WCAG 2.0 · BS 8878 · ATAG · UAAG