Web Accessibility Testing for Singapore Government e-Services
Zui Young Lim, Jia Min Chua, Kaiting Yang, Wei Shin Tan, Yinn Chai · 2020 · Proceedings of the 17th International Web for All Conference (W4A) · doi:10.1145/3371300.3383353
Summary
This paper from Singapore's Government Technology Agency presents a customized automated accessibility testing tool designed specifically for government e-service developers. The authors identified two core problems: Singapore's low accessibility maturity despite being a signatory to the UN Convention on the Rights of Persons with Disabilities (scoring just 32 out of 100 on the Digital Accessibility Rights Evaluation Index), and the poor developer usability of existing accessibility testing tools which produce overwhelming, unprioritized reports. To ground their tool in real user needs, the team conducted contextual inquiry sessions with eight persons with disabilities in Singapore (three blind, two low-vision, two with mild left hemiparesis, and one deafblind), observing them as they attempted to use government e-services. The tool builds on the open-source Axe accessibility engine by Deque Systems, adding a customized prioritization layer that weights standard Axe severity scores with a local context factor derived from the contextual inquiry findings. The entire package is designed for deployment on continuous integration/continuous delivery (CI/CD) platforms — specifically Singapore's Secure Hybrid Integration Platform (SHIP) — so that accessibility testing becomes a routine part of the development pipeline rather than a post-hoc compliance check.
Key findings
The contextual inquiry revealed that the most critical accessibility barriers in Singapore government e-services were missing labels on form elements (preventing users from completing registration and authentication), lack of notification of page changes (users unaware of page expiry or form state changes), and inaccessible CAPTCHA (completely blocking access to registered services). Medium-priority issues included unclear heading hierarchy, confusing navigation structures, and timeout problems with one-time passwords — a particularly acute issue given the ubiquity of two-factor authentication on government services. The authors' prioritization formula (New Priority Score = Axe Severity × (1 + Local Factor)) creates scores ranging from 1 to 20, producing finer-grained distinctions than the standard four-level severity scale. For example, a "serious" Axe issue flagged as high-priority locally (like missing labels) scores 10, ranking above a "critical" Axe issue not indicated in local research (score 8). The tool generates streamlined reports limited to severe violations, directly addressing the known problem that overwhelming accessibility reports discourage developer action.
Relevance
This paper offers a practical model for any government or large organization seeking to embed accessibility testing into development workflows. The approach of combining industry-standard automated testing (Axe) with locally-derived prioritization is replicable and addresses a real gap — most accessibility tools apply universal severity ratings that may not reflect the actual impact on users in specific contexts. The contextual inquiry methodology, while small-scale (N=8), demonstrates how even limited user research can meaningfully improve tool design. The CI/CD integration approach is particularly valuable, shifting accessibility from a compliance gatekeeping activity to a developer-owned quality practice. The authors honestly acknowledge limitations: automated testing cannot catch all issues, the prioritization is a first approximation, and the user sample doesn't cover all disability types. Future work aims to add developer-friendly reporting with user quotes, suggested fixes, and recommendations for manual testing where automation falls short.
Tags: automated testing · web accessibility · government services · accessibility evaluation · CI/CD · contextual inquiry · agile development
Standards referenced: WCAG · Section 508 · CRPD