Designing Accessible, Explainable AI (XAI) Experiences
Christine T. Wolf, Kathryn E. Ringland · 2020 · SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386302
Summary
This paper examines the intersection of explainable AI (XAI) and accessibility, identifying two primary concerns that the XAI field has largely overlooked. The first concern is accessibility at the XAI interface itself. While XAI techniques like visual explanations have made progress in rendering AI models comprehensible — for example, generating text descriptions alongside highlighted image regions to explain why a classifier assigned a particular label — these explanations often rely on visual modalities that are inaccessible to blind and low-vision users. A blind ML developer cannot see the blue bounding box highlighting which pixel clusters influenced a classification decision, creating a disconnect between the textual explanation and the visual evidence. The hybrid image-plus-text format that makes XAI explanations effective for sighted users creates accessibility barriers for the very populations who may most need to understand and trust AI systems. The second concern is tailoring explanations to users' diverse and dynamically changing explainability needs. The authors illustrate this through two case studies: AI-assisted aging-in-place monitoring and digital mental health support for depression. In the aging-in-place scenario, multiple stakeholders — the older adult, their adult children, healthcare providers, care facility administrators — all interact with the same AI monitoring system but have vastly different technical aptitudes, attention levels, and information needs that change over time as the older person's cognitive abilities shift.
Key findings
The aging-in-place case reveals that explainability needs are not just diverse across users but dynamic within individual users over time. A caregiver may develop experiential competence and need deeper explanations, while the monitored older adult may need progressively simpler ones as cognitive abilities decline — requiring the XAI system to adapt in both directions simultaneously. The digital mental health case introduces an even more fundamental challenge: statistical models are premised on historical data predicting future trajectories, but mental health conditions like depression are situationally emergent rather than linearly progressive. A model trained on historical mood data may not capture that "wasn't good" today does not mean tomorrow will be worse. This creates a misalignment between the model's underlying logic and the lived experience of the condition it models, raising the question of how to explain such misalignments to users. The therapeutic context also complicates when and how to provide explanations — therapists strategically withhold information at certain moments for therapeutic benefit, a nuanced dynamic that AI chatbots providing mental health support would need to navigate. The authors argue that XAI cannot be prescriptively programmed but requires situated action, interaction, and mutual meaning-making.
Relevance
This paper makes a crucial contribution by highlighting that the growing field of XAI has not adequately considered accessibility — a significant oversight given that people with disabilities are among the most affected by AI decisions and most reliant on AI-powered assistive tools. For accessibility practitioners, the key takeaway is twofold: first, the interfaces through which AI explains itself must be accessible (multimodal, screen-reader compatible, adaptable to cognitive needs); and second, explanations must be tailored not just to different users but to the same user at different times and in different contexts. The aging-in-place and mental health cases demonstrate that XAI accessibility extends far beyond interface design into fundamental questions about what gets explained, to whom, when, and how. For organizations deploying AI systems that affect people with disabilities — whether assistive technologies, healthcare monitoring, or automated decision-making — this paper provides a framework for thinking about explainability as an accessibility requirement, not an optional feature.
Tags: explainable AI · accessibility · aging in place · mental health · digital mental health · XAI · cognitive accessibility · inclusive design
Standards referenced: GDPR