Artificial Intelligence and the Dignity of Risk
Emily Shea Tanis, Clayton Lewis · 2020 · SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386303
Summary
This paper examines the dual risks and opportunities that AI-based systems pose for people with cognitive disabilities, framed around the concept of the "dignity of risk" — the right to make self-directed choices about tradeoffs between risks and benefits, including the freedom to accept risk in return for learning and growth. The authors, both from the Coleman Institute for Cognitive Disabilities, identify several threat vectors. Automated administrative systems like job applicant screeners are trained on aggregate data patterns and may disadvantage people whose patterns of strengths, weaknesses, and life circumstances differ from the norm, leading to discrimination in housing, employment, financial services, and even romantic matching. Data gathering itself poses heightened privacy risks because people with cognitive disabilities often have uncommon combinations of attributes that make them easier to re-identify even from anonymized datasets, and the consequences of privacy violations are more serious because this population is disproportionately targeted for abuse and exploitation — for example, older Americans with cognitive disabilities being targeted by fundraising scams. At the same time, AI offers significant potential benefits: speaker-dependent speech recognition that adapts to atypical speech, travel companion applications that help with situational awareness, and flexible personal supports that could reason about a user's overall situation in ways current non-AI systems cannot.
Key findings
The paper identifies a troubling pattern: many people with cognitive disabilities and their caregivers are responding to AI risks by withdrawing entirely from participation — refusing to use online job applications, declining to share data, or avoiding AI-enabled services. While understandable, this withdrawal strategy comes at the cost of foregone benefits and may actually be reinforced by well-intentioned privacy protections like the GDPR that restrict data uses that could be beneficial. The authors propose several approaches to balance risks and benefits. Jutta Treviranus's "lawnmower of justice" concept suggests rebalancing modeling processes to give greater weight to unusual cases and less to common ones. A more practical suggestion is having classification systems output outlier cases alongside "good" candidates rather than simply filtering them out. On transparency, the authors argue that meaningful communication of data risks is difficult even for technically sophisticated people and propose a new consumer protection institution — modeled on LEED building certification — that would audit and certify companies' data practices, providing trustworthy evaluation that consumers could rely on without mastering technical details themselves.
Relevance
This paper makes a vital contribution by centering the agency and autonomy of people with cognitive disabilities in conversations about AI fairness — a group often spoken about rather than spoken with in technology policy discussions. The dignity of risk concept, drawn from disability rights philosophy (Perske, 1972), challenges paternalistic approaches where caregivers or policymakers make risk-avoidance decisions on behalf of disabled people. For accessibility practitioners, this reframes the question from "how do we protect people with cognitive disabilities from AI" to "how do we scaffold informed, self-directed engagement with AI." The consumer protection certification model is a practical proposal that could benefit all consumers while particularly serving those with cognitive disabilities who face the highest stakes. The paper complements White's and Findlater et al.'s analyses in the same SIGACCESS issue, together forming a comprehensive examination of AI fairness and disability from philosophical, technical, and policy perspectives.
Tags: AI fairness · cognitive disability · dignity of risk · privacy · algorithmic bias · self-determination · consumer protection · data ethics
Standards referenced: CRPD · GDPR