AI Over-Reliance

The tendency of users to accept AI system outputs — recommendations, classifications, or content — without sufficient critical evaluation, even when those outputs are wrong or biased. Over-reliance is a well-documented AI safety concern and is especially consequential in accessibility contexts: users who trust AI-generated alt text, image descriptions, or bias assessments without verification can propagate inaccurate or stereotypical content. Mitigations studied in the literature include cognitive forcing functions, explanations, and user education.