A Dataset of Alt Texts from HCI Publications: Analyses and Uses Towards Producing More Descriptive Alt Texts of Data Visualizations in Scientific Papers
Sanjana Shivani Chintalapati, Jonathan Bragg, Lucy Lu Wang · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22) · doi:10.1145/3517428.3544796
Summary
This paper examines the quality and availability of alt text for data visualizations in scientific publications, focusing on papers from the HCI and accessibility communities (CHI and ASSETS conferences, 2010-2020). The authors processed over 25,000 paper PDFs and found that only 4.6% (897 papers) contained at least one piece of valid alt text, yielding a dataset of 3,386 author-written alt texts. To evaluate the quality of these descriptions, the researchers applied a four-level semantic content framework developed by Lundgard and Satyanarayan. Level 1 covers basic construction details like chart type and axis labels; Level 2 addresses statistical properties such as extrema and correlations; Level 3 captures complex trends and patterns; and Level 4 includes domain-specific insights and societal context. The authors focused specifically on alt text for graphs, charts, and plots — the figure types most important to blind and low vision readers for understanding research results. Two trained annotators labeled 547 figure alt texts (2,127 sentences) according to these semantic levels, achieving strong inter-annotator agreement of 87.6%. The study also explored how alt text practices have changed over time, noting a gradual increase in alt text availability after CHI began requiring it in 2014, though overall rates remain below 15% of papers in their sample.
Key findings
The analysis revealed significant gaps in the semantic content of author-written alt text. While the vast majority of alt texts contained Level 1 information (chart type, axes, labels), far fewer included higher-level content that blind and low vision users find most valuable. Only about 50% of alt texts mentioned extrema or outliers (Level 2), and just 31% described major trends or comparisons conveyed by the graph (Level 3). Most alt texts contained only one or two semantic levels of information, despite research showing that BLV users benefit most from Levels 1-3. The distribution of maximum semantic levels was roughly even across Levels 1, 2, and 3, meaning a third of alt texts never progressed beyond basic chart identification. Over time, the proportion of alt text containing Level 1 information remained consistently high, but there was no significant improvement in the inclusion of Level 2 or Level 3 content. The relationship between alt text length and semantic richness showed that longer alt texts tended to contain more levels, though length alone was not a reliable predictor of quality. The authors trained classification models (Random Forest, BERT, SciBERT) to automatically detect semantic levels in alt text sentences, achieving around 91% accuracy with BERT-based models — suggesting that automated tools could feasibly provide real-time feedback to authors about missing semantic content.
Relevance
This study provides essential empirical evidence about the state of figure accessibility in academic publishing — a domain where inaccessible data visualizations exclude blind and low vision researchers and readers from engaging with scientific findings. The Lundgard and Satyanarayan semantic level framework offers a practical, actionable rubric that any author can use to evaluate and improve their alt text for charts and graphs. The key takeaway for practitioners is that effective alt text for data visualizations must go beyond describing what a chart looks like (type, axes, colors) to communicate what the data means — trends, comparisons, outliers, and context. The released dataset of 3,386 real-world alt texts and 547 annotated examples provides a valuable resource for training machine learning models and developing authoring tools. For publishers and conference organizers, the findings underscore that simply requiring alt text is insufficient; guidelines must also address content quality and encourage inclusion of statistical and trend information. The classifier models demonstrate that automated feedback tools are technically feasible and could be integrated into publishing workflows to prompt authors toward more complete descriptions.
Tags: alt text · data visualization · scientific documents · blind and low vision · document accessibility · machine learning · natural language processing · dataset
Standards referenced: WCAG 2.0