Beyond Sight: Empowering Visually Impaired Users with Audible Graphs
Wajdi Aljedaani, Uday Kiran Chimpiri, Durgasantosh Gaddam, Vaseem Ahammed Shaik, Yaswitha Karasala, Marcelo M. Eler · 2024 · Proceedings of the 21st International Web for All Conference (W4A) · doi:10.1145/3677846.3677864
Summary
This technical note presents a tool designed to make data visualizations accessible to people with visual impairments by converting them into audible and textual representations. The tool addresses a significant gap: while data visualization is central to modern information processing and decision-making, charts and graphs remain largely inaccessible to blind and low-vision users who rely on screen readers. The system operates through a deep learning pipeline that takes raster images of charts as input and processes them through several stages. First, a convolutional neural network (a fine-tuned ResNet pretrained on ImageNet) classifies the chart type — supporting pie charts, histograms, bar charts, and line charts. The tool then extracts the underlying data from the chart image, reconstructing the original geometric forms from raster pictures. Text extraction uses deep learning to automatically identify and categorize text elements within charts, distinguishing between legends, axis labels, tick labels, titles, and detecting text regions via OCR. Finally, the extracted data and descriptions are converted to speech using text-to-speech technology, allowing users to listen to descriptions of graphs and images through screen reader assistive technologies.
Key findings
The tool demonstrates a fully automated pipeline from chart image to audible output, requiring no manual intervention or pre-tagged data. By using convolutional neural networks for chart classification and deep learning for text extraction, the system can process raster chart images that were previously completely inaccessible to screen reader users. The approach handles four common chart types (pie, histogram, bar, and line charts) and can present extracted data as raw data tables, multi-level textual summaries, or thumbnails. The tool generates bar graph visualizations from Excel spreadsheet data with spoken descriptions that convey the same information the visual representation provides. However, the paper is a brief technical note (3 pages) and does not include formal user evaluation results with visually impaired participants, nor does it report accuracy metrics for chart classification or data extraction. The system currently does not support stacked bar charts, horizontal bar charts, or chart types beyond the four categories, with unsupported formats classified as "others."
Relevance
Accessible data visualization remains one of the most challenging problems in digital accessibility. Charts and graphs are pervasive in websites, reports, dashboards, and educational materials, yet they are typically presented as flat images with minimal or no alternative text — rendering them invisible to screen reader users. This tool represents a step toward automated solutions that could retrofit accessibility onto existing visual content without requiring manual description by content authors. For accessibility practitioners, the key implication is that AI and deep learning approaches are becoming viable for extracting structured information from visual charts, potentially complementing manual alt text efforts. However, the lack of user testing with visually impaired participants is a notable gap — the tool's actual usability and the quality of its audible output remain unvalidated. Future work should prioritize evaluation with real users and expand chart type support to cover the full range of visualizations encountered in practice.
Tags: data visualization · visual impairments · sonification · screen readers · optical character recognition · deep learning · assistive technology · text-to-speech