(Natural Language) Interaction with Graphical Representations of Statistical Data
Leo Ferres, Petro Verkhogliad, Louis Boucher · 2007 · Proceedings of the 2007 International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1243441.1243446
Summary
This paper from Carleton University and Statistics Canada presents iGraph-Lite, a system that makes statistical graphs published in "The Daily" (Statistics Canada's main dissemination publication) accessible to blind and visually impaired users through natural language descriptions and interactive exploration. The system bridges two previously separate research traditions: natural language generation (NLG) systems that produce static text summaries of graphical data but offer little interactivity, and sonification tools that make graphs interactive through sound but do not leverage the richness of natural language. iGraph-Lite combines both: users can receive comprehensive verbal descriptions of graphs and interactively navigate through data points using keyboard commands, with the system generating contextual natural language responses via text-to-speech. The technical pipeline starts when a statistician creates a graph in Excel: an auxiliary XML file is automatically generated containing titles, scales, values, colors, and styles. Graph analysis classes then process this XML into a richer representation that includes computed properties like maximum/minimum values, general trend shape, and contextually interpreted axis labels (e.g., converting "01", "02" into "January", "February" when the axis title reads "Month").
Key findings
The system provides several interaction commands demonstrated through a sample session with an operating profits graph. The 'd' (describe) command generates a comprehensive overview: graph type, title, axis descriptions using spatial terms ("vertical" and "horizontal" rather than "x" and "y"), data range, general trend, and key data points. Arrow keys allow point-by-point navigation with contextual descriptions of changes between consecutive points ("There is a small increase between Q1 and Q2 from 25.9 to 26.7"). The 's' (skip) command enables jumping over multiple data points — essential for datasets with thousands of points where sequential exploration would be impractical. The 'w' (where) command helps users maintain orientation within the graph. The design decisions were informed by studies with sighted, blind, and visually impaired participants. The authors acknowledge limitations including the system's inability to infer relationships between axis values (failing to connect quarter labels with year numbers, producing awkward descriptions like "between Q2 and Q1" instead of "between Q2 1996 and Q1 1997").
Relevance
This paper addresses a critical and still-underserved area of web accessibility: making data visualizations accessible beyond simple alt text. Static text descriptions of graphs, while necessary, cannot replicate the exploratory understanding that sighted users gain from visual patterns — iGraph-Lite's interactive navigation allows blind users to explore data at varying levels of detail, much as sighted users can zoom in and scan across a graph. The government context is significant: national statistics agencies publish enormous quantities of graphical data that citizens need to access, and this work demonstrates a practical pipeline for making that data accessible at scale by leveraging the XML already generated during chart creation. The natural language choices — using spatial terms, describing trends qualitatively ("small increase"), and providing orientation commands — reflect careful consideration of how non-visual users build mental models of data. For modern practitioners, the paper anticipates current approaches to chart accessibility including WCAG requirements for complex image descriptions and newer tools that generate natural language summaries of data visualizations. The skip command for large datasets is a particularly practical innovation that addresses the scalability problem in non-visual data exploration.
Tags: data visualization · blind users · visual impairment · natural language processing · web accessibility · accessible graphics · text-to-speech · data accessibility · government