PLUMB: Displaying Graphs to the Blind Using an Active Auditory Interface
Robert F. Cohen, Rui Yu, Arthur Meacham, Joelle Skaff · 2005 · Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '05) · doi:10.1145/1090785.1090820
Summary
This short paper presents PLUMB (exPLoring graphs at UMB), a system developed at UMass Boston that enables blind users to explore relational graphs through an active auditory interface on a tablet PC. The system addresses a fundamental barrier in computer science education: many concepts are taught through diagrams showing nodes and edges, which are inherently visual. PLUMB displays a drawn graph on the tablet screen and uses a combination of audio cues — MIDI tones, synthesized speech, and pre-recorded sound files — to communicate the graph structure as the user moves a stylus across the surface. The system uses pitch variation to indicate progress along edges, loudness variation to represent distance from nodes or edges, and speech to provide textual descriptions of graph elements. Significant events like entering a node or moving outside the graph area trigger pre-recorded sound notifications. Graphs are represented internally as XML documents with corresponding XML audio configuration files, allowing flexible mapping between visual and auditory representations.
Key findings
The PLUMB system demonstrates an active exploration approach to non-visual graph access, where users physically move a pen input across the tablet surface to discover graph structure, as opposed to passive systems that present an entire representation at once. The authors established four key design guidelines: audio cues must be consistent (same action produces same feedback), real-time (sounds play immediately on contact and stop when the user moves away), informative (conveying both current location and navigation direction), and user-friendly. The system uses HSB colour model properties to map audio characteristics — hue corresponds to pitch changes along edges, while saturation corresponds to loudness changes representing distance. Beyond computer science education, the authors identify applications in map reading, street navigation, fire escape route learning, and enabling blind users to create visual information rather than just consume it. The paper positions the tablet PC as a particularly suitable platform due to its pen-based input (conducive to exploring complex structures), compact size, and widespread multimedia capability.
Relevance
PLUMB represents an early and creative approach to making relational and spatial information accessible through active auditory exploration — a paradigm that remains relevant as data visualisation becomes increasingly important across education and professional contexts. The distinction between active and passive exploration that the authors draw is an important design consideration: active exploration gives users agency and spatial awareness, while passive systems can overwhelm with information presented all at once. For accessibility practitioners working on STEM education or data visualisation, the design principles outlined here — consistency, immediacy, informativeness, and usability — provide a useful framework. The work also anticipates the growing need for non-visual access to graphs, charts, and diagrams that has become even more pressing with the proliferation of data-driven content on the web.
Tags: sonification · graph accessibility · blindness · auditory interface · computer science education · non-visual interaction · data visualization · STEM accessibility