An experimental sound-based hierarchical menu navigation system for visually handicapped use of graphical user interfaces
Arthur I. Karshmer, P. Brawner, G. Reiswig · 1994 · Proceedings of the First Annual ACM Conference on Assistive Technologies (Assets '94) · doi:10.1145/191028.191063
Summary
This paper addresses the accessibility crisis created by the shift from character-based computer interfaces to graphical user interfaces (GUIs) in the early 1990s. The authors note that earlier text-based systems were well-served by assistive technologies like screen readers, braille output devices, and speech synthesizers, but the emergence of GUIs rendered these tools largely ineffective. The visual nature of GUI elements — icons, windows, and particularly hierarchical menu structures — presented fundamental challenges for blind and visually impaired users. The researchers developed an experimental system that uses tonal audio cues to enable blind users to navigate the same hierarchical menu structures used in standard GUIs. Rather than requiring a separate, simplified interface, the approach maps the spatial and structural properties of nested menus to sound, allowing visually impaired users to work within the same computing environment as sighted users. The paper describes the sound-based interface design and the testing methodology developed to evaluate its effectiveness.
Key findings
The system demonstrates that tonal audio cues can effectively represent the hierarchical structure of GUI menus, providing blind users with navigational information that sighted users obtain visually. By encoding menu depth, position, and structure through sound properties such as pitch and tone patterns, users can build a mental model of where they are within a complex menu hierarchy. The approach is designed to be inexpensive and easy to integrate into existing user interfaces, making it a practical solution rather than a purely research prototype. The testing methodology developed alongside the system provides a framework for evaluating the usability of sound-based navigation, comparing how blind users perform with tonal cues versus other navigation methods. The work represents an early attempt to address GUI accessibility through auditory augmentation rather than trying to convert visual interfaces entirely to text.
Relevance
This paper captures a pivotal moment in accessibility history — the transition from text-based to graphical computing that threatened to exclude blind users from mainstream technology. The challenge of making GUIs accessible would dominate assistive technology research throughout the 1990s and ultimately lead to accessibility APIs, screen reader innovations like off-screen models, and eventually standards like WAI-ARIA. The tonal navigation approach described here is an early example of using non-speech audio (sonification) to convey interface structure, a concept that has evolved into modern earcons and auditory icons used in assistive technology. For practitioners, the paper underscores that accessibility must be considered proactively when interface paradigms shift — a lesson equally relevant today as interfaces evolve toward gesture, voice, and spatial computing.
Tags: GUI accessibility · sonification · auditory interface · blind and low vision · menu navigation · screen readers · earcons