Auditory feedback to compensate audible instructions to support people with visual impairment
Gabriele Galimberti · 2021 · ASSETS '21: The 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3476477
Summary
This doctoral consortium extended abstract addresses a practical challenge faced by people with visual impairments who rely on auditory feedback from mobile devices: background noise in real-world mobility contexts can mask device audio output, degrading the information it contains and making navigation instructions unreliable or incomprehensible. The research is situated at the University of Milano-Bicocca and builds on the author's prior work on sonification-based navigation instructions for people with visual impairments. Galimberti investigates three audio compensation techniques designed to make device feedback audible in noisy environments without being excessively intrusive or masking important environmental sounds. The three techniques are: (1) speech slowing, which reduces the speed of text-to-speech output to help users follow verbal instructions and prevent impulsive ambient sounds from masking speech; (2) volume increase, which raises audio feedback volume to match or exceed the peak volume of environmental noise; and (3) adaptive equalization, which selectively boosts the gain of specific frequencies that are being masked by background noise, increasing the signal-to-noise ratio by up to 10 dB in affected frequency bands. The author also draws on prior research exploring different sonification methods for turn-by-turn navigation — including impulse sounds, musical sequences, and ping sounds — finding that musical sequence sonification combined with a supporting ping sound significantly reduced rotation errors for people with visual impairments.
Key findings
A preliminary evaluation with 15 sighted participants tested the three compensation techniques against two baselines (uncompensated speech with background noise and speech without background noise). Participants listened to text-to-speech phrases played alongside mobility noise soundscapes through bone conduction headphones and reported what they understood. Results showed that volume increase and adaptive equalization both improved the percentage of correctly understood words compared to uncompensated audio in noisy conditions, while speech slowing provided only partial improvement. Critically, none of the compensation techniques significantly increased the perceived intrusiveness of the audio feedback relative to real-world sounds — an important consideration because overly intrusive audio could mask environmental cues like car horns or approaching vehicles that are safety-critical for people with visual impairments navigating urban environments. However, background noise offsets were found to be more intrusive than uncompensated cases, presenting a design challenge: compensations must improve audibility without worsening intrusiveness. The research is ongoing, with plans to extend these compensation techniques to non-verbal sonification feedback and to test with participants who have visual impairments.
Relevance
This research addresses a gap between laboratory-tested navigation aids and their real-world usability for people with visual impairments. Many assistive navigation tools rely on audio output but are designed and tested in quiet environments, overlooking the reality that urban mobility involves significant background noise from traffic, pedestrians, and construction. The compensation techniques explored here — particularly adaptive equalization — could be integrated into existing navigation apps and screen readers to dynamically adjust audio output based on ambient noise levels. The use of bone conduction headphones in the study design is also noteworthy, as these allow users to hear both device audio and environmental sounds simultaneously, which is a safety consideration often overlooked in assistive technology design. While the work is preliminary and has not yet been tested with the target user group, it establishes a foundation for adaptive audio systems that could make mobile assistive technologies more reliable in the diverse acoustic environments where they are actually needed.
Tags: visual impairment · auditory feedback · sonification · mobile accessibility · navigation · noise compensation · text-to-speech