Tactile Captions

An enhanced captioning approach that supplements traditional text-based captions with vibrotactile feedback, allowing deaf and hard of hearing viewers to feel non-speech sounds (such as phone rings, doorbells, footsteps, or objects falling) through a wrist-worn or body-worn vibration device while watching captioned video. The vibration patterns replicate the temporal characteristics of the original sound — its rhythm, duration, and amplitude envelope — so the viewer can perceive the sound's qualities without reading a text description. Tactile captions address the attention-splitting problem of visual-only captions, where DHH viewers must constantly shift gaze between reading caption text and watching the visual action. By delivering non-speech information through touch rather than text, tactile captions free the viewer's eyes to watch the scene. Research has shown that tactile captions significantly improve recall and identification of non-speech events compared to standard captions alone.