Exploring an Ambiguous Technique for Eyes-Free Mobile Text Entry
Dylan Gaines · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018) · doi:10.1145/3234695.3240991
Summary
This student research competition paper from Michigan Technological University presents Tap123, an ambiguous eyes-free keyboard for mobile text entry that does not require users to tap specific keys. The system is motivated by the limitations of existing blind text entry methods: VoiceOver-based typing is location-dependent and requires multiple touch events per character; Braille-based methods (BrailleType, Perkinput, TypeInBraille, BrailleTouch) require knowing Braille, which fewer than 10% of legally blind Americans can read. Tap123 builds on the theory of ambiguous input by mapping finger count (1-3 fingers) and screen side (left or right) to QWERTY keyboard rows. For example, one finger on the left side maps to the letters Q, W, E, R, T (all possible letters on the left side of the top QWERTY row). The user taps a sequence corresponding to each letter in a word, then swipes right to send the sequence to the VelociTap statistical decoder, which returns an N-Best list of the 6 most likely words based on the ambiguous taps and language context. Users navigate the N-Best list by swiping up/down. Left swipes backspace individual taps or delete entire words. All feedback is audio-based: taps speak corresponding letters, recognition speaks the selected word, and a long press reads the full entry field.
Key findings
Four sighted participants (with the device hidden from view to simulate eyes-free conditions) completed 8 sessions of approximately 40 minutes each, progressing from single words to unrestricted phrases. Starting entry rate was 4.32 WPM in Session 1, reaching 19.09 WPM across the final 3 sessions — a significant improvement that is highly competitive with existing methods. For comparison, VoiceOver typing achieves 2.11 WPM and BrailleType 1.45 WPM, making Tap123 roughly 9x and 13x faster respectively. The final speed is slightly over half of Perkinput's 38.02 WPM (which requires Braille knowledge and two-handed entry). Character Error Rate started at 4.05% and improved to 2.08% in the final sessions — well below VoiceOver's 14.12% and BrailleType's 8.91%. Backspaces Per Tap (a measure of initial tap accuracy) dropped dramatically from 0.199 in Session 1 to a stable 0.068 for the remaining sessions, indicating rapid learning of the finger-count-to-row mapping. The learning curve showed most participants struggled initially but improved substantially, with consistent performance by session 3-4.
Relevance
Tap123 demonstrates that ambiguous input — where each tap maps to multiple possible letters resolved by a statistical decoder — is a viable and potentially superior approach to eyes-free mobile text entry. The key advantage over VoiceOver is speed (9x faster) without requiring visual feedback or precise key targeting; the advantage over Braille methods is not requiring specialized literacy that 90% of blind Americans lack. For accessibility practitioners and keyboard designers, the important insight is that leveraging QWERTY familiarity (most users already know approximate key positions) combined with statistical language models can compensate for the imprecision inherent in eyes-free input. The current limitation is that all participants were sighted, and testing with blind and low vision users is planned as future work — along with improvements to word deletion, keyboard closing gestures, and probabilistic decoding for fuzzy matching. The system is being reviewed by three blind/low-vision accessibility experts.
Tags: blindness · low vision · text entry · mobile accessibility · eyes-free interaction · touchscreen accessibility · input methods · ambiguous keyboard