Speed-Dial: A Surrogate Mouse for Non-Visual Web Browsing
Syed Masum Billah, Vikas Ashok, Donald E. Porter, IV Ramakrishnan · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17) · doi:10.1145/3132525.3132531
Summary
This paper presents Speed-Dial, a system that uses an off-the-shelf Microsoft Surface Dial as a surrogate mouse for non-visual web browsing. The core problem it addresses is the fundamental mismatch between how sighted and blind users interact with the web: sighted users can visually scan a page and point-and-click with a mouse almost instantaneously, while blind screen reader users must navigate serially through content using keyboard shortcuts, building a mental model of the page as they go. This serial process causes significant content and cognitive overload. Speed-Dial bridges this gap by interfacing the physical Dial with a semantic model of web pages. The system automatically constructs a Semantic Tree — a cleaned-up, semantics-based abstraction of the HTML DOM — that groups related page elements into meaningful entities (forms, search results, menus, filters, calendars). Users navigate this tree using simple Dial gestures: rotate left/right to move between siblings, press to go to a child entity, double-press to go to the parent, and triple-press to search and jump to an entity. The system provides adaptive haptic feedback — a tick for each rotation and a continuous buzz when reaching entity boundaries. A Web-Entity Ontology of over 100 entity descriptions, built from analysis of 200 popular websites, drives the semantic extraction and provides custom Dial interface specifications for complex widgets like calendars, autocomplete boxes, drop-downs, and data grids that are notoriously difficult for screen reader users.
Key findings
A user study with 12 blind participants (ages 25-65) on travel booking websites showed dramatic improvements across all measures. For a form-filling task (T1), Speed-Dial reduced mean completion time by 73% compared to screen reader only (60.4s vs. 224.2s) and by 5.7% compared to screen reader with speech commands (66.2s). For a search-results navigation task (T2), completion time was reduced by 80.2% vs. screen reader only (76.9s vs. 389.6s) and 62.8% vs. speech (207.0s). Keyboard shortcut usage dropped by 97.1% and 90.6% for T1 and T2 respectively when using Speed-Dial compared to screen reader alone. The mean SUS usability score for Speed-Dial (85.0) was 18.3% higher than screen reader only (69.3) and 4.6% higher than speech (81.04), with the differences between screen reader and both Speed-Dial and speech being statistically significant (p<0.01). All participants unanimously rated knowing entity boundaries as maximally important (5.0/5.0, SD=0.0), and rated the ease of rotating the Dial vs. pressing keyboard shortcuts at 4.5/5.0. Participants reported feeling more confident exploring content with the Dial because Speed-Dial always confines navigation within entity boundaries, preventing the disorienting experience of unknowingly crossing into unrelated page sections. A key advantage was uniformity: Speed-Dial provides the same interaction experience regardless of which screen reader is used, eliminating the need to learn different shortcut vocabularies.
Relevance
This research challenges the assumption that keyboard-based screen reader navigation is the only viable paradigm for non-visual web browsing. For accessibility practitioners and developers, the key insight is that the gap between sighted and blind web experiences is not just about making content technically accessible — it is about the interaction paradigm itself. The semantic model approach demonstrates that understanding the meaning and structure of web content (not just its HTML syntax) is critical for efficient non-visual access. The custom interfaces for complex widgets like calendars, autocomplete boxes, and data grids address some of the most persistent pain points in screen reader usability. The study also highlights how haptic feedback can serve as an orientation mechanism, helping blind users understand where they are in a page without audio. Limitations include the reliance on Internet Explorer (chosen because it was still popular among blind users at the time), the Web-Entity Ontology requiring manual creation of new entity descriptions, and the small study sample confined to travel booking sites.
Tags: screen readers · web accessibility · blindness · haptic technology · input devices · semantic web · assistive technology · non-visual interaction