Exploring feasibility of wrist gestures for non-visual interactions with wearables
Shirin Feiz, I. V. Ramakrishnan · 2019 · Proceedings of the 16th International Web for All Conference (W4A) · doi:10.1145/3315002.3317570
Summary
This paper investigates the accessibility of wrist gestures as an input modality for people with visual impairments (PVIs) interacting with smartwatches. Smartwatches hold particular promise for PVIs: they are always available, enable one-handed use (important when the other hand holds a white cane or guide dog), offer more privacy than a large phone screen, and are harder to steal. However, touch-based interaction with small smartwatch screens is extremely challenging for blind users, and screen magnifiers are ineffective on such small displays. Voice input is an alternative but is often impractical in noisy or public settings. Wrist gestures — movements of the hand wearing the watch — offer one-handed, eyes-free interaction, but their accessibility for PVIs had not been studied. The authors conducted a within-subjects study with 13 participants (7 blind, 6 low vision; ages 30-65) testing 10 wrist gestures: 2 discrete gestures from Android smartwatches (up flick and down flick) and 8 continuous uni-stroke gestures from the WristWhirl system (horizontal, vertical, major diagonal, minor diagonal, triangle, rectangle, circle, and question mark). Each participant performed all gestures 5 times in randomized order on an Android smartwatch with a logging app and the WristWhirl system. Gestures were taught through verbal description and physical hand guidance by the experimenter.
Key findings
The study measured both user accuracy (whether the experimenter could identify the intended gesture from the wrist movement) and system accuracy (whether the system correctly recognized the gesture). User accuracy for Android smartwatch gestures was high (around 80%), while among WristWhirl gestures, vertical (84.6%), circle (81.5%), and horizontal (60.0%) were most accurately performed. System accuracy averaged significantly lower (38.3%) than user accuracy (60.6%), indicating that even when users performed gestures correctly, the systems frequently misinterpreted them. The accuracy drop was most pronounced for Android up/down gestures and horizontal/vertical WristWhirl gestures. On subjective difficulty ratings (Borg CR10 scale), Android gestures were rated "somewhat hard" (avg 4.30) while WristWhirl gestures were rated "moderate" (avg 3.36), with vertical being easiest (2.67) and minor diagonal hardest (4.10). Three critical accessibility insights emerged: (1) Shape awareness — many blind participants were unfamiliar with visual shapes like triangles, rectangles, and question marks, and could describe shapes verbally but not reproduce them as mid-air gestures; (2) Orientation confusion — both blind and low-vision participants interpreted "vertical" and "horizontal" relative to 3D physical space rather than relative to wrist orientation, causing systematic errors; (3) Practical preferences — all participants preferred gestures mapped to high-level tasks (making calls, checking notifications) rather than low-level operations (menu browsing, item selection), and wanted to customize gesture-to-task mappings.
Relevance
This study provides foundational evidence for designing accessible wrist gestures for wearable devices — an increasingly important interaction modality as smartwatches become mainstream. The findings have direct implications for wearable device manufacturers and app developers. The key design principles that emerged are that accessible wrist gestures should: use shapes that are easy to envision non-visually (simple lines and circles rather than complex shapes like question marks or diagonals); be invariant to starting position and ending position; be invariant to wrist orientation (interpreting motion relative to physical space, not the wrist); and provide non-visual feedback during gesture execution, since blind users lack the visual confirmation that sighted users get when tracing shapes in mid-air. The significant gap between user accuracy and system accuracy also highlights that gesture recognition algorithms need to be more tolerant of the natural variation in how PVIs perform gestures. For accessibility practitioners, the study reinforces that interaction modalities designed for sighted users cannot simply be assumed to transfer to non-visual use — even seemingly simple gestures involve spatial and shape concepts that are experienced differently without vision.
Tags: wearable technology · smartwatch · gesture-based interaction · visual impairment · blindness · low vision · one-handed interaction · input modality · non-visual interaction