Technical Perspective: Computation Where the (Inter)Action Is
Jeffrey P. Bigham · 2022 · Communications of the ACM · doi:10.1145/3531446
Summary
This one-page technical perspective accompanies the SoundWatch paper in Communications of the ACM. Bigham uses SoundWatch — a smartwatch prototype that detects audio events and displays descriptions for deaf and hard-of-hearing people — as a lens to explore broader questions about where computation should happen as devices move closer to our bodies. He argues that access technology has always been a window into the future: speech recognition became mainstream because people who found it difficult to type adopted it first, and smartwatches represent the next frontier by placing computation in a glanceable, always-attached form factor. Bigham draws an important distinction between sensing and understanding: SoundWatch telling you to "go open the door" versus alerting you it "might have heard a doorbell." The latter better protects user agency by enabling the human to reason about context — a design principle applicable far beyond accessibility.
Key findings
The piece identifies several key tensions in wearable accessibility computing. First, the trade-off between computational power and proximity to the user: the closer computation is to the interaction, the less powerful it is likely to be, creating a new human-centered design challenge. SoundWatch recognises 20 sounds using a distributed architecture (smartwatch, smartphone, remote server), but scaling to thousands of sounds will require more powerful devices. Second, the form factor matters enormously: smartwatch screens are always glanceable while phones are hidden in pockets; phones are intentionally carried while watches are attached — these differences fundamentally change interaction design. Third, model compression and efficient ML research are critical for bringing capable recognition to low-power wearable devices. Bigham argues that deciding where computation should happen is not only a technical question but a human one, requiring HCI research to understand user expectations across changing device capabilities.
Relevance
This brief but insightful commentary frames accessibility as a leading indicator for mainstream computing trends — a powerful argument for the strategic importance of accessibility research. For practitioners, Bigham's distinction between "sensing" (detecting a doorbell) and "understanding" (telling you to open the door) is a crucial design principle for any AI-powered assistive technology: systems should present information that enables users to exercise judgment rather than making decisions for them. This echoes themes from his later work on Morae (proactive pausing for user agency in UI agents). The piece is also relevant to anyone working on wearable accessibility, as it articulates the architectural challenges of distributing computation across watch, phone, and cloud — trade-offs that remain unresolved. The observation that early access technologies were "bulky, slow, inaccurate, and severely limited" but adopted anyway because they provided value underscores that accessibility users are often willing early adopters who drive innovation.
Tags: wearable technology · deaf and hard of hearing · sound recognition · smartwatch · assistive technology · machine learning · edge computing · accessibility