Three Modalities, Two Design Probes, One Prototype, and No Vision: Experience-Based Co-Design of a Multi-modal 3D Data Visualization Tool
Sanchita S. Kamath, Aziz N. Zeidieh, Venkatesh Potluri, Sile O'Modhrain, Kenneth Perry, JooYoung Seo · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26) · doi:10.1145/3772318.3791272
Summary
Kamath et al. report an Experience-Based Co-Design (EBCD) process in which five blind or low-vision (BLV) researchers and one sighted collaborator built a web-native, multimodal prototype for exploring 3D surface plots - a class of visualisation central to VUV spectroscopy, biomedical imaging, meteorology, and geoscience but historically inaccessible to BLV analysts. Rather than running a conventional user study, the team adopted EBCD (originally from health services research) to treat lived BLV expertise as core design material. The process unfolded across two months. Phase 1 produced an initial high-fidelity web prototype through daily and weekly collaboration between two blind researchers and one sighted implementer. Phase 2 ran in three thrusts: building a low-fidelity tactile probe (a muslin cloth draped over variable-height plastic tubes on a polystyrene foam board, with braille-labelled axes); refining the digital probe; and running two 90-minute co-design sessions where all five BLV co-designers compared the tactile and digital versions directly. Core analytic tasks targeted throughout were 3D orientation, landmark and peak finding, comparing local maxima versus global trends, gradient tracing, and identifying occluded features. The resulting prototype is built in five strict layers (Data, Engine, Accessibility, UI, Application) using Web Audio API, WebGL, and keyboard-only navigation, with about 15-30 ms keystroke-to-audio latency. Key features include reference sonification (a fixed 300 Hz origin tone triggered by the 0 key, plus replay of current position), stereo panning for the X-axis, loudness + reverb for Z-axis depth, a two-tier 20x20 wireframe overview with drill-down to point mode, a statistical Jump-to-Peak function, and a configurable buffer that lets users save regions, toggle between aggregated (mean) and sequential playback, and scope natural-language queries to a Gemini 2.5 Pro AI assistant.
Key findings
Session 1 surfaced an 'experiential gap': the tactile probe felt rich and spatially coherent, while the initial digital version felt abstract and disorienting even though it was technically accessible. Co-designers' pre-digital expectations about coordinate-axis orientation (Y on the board, Z pointing out) did not match the digital prototype's conventional vertical-Y/depth-Z layout, motivating a fixed reference sonification feature. Stereo panning worked well for X-axis tracing, but volume alone proved insufficient for depth perception on Z - reverb augmentation was required. Three of four features prioritised in Session 1 were implemented and re-tested in Session 2: reference sonification with replay (DG3) was fully validated, stereo + volumetric audio (DG4) was partially validated with refinements needed on depth cues, and the configurable buffer (DG5) was conceptually validated but surfaced four priority usability issues around two-step selection flow, boundary awareness, aggregation purpose, and save confirmation. Co-designers also generated speculative directions for embodied interaction: persistent haptics (e.g., Inverse3, Graphiti multi-line braille), mixed-reality play-mat-style physical navigation, and alternative input via trackballs, dials, or CNC-style hand trackers - all rejecting keyboard-centric interaction in favour of modalities that match the continuity of 3D data.
Relevance
For accessibility practitioners and researchers working on data visualisation, this paper is valuable on two levels. First, it delivers a concrete, web-native architecture that does not require specialised hardware - the separation of Data, Engine, Accessibility, UI, and Application layers is directly reusable by teams building non-visual analytic tools, and the four design principles the authors distil (multi-parameter redundancy, progressive disclosure, buffer-aggregation as analytic scoping, persistent spatial reference) translate cleanly to other continuous-surface domains. Second, the paper is an unusually detailed template for running EBCD in an HCI project, including the explicit mapping between canonical EBCD stages and the team's adapted activities. Limitations are real and candidly discussed: six expert BLV co-designers, a cross-sectional two-session design, three dense and fairly clean datasets, and no longitudinal or novice-user validation. Practitioners should treat the feature set as empirically grounded for expert analysts, not as a drop-in solution for BLV learners encountering 3D plots for the first time.
Tags: blind and low vision · data visualization · 3D visualization · sonification · spatial audio · tactile graphics · experience-based co-design · multimodal interaction · STEM accessibility · non-visual interaction · participatory design · co-design · web accessibility