Toward a User Experience Tool Selector for Voice User Interfaces
Andreas M. Klein · 2021 · Proceedings of the 18th International Web for All Conference (W4A) · doi:10.1145/3430263.3456728
Summary
This doctoral consortium paper proposes the development of a UX tool selector for voice user interfaces (VUIs), presented as a web application that recommends appropriate measurement tools for evaluating VUI quality. Voice assistants such as Alexa, Google Assistant, and Siri have become ubiquitous across devices and systems, yet the ability to systematically measure and improve their user experience remains underdeveloped. While several approaches to measuring VUI UX quality exist, they require extensive resources and there are no general guidelines for applying UX assessment tools specifically to VUIs. The author notes a particularly important finding from pilot research: people with disabilities use VUIs very intensively for activities such as web browsing and gaming, making UX quality evaluation especially relevant for accessibility. The proposed tool selector follows the Design Science Research Methodology (DSRM) and addresses four research questions: identifying the state of the art in VUI UX measurement, understanding users' needs and concerns, determining how to design such a selector, and evaluating whether it provides suitable recommendations. The selector draws from a UX measurement toolbox containing both standard and new VUI-specific assessment methods, and uses the context of use to determine which tool to recommend.
Key findings
The research is at an early stage, with the paper outlining the methodology and planned contributions rather than reporting completed results. A pilot study has already revealed that users have concerns about voice assistants and identified technical limitations with current VAs. The pilot also found that people with disabilities are particularly intensive VUI users, relying on them for everyday tasks like web surfing and gaming — highlighting an underserved population whose UX needs are not being systematically measured. The planned contribution includes: a comprehensive literature review of VUI UX measurement approaches, empirical studies to understand user needs, the design and development of the web-based tool selector application, and an evaluation of whether the selector provides suitable context-dependent recommendations. The DSRM framework structures this into six iterative steps from problem identification through evaluation.
Relevance
This research addresses an important gap at the intersection of voice interaction and accessibility. As voice assistants become a primary interface for people with disabilities — particularly those with motor or visual impairments who may find traditional GUIs difficult — the ability to systematically evaluate VUI user experience becomes critical for ensuring these interfaces actually serve their users well. The proposed tool selector could lower the barrier for practitioners who want to evaluate VUI accessibility but lack the expertise to choose appropriate measurement methods. While this is an early-stage doctoral proposal rather than a completed study, it highlights the need for standardized, context-aware UX evaluation approaches for voice interfaces, which remain a relatively new and rapidly evolving modality. The focus on making evaluation accessible without extensive prior research aligns well with practical accessibility work.
Tags: voice user interfaces · user experience · evaluation methods · voice assistants · conversational user interfaces · accessibility testing