Designing Voice-Assisted Technology (VAT) Training for Activities of Daily Living (ADLs) for Adults with Cognitive-Communication Needs (CCNs) at Home
Claire O'Connor, Lauren H Kim, Ginna Byun, Priyal Vora, Yao Du · 2023 · ASSETS 2023: The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3615656
Summary
This paper describes the design, implementation, and preliminary evaluation of a 6-week virtual training program that teaches adults with cognitive-communication needs (CCNs) resulting from traumatic brain injury (TBI) to use Amazon Alexa voice commands for activities of daily living (ADLs). The program was developed through interdisciplinary collaboration between three graduate speech-language pathology (SLP) student clinicians supervised by a licensed SLP, and two informatics researchers with prior experience in Alexa research. Over three months, the clinical and informatics teams conducted a mapping activity correlating ADLs with four cognitive-communication domains (attention, memory, problem-solving, and sequencing), generating 265 user stories across 11 Alexa command categories and verifying 281 corresponding voice commands. The resulting curriculum covered five weekly training modules: Scheduling and Reminders, Entertainment, Self-Care and Medical Needs, News and Facts/Communication, and Meal Preparation. Three adults with TBI (ages 25, 55, and 82; all 5-6 years post-injury) and their caregivers participated in 1-hour weekly virtual sessions via Zoom, each receiving an Amazon Echo Show 5. Clinicians used evidence-based practice strategies including errorless learning, multimodal prompting (visual and verbal cues), and command breakdowns to teach participants to articulate concise commands. Baseline cognitive-linguistic assessments were administered using the CLQT+, WAB, RIPA-2, and an adapted Lawton-Brody Instrumental ADL Scale. Video analysis with inter-rater reliability of 88.6% for accuracy and 89.4% for cueing was used to track participant performance across sessions.
Key findings
All three participants showed positive trajectories in VAT use over the training period. P1 (age 55) maintained 100% command accuracy across most sessions with minimal to no cueing, demonstrating high independence. P2 (age 25) attended only 2 of 6 sessions but achieved 86-100% accuracy when present. P3 (age 82), who had the most difficulty due to spontaneous speech patterns that caused Alexa to "cut him off," improved from 59% accuracy in Week 1 to 66% in Week 5, with cueing needs decreasing from 5 verbal cues initially to 8 out of 10 successful cues by the final week, indicating growing receptivity to instruction. Key benefits reported by participants included increased ability to independently perform information-seeking tasks, set reminders for medications and appointments, and reduced caregiver burden. Challenges included voice recognition difficulties (Alexa not detecting speech, mumbling), producing concise commands rather than conversational speech, navigating device controls (on/off, mute), and frustration when the device returned incorrect or unwanted information. Participants learned from each other during group training, with P1 learning about medication reminders from P3. Caregivers recommended future training focus on complex ADLs like meal preparation and medication management, and suggested written instructions to supplement video training.
Relevance
This study fills an important gap by providing a clinician-designed, systematic training curriculum for teaching voice assistant use to adults with acquired cognitive-communication disabilities. For accessibility practitioners, the detailed curriculum design process, including the mapping of 265 user stories to specific Alexa commands filtered for cognitive feasibility (short utterances, 1-2 step commands), offers a replicable framework for developing VAT training in clinical and community settings. The telehealth delivery model demonstrates that virtual training can effectively reach geographically dispersed participants, though the study highlights that remote instruction limits the ability to model and prompt device use. The finding that participants with TBI can learn and retain voice commands with appropriate scaffolding supports the broader case for voice assistants as cognitive prosthetics for daily living. However, the very small sample size (N=3), variable attendance, and lack of a control group limit generalizability. Future work should expand to a 12-week curriculum with larger cohorts and include mental health support modules, given that approximately 28% of stroke survivors and 20.5% of TBI survivors experience anxiety disorders.
Tags: voice assistants · cognitive-communication needs · traumatic brain injury · activities of daily living · telehealth · speech-language pathology · Amazon Alexa · cognitive rehabilitation · assistive technology training · independent living