Intelligent Assistive Communication and the Web as a Social Medium
Karl Wiegand · 2014 · Proceedings of the 11th Web for All Conference (W4A) · doi:10.1145/2596695.2596725
Summary
This short paper presents doctoral research on applying natural language processing (NLP) and machine learning to improve augmentative and alternative communication (AAC) systems, with a particular focus on enabling web-based social interaction for people with severe speech impairments. The author identifies that individuals with conditions such as cerebral palsy or amyotrophic lateral sclerosis (ALS) often have co-occurring physical impairments that limit their use of sign language or written communication, making AAC devices their primary means of expression. Current AAC systems, both letter-based (2-5 words per minute) and icon-based (under 15 words per minute), are slow and physically fatiguing due to the sequential, discrete selection process they require. The research challenges three fundamental design assumptions in icon-based AAC: that icons must be selected in syntactic order, that the system receives exactly the intended set of icon selections, and that each icon requires a discrete selection action. By relaxing these assumptions using computational intelligence, the work aims to shift the cognitive and physical burden from the user to the system.
Key findings
The author developed two working systems that challenge conventional AAC design assumptions. RSVP-iconCHAT is being integrated with a surface-level brain-computer interface (BCI) for users with locked-in syndrome, allowing free-order icon selection. SymbolPath is an overlay system for existing icon-based AAC devices that enables continuous motion input with superset selections in free order, available on the Android platform. Initial user feedback indicated both systems provide desirable and effective new approaches across a range of populations. SymbolPath in particular opens possibilities for novel input methods using continuous signals such as vowel sounds or electromyography (EMG). The research also gathered empirical evidence on how contextual cues like date, time, and location can help predict intended language usage, especially in social networks and text messaging contexts.
Relevance
This paper highlights a critical but often overlooked aspect of web accessibility: the barriers that AAC users face when trying to participate in online social interaction. While most web accessibility work focuses on sensory or motor access to content, this research addresses the fundamental challenge of communication speed and fatigue for people who rely on assistive communication devices. As AAC users increasingly turn to social media platforms like Facebook, Twitter, and web-based chat for social connection — often because they have limited local conversation partners — the gap between the pace of online communication and AAC output rates becomes a significant barrier to participation. The work points toward a future where intelligent interfaces combined with accessible web standards could enable meaningful social participation for people with complex communication needs.
Tags: augmentative and alternative communication · AAC · natural language processing · machine learning · social media accessibility · brain-computer interface · speech impairment · icon-based communication