Context aware documenting for aphasics
Thomas de Wolf, Dirk W.H. Gooren, Jean-Bernard O.S. Martens · 2008 · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '08) · doi:10.1145/1414471.1414541
Summary
This paper describes two student design projects aimed at creating tools for people with aphasia to document and share their everyday experiences. Aphasia, a communication impairment caused by brain damage (typically from stroke), affects about 0.3% of the population, leaving intelligence largely intact while severely reducing the ability to speak, comprehend, read, and write. The designers adopted an empathic design approach, prohibiting themselves from speaking for one day and caring for a person with aphasia for another, to understand the user experience firsthand. Because direct communication with the target users was difficult, the team used proxy feedback from 10 caregivers (speech therapists, social workers, and others) who evaluated design concepts presented in a booklet of 10 scenarios. The two final prototypes both use automated image capturing and provide browsing of collected images. "Pixcel" is a static living-room system using three webcams pointed at key social areas (couch, dining table, door) that automatically capture images when movement is detected, with a linear LED array on each webcam for browsing via touch. "Taile" is a mobile device that captures images and short sounds throughout the day, triggered by movement or user-initiated touch, with a timeline-based browsing interface where blue areas represent automatic captures and green areas represent user-initiated ones.
Key findings
The proxy feedback process revealed several important design considerations for people with aphasia: pictures are the most effective medium for communicating stories about daily experiences, since many people with aphasia have trouble with abstractions and detailed actions (often due to one-sided paralysis). A chronological presentation was identified as the best way to help users organize their thoughts and tell a story. The caregivers who provided feedback noted that collected pictures and sounds can help start conversations, but cautioned that an abundance of information would do more harm than good — simplicity is essential. Drawing was deemed unsuitable for most users with aphasia. Only 2 of 10 caregivers provided detailed feedback on the design booklet, suggesting that a more personal, hands-on approach to gathering proxy feedback would have been more effective. The two prototypes differ primarily in mobility: Pixcel creates a dedicated storytelling space in the living room, while Taile is portable and separates image capture from viewing. Both concepts assume that pictures establish the basic elements of a story and can serve as a starting point for conversation between people with aphasia and those close to them.
Relevance
This research addresses communication accessibility for a population that is often underserved by digital technology — people with aphasia whose linguistic abilities are severely impaired but whose cognitive and social needs remain. The approach of using automated image capture to build a visual diary removes the burden of deliberate documentation, which is particularly important for users who may have physical limitations (one-sided paralysis) alongside their communication impairments. For accessibility practitioners, the project illustrates several valuable lessons: the empathic design method of experiencing communication limitations firsthand, the use of proxy feedback when direct user testing is challenging, and the principle that assistive communication tools should reduce rather than increase cognitive and physical demands. The tangible interaction design — browsing images by sliding a finger along an LED array or along a physical timeline — demonstrates alternatives to screen-based interfaces that may be more intuitive for some users. The work also highlights the social dimension of accessibility: the goal is not just to capture information but to support social storytelling and connection with others.
Tags: aphasia · tangible interaction · context-aware computing · augmentative and alternative communication · storytelling · photo-based communication · participatory design · independent living