The TalkingBox.: Revealing Strengths of Adults with Severe Cognitive Disabilities
Filip Bircanin, Laurianne Sitbon, Bernd Ploderer, Andrew Azaabanye Bayor, Michael Esteban, Stewart Koplick, Margot Brereton · 2020 · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2020) · doi:10.1145/3373625.3417025
Summary
This paper presents a case study of TalkingBox, a tangible communication device co-designed with Chris (pseudonym), a young adult with a severe cognitive disability, and his support network at a community day centre in Brisbane, Australia. Chris is minimally verbal, communicates primarily through gross physical actions (going to the kitchen means "I need my poppers"), finger-flicking, and body movements, and had previously lost his ability to use a Picture Exchange Communication System (PECS) board after a serious illness. The design process, built on two years of fieldwork by the lead researcher (a former social worker trained in special education), began by identifying Chris's strengths rather than deficits: a remarkable memory (he could locate his car among hundreds at a shopping mall), an affinity for rectangular shapes and street signs, and a past ability to play memory matching games. TalkingBox combines a simple off-the-shelf gift box augmented with a Raspberry Pi 3, RFID reader, and speaker. Users insert RFID-tagged picture cards into a slot; when two matching cards are placed consecutively, the box plays an associated sound (a spoken word, e.g., "bowling"). The first set of 25 card pairs featured photographs of personally meaningful items taken by Chris's support worker Mitch — his bed, favorite sandwich, beach, chips.
Key findings
Over a seven-week field study, TalkingBox produced four significant outcomes beyond its intended purpose of supporting symbolic communication. First, as a personalized memory game, Chris adopted it immediately into his daily routine (10-15 minutes per day), and by end of week two began waiting for the spoken word to finish before inserting the next pair — showing emerging interest in the audio-symbolic connection. Second, as an unanticipated social catalyst, the device attracted other adults at the centre who approached, sat with Chris, and engaged — one minimally verbal peer began mimicking spoken words ("beach," "skyzone," "chips"), and Mitch commented "he has never seen these two guys playing together" like that. A transparent intermediary box was designed so Chris could place cards for a peer (Ryan, who has severe autism) to retrieve and insert, bridging the interaction gap. Third, as a visual probe, the device revealed previously unknown visual discrimination skills — when street sign cards were added, Chris learned over three weeks to match signs that required detailed discrimination (e.g., distinguishing similar turn-left vs. turn-right signs). Fourth, as a living portfolio, TalkingBox shifted how others perceived Chris: support workers who previously saw him as "an enigma" and "anxious and introverted" began approaching and saying "I didn't know this, this is cool." A senior manager proposed expanding the design to other centres. Chris's parents, seeing his progress, decided to re-engage speech therapy: "We are considering speech therapy. It seems that he is in a good place now." Speech and occupational therapists who visited recommended building clinical milestones around TalkingBox.
Relevance
This paper offers a powerful alternative framing for assistive technology design for people with severe cognitive disabilities: rather than a deficit-oriented approach (what can't the person do?) or a solution-oriented approach (what technology will fix the problem?), it demonstrates a strengths-based approach where technology serves as a probe that reveals capabilities, interests, and social opportunities. For accessibility practitioners and AAC researchers, the key insight is that the TalkingBox was not designed as a communication device per se — it was designed around an existing strength (memory matching) that happened to create conditions for communication, social interaction, and skill revelation. The tangible, mid-tech design (no screens, simple card-slot interaction) was critical: it avoided the complexity of tablet-based AAC systems that had previously failed for Chris, and its physical presence in the room naturally attracted peer engagement. The concept of assistive technology as a "living portfolio" — an artifact that makes a person's strengths visible to their community — is a significant conceptual contribution. Limitations include the single-participant case study, dependence on one support worker (Mitch), confounding factors of novelty and researcher presence, and questions about long-term sustainability.
Tags: intellectual disability · augmentative and alternative communication · tangible interaction · co-design · participatory design · cognitive accessibility · empowerment · social interaction