Movement as Medium: Personalisation of Instruments for Inclusive Creative Expression in Disability-Led Performance
Sam Trolland, Melinda Smith, Alon Ilsar, Jon McCormack · 2026 · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26) · doi:10.1145/3772318.3791955
Summary
This CHI 2026 practice-based case study argues for personalisation — rather than generalised accessibility — as a design methodology for embodied gestural instruments used by performers with physical disability. The authors frame their contribution against Accessible Digital Musical Instrument (ADMI) research that often relies on corrective, medical-model assumptions and generic gesture vocabularies that homogenise expression. They draw explicitly on the Capability Approach (Sen, Mitra) and Capability Sensitive Design (Oosterlaken) to position disability as a shortfall in practical opportunities to do and be what one values, arguing that technology should be judged by whether it amplifies the specific capabilities a person has reason to value. The work centres on a multi-year collaboration with Dr Melinda Smith ("Mel"), a professional dancer and visual artist with physical disability. Using Practice-based Research (PbR) and Research-through-Design (RtD), the team iterated across four phases — from wrist-worn sensors to a handheld, digitally fabricated "interactive paintbrush" with custom PCB, wireless firmware, and real paintbrush bristles — that drove sound (Ableton Live, quadraphonic), visuals (Unity, via OSC and MIDI), and stage lighting (ETC console) in real time. The instrument was ecologically validated in Conduit Bodies, a five-show season at Melbourne Fringe Festival 2024 that won a Green Room Award. The methodology was then extended through four co-led workshops with six young people (ages 5-16) with motor, sensory, and communication disabilities at the Cerebral Palsy Education Centre (CPEC), supported by speech pathologists and occupational therapists.
Key findings
Personalisation had to extend beyond parameter tuning to the physical form, metaphor, and ownership of the instrument. The shift from body-worn sensors to a handheld paintbrush came from Mel's embodied observation that she has greater control gripping a brush than moving a wrist-mounted device, and the paintbrush form embedded her identity as a painter into the system. Digitising her own paintings and using them as visual textures in the performance — and later using workshop participants' photographs — shifted personalisation from accommodating impairment to foregrounding creative authorship. In workshops, manual per-session calibration by the researcher worked but did not scale; an automatic calibration algorithm that learned each participant's movement range in real time preserved range accommodation while removing facilitator bottlenecks, and unexpectedly supported collaborative multi-user modes. Speech pathologists reported that several young participants with "limited to no functional use of their hands" created digital art independently for the first time using eye, head, or small hand movements. The authors distill five design strategies: check in regularly with performers; build flexible interaction prototypes; celebrate both movement diversity and creative vision; fine-tune the whole experience of interaction (not isolated tech tests); and "less is more" — simplify mappings and allow moments without digital augmentation. Critical reception (a five-star Time Out review; a 2024 Green Room Award for Outstanding Technical Achievement) is offered as legitimate ecological-validity evidence.
Relevance
For accessibility practitioners, the paper is a concrete counterweight to "universal" ADMI or AT design: the authors argue and demonstrate that designing for one person, deeply, generates transferable design knowledge. The Capability Approach framing gives practitioners a vocabulary for justifying bespoke, high-effort work against efficiency arguments — the success metric is expanded opportunities the person has reason to value, not feature parity with a generic user. The progression from professional artist to mentored community workshops also shows how disability-led creative practice can seed downstream inclusion for others with similar bodies. Limitations to flag: the core case study is a single performer with a long prior relationship with the first author, and the workshop cohort was six participants over four sessions — generalisability is deliberately not the aim, but practitioners adapting these strategies should expect significant facilitator expertise (clinicians, speech pathologists, OTs) to be part of the operating cost. The instrument also relies on fabricated hardware and custom firmware, which raises questions about maintainability, repairability, and handover once the original designer is no longer available.
Tags: accessible digital musical instrument · gestural interaction · co-design · participatory design · physical disability · creative accessibility · performance · personalisation