Eye-Tracking-Driven Shared Control for Robotic Arms: Wizard of Oz Studies to Assess Design Choices
Anke Fischer-Janzen, Thomas M. Wendt, Daniel Görlich, Kristof Van Laerhoven · 2026 · ACM Transactions on Human-Robot Interaction · doi:10.1145/3796524
Summary
The paper presents two Wizard of Oz studies evaluating an eye-tracking-driven shared-control system for an assistive robotic arm — the 7-DoF Kinova Gen3 — designed for People with Severe Motor Disabilities (PSMD). PSMD here includes people with locked-in syndrome, cerebral palsy, late-stage ALS, and multiple sclerosis, who depend on hands-free inputs such as brain-computer interfaces, sip-and-puff switches, or gaze. In shared control, the user provides high-level intent (e.g., "drink from this cup") by fixating on an object, while the robot handles path planning, grasping, and execution. The Wizard of Oz method — in which an experimenter surreptitiously performs the automation — allowed the authors to obtain community feedback on the system before committing to a specific control algorithm. The first study was an online survey (n=34) distributed to three stakeholder groups — PSMD, their family members, and medical professionals — covering disability context, current assistive-technology use, perceived usability of the demonstrated system, and desired tasks. The second was a lab study (n=24) with non-disabled participants, split into prior-experience and non-prior-experience groups, who operated the robot directly while wearing Tobii Pro Glasses 3. Gaze data — dwell time, Areas of Interest (AoI), and missed AoI hits — were recorded to derive constraints for a future automatic object-recognition controller.
Key findings
The survey identified eating (33%), drinking (19%), environmental interaction (15%), pick-and-place (15%), and personal hygiene (11%) as priority tasks, broadly matching but refining prior ADL literature; tasks such as preparing meals and dressing were notably absent from participant priorities. Over half of survey participants stated that they or their care recipient could use the demonstrated system. Access to robotic arms was vanishingly rare in the PSMD group (1 of 14), reflecting high cost and long healthcare-approval timelines. In the hands-on study, 70% of gaze selections with a 500 ms dwell trigger succeeded; the longest observed sustained dwell was 5.6 seconds. Dwell time and missed AoI hits correlated significantly with four factors: object size (small objects produce more errors), object geometry (thin or overlapping shapes cause ambiguous selections), gaze direction (horizontal head-turning produces eye-tracker periphery errors for downward-near objects), and user intention (fixating on task-relevant sub-features such as a bottle cap reduces dwell on the centroid). Ambiguous selection between overlapping objects, the Midas Touch problem, and concerns about robot speed and safety were the main barriers. Prior-experience participants flagged more kinematic and safety concerns than novices, who focused on task realism.
Relevance
For accessibility practitioners working on assistive technology, the paper offers a rare disability-community-informed evaluation of gaze-driven robotics — a domain where users are frequently excluded from design because studies recruit non-disabled proxies. The authors' four design dependencies (object size, geometry, gaze direction, user intention) translate directly into constraints for developers building eye-tracking applications: minimum AoI size thresholds, bounding-box versus segmentation trade-offs, and the need for calibration that accounts for head-movement direction. The call to involve PSMD early — via online surveys accessible on the user's own device at their preferred time — is a practical methodological contribution for teams struggling to recruit participants with high support needs. Limitations include a small corpus (n=34 survey, n=24 hands-on), use of a single eye-tracker model, hard-coded trajectories in the prototype, and no representation of participants with intellectual disabilities in the stakeholder pool.
Tags: assistive robotics · eye tracking · gaze input · shared control · motor disability · activities of daily living · human-robot interaction · wizard of oz · participatory design
Standards referenced: WHODAS 2.0