Designing a Customizable Picture-Based Augmented Reality Application For Therapists and Educational Professionals Working in Autistic Contexts
Tooba Ahsen, Christina Yu, Amanda O'Brien, Ralf W. Schlosser, Howard C. Shane, Dylan Oesch-Emmel, Eileen T. Crehan, Fahad Dogar · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22) · doi:10.1145/3517428.3544884
Summary
This paper presents CustomAR, a mobile augmented reality application designed to allow therapists and educational professionals to create and customize picture-based AR experiences for use in autism-related learning activities. The work addresses a critical gap: while prior research has demonstrated that AR can increase focus and engagement in autistic children and support tasks like daily living skills, emotion recognition, and communication, existing AR applications for autism lack customization capabilities. Therapists need to tailor learning exercises to each child's unique interests, abilities, and behavioral profile, but most AR tools come with pre-built, fixed content that cannot be adapted. The application was developed through a two-phase participatory design process. In Phase I, the research team collaborated with four speech language pathologists at Boston Children's Hospital over several months to create an initial AR application targeting symbolic development — helping minimally verbal autistic children understand that 2D pictures represent 3D objects. When children pointed to a picture, the app superimposed a matching 3D model, reinforcing the picture-object relationship for choice-making activities. Informal testing revealed that the fixed content was too restrictive, especially when COVID-19 shifted therapy online and children no longer had access to the clinic's specific objects. In Phase II, the team redesigned the application as a customizable authoring tool. Therapists can now upload their own target images, associate them with audio prompts, videos, and 3D models (from a built-in library of 40 models), and adjust animations, positioning, and display settings. A key innovation is the "freeze" feature, which captures the AR experience on screen so it remains visible even after the device moves away from the target image — addressing the practical reality that autistic children may have limited attention and may not stay near the target image. A 2-week diary study with 7 therapists and educational professionals evaluated the application.
Key findings
Therapists found the application's customization options sufficient for creating AR experiences across a wide range of learning activities, including choice-making, teaching daily living skills (tooth-brushing, hand-washing), emotion recognition, visual schedules, collaborative group work, and reinforcing language concepts. Participants appreciated that the application provided a balanced level of flexibility — enough options to be creative without overwhelming users with complexity. The limited set of customization categories (audio, video, 3D models) with open-ended content within each category was seen as a successful design approach. The freeze feature was unanimously valued for working with autistic children who have limited attention spans or difficulty staying near the target image. Therapists envisioned using it in group settings where they could trigger the AR experience, freeze it, and then move with the device to show individual children. The feature also supports scenarios where therapists need to leave the target image on a desk and follow a child who wanders. Several practical challenges emerged: the limited library of 3D models was the most significant barrier, as therapists wanted models matching each child's specific interests. Participants also highlighted the need for just-in-time content creation during therapy sessions rather than requiring advance preparation, the ability to share AR experiences between therapists and families to support generalization of skills across environments, and the potential for a simplified interface that could allow autistic children themselves to create AR experiences.
Relevance
This research provides important design insights for developers creating customizable assistive technology applications, particularly for autism contexts. The key lesson — that customization should offer flexibility within a constrained set of options rather than unlimited freedom — is broadly applicable to tools targeting therapists and educators who need practical, quick-to-use technology. The finding that too many options causes users to get "bogged down" while too few limits usefulness reflects a design tension common in assistive technology. The freeze feature represents a practical innovation that any AR application targeting users with attention difficulties should consider implementing. The study also underscores the importance of involving therapists and educators as co-designers, as they identified real-world constraints (dynamic therapy environments, children's wandering behavior, COVID-19 disruptions) that researchers alone might not anticipate. For accessibility practitioners, the paper demonstrates how AR can extend existing evidence-based practices like visual supports and PECS, rather than replacing them, making adoption more natural for professionals already using picture-based strategies.
Tags: augmented reality · autism · visual supports · customization · therapy · education · participatory design · AAC