Demonstration: Screen Reader Support for a Complex Interactive Science Simulation
Taliesin L. Smith, Clayton Lewis, Emily B. Moore · 2016 · ASSETS '16: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2982142.2982154
Summary
This demonstration presents an accessible version of "Balloons and Static Electricity," a PhET interactive science simulation that teaches electrostatic charge concepts. PhET simulations are run over 75 million times annually, yet most are inaccessible to blind learners due to their reliance on dynamic visual representations and mouse/touch interaction. The researchers developed three key components for accessibility: (1) a Parallel Document Object Model (PDOM)—an HTML data structure representing simulation objects that screen readers can access, since the original graphical representations are inaccessible; (2) keyboard navigation with custom hotkeys to move the balloon to important areas and perform drag/rub interactions; and (3) natural language descriptions read by the screen reader during exploration and interaction. The approach was constrained by pedagogical requirements: no specialized hardware (must work in standard classrooms), and accessibility features must be integrated into the same simulation used by sighted learners rather than a separate "accessible version." This ensures blind students can participate in the same learning activities as their peers.
Key findings
The researchers identified three challenges unique to complex interactive simulations that extend beyond existing guidance for static content and simple interactive graphics: **Unpredictable event sequences:** Unlike video description where the describer knows the sequence of events, interactive simulations have user-driven, unpredictable sequences. The solution was creating modular descriptions intelligible when reordered and recombined. The researchers noted that future work should develop history-dependent descriptions where repeated events receive shorter descriptions than first occurrences. **Objects as both controls and displays:** The balloon posed unique challenges—it behaves like a control (moving it changes simulation state), but also as a display (visual charge indicators appear on its surface, and it moves autonomously when released due to electrostatic attraction). This required three distinct description types: (1) object description when exploring (charge level, position, effects on nearby objects); (2) interaction description explaining how to use non-standard drag/release controls; (3) action effect description of what happened when the user acts. **States versus state changes:** Rubbing the balloon produces both a new state (charge distribution after motion) and an event (charges moving from sweater to balloon). Sighted users see these integrated; blind users need separate access. The solution used ARIA Live Regions to immediately announce action effects, while current state information remained accessible via PDOM navigation. Change descriptions were stored as hidden content to prevent out-of-context playback.
Relevance
This work addresses a significant gap in STEM education accessibility. Interactive simulations are increasingly central to science pedagogy, offering discovery-based learning that static content cannot provide. Making these simulations accessible enables blind students to engage with the same inquiry-based learning experiences as sighted peers, rather than receiving inferior alternative materials. The technical contributions—particularly the PDOM architecture and the framework for describing dynamic interactive content—are applicable beyond PhET to other interactive educational tools, data visualizations, and web applications with complex state management. The distinction between state descriptions (navigable, re-readable) and change descriptions (immediate, contextual, hidden when not relevant) provides a useful model for dynamic content accessibility. The constraints the team worked under (standard hardware, integrated accessibility) reflect real-world deployment requirements. Solutions requiring specialized equipment or separate "accessible versions" often fail to reach students who need them. By building accessibility into the mainstream simulation, the approach ensures that blind students can participate in group activities and that teachers need not manage multiple versions of educational materials.
Tags: screen reader · STEM education · interactive simulation · PhET · blind accessibility · ARIA · accessible description · science education · keyboard navigation
Standards referenced: ARIA