Making the Blockly Library Accessible via Touchscreen
Logan B. Caraco, Sebastian Deibel, Yufan Ma, Lauren R. Milne · 2019 · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2019) · doi:10.1145/3308561.3354589
Summary
This demonstration paper presents two prototype designs for making Google's Blockly library — the foundation underlying popular block-based programming environments (BBPEs) like Scratch, MIT App Inventor, and the Code.org curriculum — accessible to users with visual impairments on touchscreen devices. BBPEs allow learners to construct programs by dragging and dropping puzzle-like code blocks, an inherently visual and gesture-dependent interaction that excludes blind and motor-impaired students. As computer science education becomes standard in K-12 curricula, this accessibility gap is increasingly problematic. The authors built on their prior work with Blocks4All (a purpose-built accessible BBPE) and targeted the Blockly library specifically because making it accessible would cascade benefits to all applications built on it. The work focuses on touchscreen devices (iPads and Android tablets) because evidence shows many students with visual impairments learn assistive technology on touchscreens before desktop computers. The implementations use pure JavaScript with no external libraries, follow WCAG 2.0 AA standards, and use ARIA labels and roles to expose block structure to screen readers (TalkBack on Android, VoiceOver on iOS). The code is open source on GitHub with the goal of eventual integration into the core Blockly library.
Key findings
The two designs offer contrasting approaches to the fundamental tension between linearizing spatial code and preserving its 2-D structure. The Hierarchical List Design converts the block workspace into a navigable list placed at the screen edge for easy non-visual discovery. Blocks are organized into lettered stacks with nesting reflected in sub-lists, and ARIA labels are auto-generated from block data. Selecting a block opens a detail view with a breadcrumb trail showing code scope, movement options, parent/child navigation, and editing controls for dropdowns and text fields. Block movement mirrors drag-and-drop: selecting a block reveals valid connection points in the list. The Spatial Design preserves the original 2-D layout but overlays accessible rectangles on each row and editable field, each with ARIA labels. Users can touch blocks directly (useful for those with some vision) or swipe sequentially in reading order. The design adds "End of Block" markers for nested statements and parenthesis annotations around logic/arithmetic operators to clarify structure. Connected blocks are reorganized vertically rather than inline to improve screen reader selectability. Neither design has yet been tested with users with visual impairments — user testing is planned as future work.
Relevance
This work tackles a critical equity issue in computing education: as block-based programming becomes the standard entry point for K-12 computer science, blind and low-vision students are effectively locked out. By targeting the Blockly library rather than building a standalone tool, the authors chose a high-leverage intervention point — accessibility improvements to Blockly could benefit Scratch, Code.org, App Inventor, and countless other educational platforms simultaneously. The two design approaches represent a genuine design tension in accessible programming: linearization is simpler to navigate but loses spatial relationships, while preserving 2-D layout maintains the original metaphor but creates navigation complexity. The main limitation is the absence of user evaluation — the designs remain untested with the target population. For developers building educational coding tools, this work provides concrete ARIA labeling patterns for making visual block structures screen-reader accessible, applicable beyond Blockly to any drag-and-drop interface.
Tags: visual impairment · block-based programming · education · screen readers · ARIA · touchscreen accessibility · K-12 education · coding education
Standards referenced: WCAG 2.0 · ARIA