WordMelodies: an Inclusive Mobile App Supporting the Acquisition of Literacy Skills
Dragan Ahmetovic, Cristian Bernareggi, Irene Mantegazza, Sergio Mascetti · 2021 · Proceedings of the 18th International Web for All Conference (W4A '21) · doi:10.1145/3430263.3452443
Summary
This paper presents WordMelodies, a cross-platform mobile edutainment application designed for inclusive literacy education for primary school children with and without visual impairments or blindness (VIB). The app was motivated by a teacher of children with VIB in the USA who identified a significant gap in accessible applications for practicing literacy skills. While assistive technologies for mathematics education for students with VIB have been extensively researched, literacy education tools remain scarce and often limited to Braille proficiency. The app was developed through two design iterations informed by formative studies with domain experts — including a congenitally blind assistive technology expert and teachers of children with VIB. WordMelodies uses a Universal Design approach, employing "audio-icons" that combine visual and auditory representations so that all interactive elements are accessible through both sight and hearing. The app includes over 80 exercise types across multiple families (drag-and-drop, multiple choice, word grids, word completion) with configurable difficulty levels. It was built using React Native for cross-platform availability on iOS and Android, though this required custom native code to work around accessibility limitations in React Native, such as the inability to trigger events on screen reader focus changes. The app is available in English and Italian, with language-specific exercises rather than simple translations, recognizing that different languages require different literacy skill practice.
Key findings
Accessibility evaluation with three adult participants with VIB revealed practical issues: two participants with low vision reported poor colour contrast in GUI elements, and a blind participant found that Italian translations of verbal messages (originally in English) were unclear despite being linguistically correct — highlighting that direct translation is insufficient for accessible app localization. A React Native-specific issue was that screen reader focus order did not follow logical UI element order when using flick gestures, requiring careful element dimensioning to resolve. Preliminary usage data from one month showed 194 unique users, of whom 27 used a screen reader. iOS devices were more common (128 users) but Android represented a significant share (66 users, about one-third), exceeding prior literature estimates of 20% Android usage among people with VIB. Tutorials were activated by 99 users (over half), with 21 specifically accessing the drag-and-drop tutorial. A total of 1,070 exercises were completed, with drag-and-drop exercises ranking as the second most popular in both languages and showing error rates consistent with other exercise types (~10%). Notably, 66 Italian users also performed exercises in English, suggesting interest in using the app for foreign language learning.
Relevance
WordMelodies demonstrates important principles for accessible educational app development. The Universal Design approach — designing one app usable by all children rather than a separate tool for those with VIB — directly addresses the social exclusion that can result from specialized-only assistive technologies. For mobile accessibility practitioners, the paper offers practical lessons about cross-platform development challenges: React Native's accessibility gaps required platform-specific native code, and screen reader focus ordering required careful attention to element dimensions. The finding that language localization for accessible apps must go beyond translation — requiring language-specific exercise design and natural-sounding instructions — is relevant to any team building multilingual accessible applications. The usage data showing significant Android adoption among users with VIB challenges the assumption that accessibility apps can target iOS only. The app's release during COVID-19 school lockdowns also highlights the critical importance of accessible remote learning tools.
Tags: visual impairment · education accessibility · mobile accessibility · inclusive design · screen readers · literacy · edutainment