Investigating Sign Language Interpreter Rendering and Guiding Methods in Virtual Reality 360-Degree Content
Craig Anderton · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22) · doi:10.1145/3517428.3563373
Summary
This paper explores how sign language interpretation can be effectively presented in virtual reality 360-degree (VR360) three-degrees-of-freedom video content. The research addresses a significant gap: while VR accessibility guidelines exist for subtitles, there has been almost no empirical testing specific to sign language presentation in VR environments. The author notes that for many members of the Deaf community, sign languages are their primary language, and text captioning can be inaccessible due to grammar differences between sign languages and written text. The study uses a within-subject experimental design comparing two independent variables: rendering mode (fixed-position vs. always-visible) and visual guiding method (arrows vs. radar). Fixed-position rendering attaches the sign language interpreter video to the 360-degree video sphere at three positions spaced 120 degrees apart, while always-visible rendering attaches the interpreter to the user's field of view via a heads-up display. Arrow guides point outward toward active speakers, while radar uses a circular 2D minimap showing speaker and viewer positions. Eight participants with diverse backgrounds — including BSL users, Makaton users, and VR-experienced non-signers — tested four virtual environments created in Unreal Engine 5, completing questionnaires measuring presence, simulator sickness, usability, and user experience after each video.
Key findings
Fixed-position rendering produced significantly higher overall presence scores than always-visible rendering (M = 3.77 vs. M = 2.65, p = .033, d = 1.01), including higher spatial presence and involvement subscale scores. However, fixed-position rendering caused significantly more content blocking than always-visible rendering (Mdn = 3 vs. Mdn = 6 on blocking question, p = .046). Arrows were dramatically more usable than radar guiding, with SUS scores of 92.8 (A+ grade) versus 63.1 (C- grade, marginal acceptability), a very large statistically significant difference (p < .001, d = 3.53). Arrows also produced significantly higher sign language understanding (Mdn = 7 vs. 5.5, p = .026) and enjoyment (Mdn = 6.5 vs. 5.5, p = .041). Qualitative feedback revealed that participants found radar's field-of-view triangle confusing and missed directional cues, while arrows were universally understood. Sickness scores were low across all conditions with no significant differences. Timing was the main criticism of arrows, with participants reporting insufficient time to turn their heads between speakers.
Relevance
This research is important for practitioners working on accessible immersive media because it provides the first empirical sign language-specific data for VR360 content, rather than relying on adapted subtitle findings. The clear finding that arrows outperform radar as a guiding method gives developers a concrete design recommendation. The presence-versus-blocking trade-off between rendering modes suggests that both should be offered as user-configurable options, aligning with personalization principles in accessible design. The study also highlights that no commercially available VR360 media player currently supports sign language beyond burned-in video, pointing to a significant gap in the market. While the small sample size (n=8) and limitation to 3DoF content mean these findings need validation, they establish an important baseline for XR accessibility guidelines that include sign language users.
Tags: sign language · virtual reality · extended reality · 360-degree video · deaf accessibility · immersive media · user experience · usability testing
Standards referenced: UK Equality Act 2010 · UN Convention on the Rights of Persons with Disabilities · W3C Sign Language Guidelines