Reviews

Search results

• Visualization of Speech Prosody and Emotion in Captions: Accessibility for Deaf and Hard-of-Hearing Users

  Caluã de Lacerda Pataca, Matthew Watkins, Roshan Peiris, Sooyeon Lee, Matt Huenerfauth · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)

  This CHI 2023 paper tackles a dimension of captioning that has gone largely unaddressed for four decades: captions depict words but strip out the prosody and emotion carried by a speaker's voice. The authors argue that while automatic speech recognition (ASR) has reduced word…

  captioning · deaf and hard of hearing · prosody · affective computing · videoconferencing accessibility

• Access on Demand: Real-time, Multi-modal Accessibility for the Deaf and Hard-of-Hearing based on Augmented Reality

  Roshan Mathew, Brian Mak, Wendy Dannels · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This experience report documents two deaf researchers' hands-on evaluation of Access on Demand (AoD), an augmented reality application developed at Rochester Institute of Technology that delivers real-time captioning and American Sign Language (ASL) interpretation through Vuzix…

  augmented reality · deaf and hard of hearing · smart glasses · captioning · sign language interpretation

• Understanding Social and Environmental Factors to Enable Collective Access Approaches to the Design of Captioning Technology

  Emma McDonnell · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This doctoral consortium paper presents a dissertation research program that reimagines how captioning technology should be designed by applying the disability justice principle of collective access — the idea that accessibility is a shared responsibility of all group members,…

  captioning · collective access · disability justice · deaf and hard of hearing · co-design

• "It's Just Part of Me:" Understanding Avatar Diversity and Self-presentation of People with Disabilities in Social Virtual Reality

  Kexin Zhang, Elmira Deldari, Zhicong Lu, Yaxing Yao, Yuhang Zhao · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This paper investigates how people with disabilities (PWD) perceive, design, and use avatars for self-presentation and disability disclosure in social virtual reality (VR) platforms. The researchers employed a two-part methodology: first, a systematic review of 15 popular…

  social virtual reality · avatars · disability disclosure · self-presentation · deaf and hard of hearing

• Designing Gestures for Digital Musical Instruments: Gesture Elicitation Study with Deaf and Hard of Hearing People

  Ryo Iijima, Akihisa Shitara, Yoichi Ochiai · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This study addresses a gap in how deaf and hard of hearing (DHH) people can enjoy playing musical instruments using smartphones. While DHH people can and do enjoy music — often sensing it through vibrations transmitted by instruments or through body movements while performing —…

  deaf and hard of hearing · music accessibility · gesture elicitation · digital musical instruments · haptic feedback

• Beyond Subtitles: Captioning and Visualizing Non-speech Sounds to Improve Accessibility of User-Generated Videos

  Oliver Alonzo, Hijung Valentina Shin, Dingzeyu Li · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This paper investigates a significant gap in video captioning: the representation of non-speech sounds. While automatic speech recognition (ASR) has become widely available for generating captions on platforms like YouTube, TikTok, and Zoom, these systems focus exclusively on…

  deaf and hard of hearing · captioning · non-speech sounds · automatic captions · video accessibility

• Support in the Moment: Benefits and use of video-span selection and search for sign-language video comprehension among ASL learners

  Saad Hassan, Akhter Al Amin, Caluã de Lacerda Pataca, Diego Navarro, Alexis Gordon, Sooyeon Lee, Matt Huenerfauth · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This paper investigates technology to support American Sign Language (ASL) learners in comprehending challenging sign-language videos by enabling in-context dictionary lookup. The researchers conducted two studies at Rochester Institute of Technology. Study 1 interviewed 14 ASL…

  sign language · American Sign Language · ASL · video comprehension · dictionary search

• Technical Perspective: Computation Where the (Inter)Action Is

  Jeffrey P. Bigham · 2022 · Communications of the ACM

  This one-page technical perspective accompanies the SoundWatch paper in Communications of the ACM. Bigham uses SoundWatch — a smartwatch prototype that detects audio events and displays descriptions for deaf and hard-of-hearing people — as a lens to explore broader questions…

  wearable technology · deaf and hard of hearing · sound recognition · smartwatch · assistive technology

• Analyzing Deaf and Hard-of-Hearing Users' Behavior, Usage, and Interaction with a Personal Assistant Device that Understands Sign-Language Input

  Abraham Glasser, Matthew Watkins, Kira Hart, Sooyeon Lee, Matt Huenerfauth · 2022 · Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper investigates how Deaf and Hard-of-Hearing (DHH) ASL signers would actually use a personal assistant device (such as an Amazon Echo Show or Google Home) if it could understand American Sign Language commands. The authors position their work against the…

  sign language · american sign language · deaf and hard of hearing · personal assistants · voice assistants

• Remotely Co-Designing Features for Communication Applications using Automatic Captioning with Deaf and Hearing Pairs

  Matthew Seita, Sooyeon Lee, Sarah Andrew, Kristen Shinohara, Matt Huenerfauth · 2022 · Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper addresses two intertwined problems. First, methodologically, how can co-design research involving both Deaf/Hard-of-Hearing (DHH) and hearing participants be conducted remotely during and beyond COVID-19, when in-person sessions are not possible and masks…

  automatic speech recognition · deaf and hard of hearing · participatory design · co-design · videoconferencing

• Watch It, Don't Imagine It: Creating a Better Caption-Occlusion Metric by Collecting More Ecologically Valid Judgments from DHH Viewers

  Akhter Al Amin, Saad Hassan, Sooyeon Lee, Matt Huenerfauth · 2022 · Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper builds a better automated metric for the severity of caption-occlusion — the problem of closed captions blocking important on-screen visual content during television programming. DHH viewers consistently report that even perfectly transcribed captions become…

  captioning · captions · caption occlusion · deaf and hard of hearing · television accessibility

• Methods for Evaluating the Fluency of Automatically Simplified Texts with Deaf and Hard-of-Hearing Adults at Various Literacy Levels

  Oliver Alonzo, Jessica Trussell, Matthew Watkins, Sooyeon Lee, Matt Huenerfauth · 2022 · Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper is a methodological study, not a product evaluation: the authors ask how researchers should measure the fluency of Automatic Text Simplification (ATS) output when the evaluators are Deaf and Hard-of-Hearing (DHH) adults spanning a wide range of English…

  automatic text simplification · deaf and hard of hearing · readability · reading accessibility · natural language processing

• Design and Evaluation of Hybrid Search for American Sign Language to English Dictionaries: Making the Most of Imperfect Sign Recognition

  Saad Hassan, Akhter Al Amin, Alexis Gordon, Sooyeon Lee, Matt Huenerfauth · 2022 · Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI '22)

  This CHI 2022 paper tackles a practical problem faced by the roughly 200,000 students currently studying American Sign Language (ASL) in the US: how do you look up the meaning of a sign you just saw when you don't know its English gloss? Unlike written languages, ASL has no…

  american sign language · sign language · sign language recognition · search interfaces · dictionary

• Understanding Deaf and Hard-of-Hearing Users' Interest in Sign-Language Interaction with Personal-Assistant Devices

  Abraham Glasser, Vaishnavi Mande, Matt Huenerfauth · 2021 · Proceedings of the 18th International Web for All Conference (W4A '21)

  This mixed-methods study investigates Deaf and Hard of Hearing (DHH) users' experiences with and interest in personal-assistant devices (such as Amazon Alexa and Google Home) that could understand American Sign Language (ASL) commands. The research was motivated by the growing…

  deaf and hard of hearing · sign language · personal assistants · voice interface · smart home

• Caption-Occlusion Severity Judgments across Live-Television Genres from Deaf and Hard-of-Hearing Viewers

  Akhter Al Amin, Saad Hassan, Matt Huenerfauth · 2021 · Proceedings of the 18th International Web for All Conference (W4A '21)

  This paper addresses a largely overlooked dimension of caption quality for Deaf and Hard of Hearing (DHH) viewers: the problem of captions occluding important onscreen visual content during live television broadcasts. While existing caption quality metrics focus primarily on…

  deaf and hard of hearing · captions · television accessibility · caption placement · media accessibility

• Effect of Caption Width on the TV User Experience by Deaf and Hard of Hearing Viewers

  Abraham Glasser, Joseline Garcia, Chang Hwang, Christian Vogler, Raja Kushalnagar · 2021 · Proceedings of the 18th International Web for All Conference (W4A)

  This study from Rochester Institute of Technology and Gallaudet University investigates how caption line width affects the viewing experience for Deaf and hard of hearing (DHH) viewers. Captioning technology has not kept pace with the shift from broadcast TV to diverse personal…

  captioning · deaf and hard of hearing · video accessibility · user experience · media accessibility

• Deaf and Hard-of-Hearing Users' Preferences for Hearing Speakers' Behavior during Technology-Mediated In-Person and Remote Conversations

  Matthew Seita, Sarah Andrew, Matt Huenerfauth · 2021 · Proceedings of the 18th International Web for All Conference (W4A)

  This paper presents the first quantitative evidence of Deaf and hard-of-hearing (DHH) individuals' preferences for specific speech and non-verbal behaviors from hearing conversational partners during technology-mediated communication. The researchers conducted two experimental…

  deaf and hard of hearing · automatic speech recognition · videoconferencing · communication accessibility · speechreading

• I See What You're Saying: A Literature Review of Eye Tracking Research in Communication of Deaf or Hard of Hearing Users

  Chanchal Agrawal, Roshan L Peiris · 2021 · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21)

  This paper presents a comprehensive literature review of 55 eye-tracking studies examining the gaze patterns and communication behaviours of deaf or hard-of-hearing (DHH) individuals. The authors systematically searched ACM, IEEE, Google Scholar, and linguistics and psychology…

  deaf and hard of hearing · eye tracking · eye gaze · sign language · lip-reading

• At a Different Pace: Evaluating Whether Users Prefer Timing Parameters in American Sign Language Animations to Differ from Human Signers' Timing

  Sedeeq Al-khazraji, Becca Dingman, Sooyeon Lee, Matt Huenerfauth · 2021 · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21)

  This paper investigates whether deaf and hard-of-hearing (DHH) ASL signers actually prefer sign language animations with timing parameters that match human signers, or whether they prefer some form of exaggeration for additional clarity. Adding ASL animations to websites can…

  American Sign Language · sign language animation · deaf and hard of hearing · timing parameters · avatar

• Lost in Translation: Challenges and Barriers to Sign Language-Accessible User Research

  Amelie Unger, Dieter P. Wallach, Nicole Jochems · 2021 · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21)

  This experience report describes the challenges hearing researchers face when conducting user research with sign language (SL) users, based on the authors' experience running remote focus groups with German Sign Language (GSL) users as part of the AVASAG project — which is…

  deaf and hard of hearing · sign language · user research · focus groups · language accessibility

• American Sign Language Video Anonymization to Support Online Participation of Deaf and Hard of Hearing Users

  Sooyeon Lee, Abraham Glasser, Becca Dingman, Zhaoyang Xia, Dimitris Metaxas, Carol Neidle, Matt Huenerfauth · 2021 · ASSETS '21: The 23rd International ACM SIGACCESS Conference on Computers and Accessibility

  Deaf and Hard of Hearing (DHH) individuals who prefer to communicate in American Sign Language (ASL) face a unique privacy barrier online: because ASL is a visual language conveyed through movements of the face, head, hands, and torso, and because the face carries essential…

  deaf and hard of hearing · sign language · American Sign Language · anonymization · privacy

• Deep Learning Methods for Sign Language Translation

  Tejaswini Ananthanarayana, Priyanshu Srivastava, Akash Chintha, Akhil Santha, Brian Landy, Joseph Panaro, Andre Webster, Nikunj Kotecha, Shagan Sah, Thomastine Sarchet, Raymond Ptucha, Ifeoma Nwogu · 2021 · ACM Transactions on Accessible Computing

  This comprehensive study evaluates deep learning methods for translating sign language video directly to spoken/written text—critically, without requiring the intermediate step of gloss-based recognition (manual sign-for-sign transcription). The researchers systematically…

  sign language · machine translation · deep learning · transformer · neural network

• Deaf and Hard-of-hearing Users Evaluating Designs for Highlighting Key Words in Educational Lecture Videos

  Sushant Kafle, Becca Dingman, Matt Huenerfauth · 2021 · ACM Transactions on Accessible Computing

  This empirical study investigates design preferences for keyword highlighting in video captions among Deaf and Hard of Hearing (DHH) users. While style guides like the APA Publication Manual and Chicago Manual of Style provide recommendations for highlighting text in static…

  deaf and hard of hearing · captions · captioning · video accessibility · text highlighting

• Effect of Sign-recognition Performance on the Usability of Sign-language Dictionary Search

  Saad Hassan, Oliver Alonzo, Abraham Glasser, Matt Huenerfauth · 2021 · ACM Transactions on Accessible Computing

  This paper investigates how the performance of sign-language recognition technology affects user satisfaction with ASL dictionary search systems. Unlike written languages where users can type unfamiliar words to look them up, ASL learners who encounter an unfamiliar sign cannot…

  American Sign Language · ASL · sign language recognition · dictionary search · information retrieval

• Deaf Users' Preferences Among Wake-Up Approaches during Sign-Language Interaction with Personal Assistant Devices

  Vaishnavi Mande, Abraham Glasser, Becca Dingman, Matt Huenerfauth · 2021 · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (CHI EA '21)

  This CHI 2021 Extended Abstract investigates a narrow but previously unexplored question: if future personal-assistant devices (Alexa, Google Assistant, etc.) could recognise sign language, how should Deaf users wake them up? Current wake-up mechanisms — speaking a wake-word…

  deaf and hard of hearing · sign language · personal assistants · voice interface · conversational user interfaces