Reviews

Search results

• CARTGPT: Real-Time Correction of CART Captions Using Large Language Models

  Liang-Yuan Wu, Andrea Kleiver, Dhruv Jain · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility

  This paper introduces CARTGPT, a real-time system that enhances Communication Access Realtime Translation (CART) captions by combining human-generated CART transcripts with automatic speech recognition (ASR) output and using GPT-4 to detect and correct transcription errors. CART…

  deaf and hard of hearing · real-time captioning · CART · large language models · automatic speech recognition

• Access on Demand: Real-time, Multi-modal Accessibility for the Deaf and Hard-of-Hearing based on Augmented Reality

  Roshan Mathew, Brian Mak, Wendy Dannels · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This experience report documents two deaf researchers' hands-on evaluation of Access on Demand (AoD), an augmented reality application developed at Rochester Institute of Technology that delivers real-time captioning and American Sign Language (ASL) interpretation through Vuzix…

  augmented reality · deaf and hard of hearing · smart glasses · captioning · sign language interpretation

• Towards Accessible Conversations in a Mobile Context for People who are Deaf and Hard of Hearing

  Dhruv Jain, Rachel Franz, Leah Findlater, Jackson Cannon, Raja Kushalnagar, Jon Froehlich · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '18)

  This paper presents two studies examining the communication needs of deaf and hard of hearing (DHH) people in mobile contexts (walking, transit, recreational activities) and the potential for head-mounted display (HMD) captions to address those needs. Prior research on DHH…

  deaf and hard of hearing · real-time captioning · augmented reality · head-mounted display · mobile accessibility

• Usability Evaluation of Captions for People Who Are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2018 · SIGACCESS Accessibility and Computing Newsletter (Issue 122)

  This is a SIGACCESS Newsletter article summarizing a line of research by Kafle and Huenerfauth on building a caption-quality evaluation metric that actually reflects the experience of Deaf and Hard-of-Hearing (DHH) readers — rather than simply counting speech-recognition errors.…

  automatic speech recognition · captioning · captions · caption quality · accessibility metrics

• Scopist: Building a Skill Ladder into Crowd Transcription

  Jeffrey P. Bigham, Kristin Williams, Nila Banerjee, John Zimmerman · 2017 · Proceedings of the 14th International Web for All Conference (W4A)

  This paper introduces Scopist, a JavaScript application designed to teach crowd workers stenotype — a chording-based text entry method used by professional real-time captioners — while they perform audio transcription microtasks. The research addresses a fundamental problem in…

  crowdsourcing · captioning · stenography · deaf accessibility · transcription

• Evaluating the Usability of Automatically Generated Captions for People who are Deaf or Hard of Hearing

  Sushant Kafle, Matt Huenerfauth · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper addresses a fundamental problem in automatic captioning for Deaf and Hard of Hearing (DHH) users: the standard metric used to evaluate automatic speech recognition (ASR) systems — Word Error Rate (WER) — poorly predicts how usable the resulting captions actually are…

  captioning · automatic speech recognition · deaf and hard of hearing · evaluation methods · natural language processing

• Scribe: Deep Integration of Human and Machine Intelligence to Caption Speech in Real Time

  Walter S. Lasecki, Christopher D. Miller, Iftekhar Naim, Raja Kushalnagar, Adam Sadilek, Daniel Gildea, Jeffrey P. Bigham · 2017 · Communications of the ACM

  Scribe is a system that provides on-demand, real-time captioning of live speech for deaf and hard of hearing (DHH) people by combining groups of non-expert human captionists with machine intelligence. The system addresses a critical accessibility gap: professional CART…

  real-time captioning · deaf and hard of hearing · crowdsourcing · human computation · speech recognition

• Improving Real-Time Captioning Experiences for Deaf and Hard of Hearing Students

  Saba Kawas, George Karalis, Tzu Wen, Richard E. Ladner · 2016 · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16)

  This paper takes a holistic, qualitative approach to understanding deaf and hard of hearing (DHH) university students' experiences with real-time captioning in mainstream classrooms, examining both human-based captioning (CART — Communication Access Realtime Translation) and…

  deaf and hard of hearing · real-time captioning · CART · automatic speech recognition · education

• Evaluation of Real-time Captioning by Machine Recognition with Human Support

  Hironobu Takagi, Takashi Itoh, Kaoru Shinkawa · 2015 · Proceedings of the 12th International Web for All Conference (W4A)

  This paper from IBM Research Tokyo investigates a hybrid approach to real-time captioning that combines Automated Speech Recognition (ASR) with human correction to make workplace meetings accessible for deaf and hard of hearing (DHH) employees. Professional stenography services…

  real-time captioning · deaf and hard of hearing · automated speech recognition · workplace accessibility · Japanese

• Tracked Speech-To-Text Display: Enhancing Accessibility and Readability of Real-Time Speech-To-Text

  Raja S. Kushalnagar, Gary W. Behm, Aaron W. Kelstone, Shareef Ali · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility

  This research addresses a subtle but significant barrier facing deaf and hard of hearing (DHH) students in educational settings: visual dispersion. While hearing students can simultaneously watch lecture visuals (slides, demonstrations, whiteboard) and listen to the speaker's…

  deaf and hard of hearing · speech-to-text · CART · captioning · education

• Helping students keep up with real-time captions by pausing and highlighting

  Walter S. Lasecki, Raja Kushalnagar, Jeffrey P. Bigham · 2014 · Proceedings of the 11th Web for All Conference (W4A)

  This paper addresses a fundamental problem with real-time captioning for deaf and hard of hearing (DHH) students: the mismatch between speaking rates (approximately 170 words per minute) and reading rates, which causes students to fall progressively behind the live content. The…

  deaf and hard of hearing · captioning · real-time captioning · education · inclusive classrooms

• Accessibility Evaluation of Classroom Captions

  Raja S. Kushalnagar, Walter S. Lasecki, Jeffrey P. Bigham · 2014 · ACM Transactions on Accessible Computing

  This paper presents a comprehensive evaluation of real-time captioning approaches for classroom lectures, comparing Communication Access Realtime Translation (CART), Automatic Speech Recognition (ASR), and a novel collaborative captioning system called Legion:Scribe. The authors…

  real-time captioning · deaf and hard of hearing · classroom accessibility · crowdsourcing · eye tracking

• Real-Time Caption Challenge: C-Print

  Michael S. Stinson, Pamela Francis, Lisa B. Elliot, Donna Easton · 2014 · Proceedings of the 16th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS '14)

  This demonstration paper presents C-Print, a typing-based real-time captioning system developed over 25 years by researchers at the National Technical Institute for the Deaf (NTID) at Rochester Institute of Technology. C-Print provides communication access for deaf and hard of…

  deaf and hard of hearing · real-time captioning · communication access · transcription · mobile accessibility

• Real-Time Captioning with the Crowd

  Walter S. Lasecki, Jeffrey P. Bigham · 2014 · Interactions

  This article presents Scribe, a crowdsourced real-time captioning system that allows groups of non-expert typists to collectively produce captions at the speed of natural speech — a task that normally requires highly trained professional stenographers. The authors motivate the…

  real-time captioning · crowdsourcing · deaf and hard of hearing · speech-to-text · human computation

• Legion Scribe: Real-Time Captioning by Non-Experts

  Walter S. Lasecki, Raja Kushalnagar, Jeffrey P. Bigham · 2014 · ACM SIGACCESS Conference on Computers and Accessibility

  This demonstration paper presents Legion:Scribe, a crowd-powered captioning system that enables groups of 3-5 non-expert typists to collectively produce real-time captions with less than 5 seconds of latency. The system addresses the prohibitive cost of professional…

  real-time captioning · crowdsourcing · deaf and hard of hearing · speech-to-text · human computation

• Legion Scribe: Real-Time Captioning by the Non-Experts

  Walter S. Lasecki, Christopher D. Miller, Raja Kushalnagar, Jeffrey P. Bigham · 2013 · Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A)

  This demo paper introduces Legion Scribe, a system that enables real-time captioning of speech by harnessing 3-5 ordinary typists working simultaneously, rather than relying on expensive professional stenographers. Real-time captioning provides text equivalents of spoken…

  captioning · deaf and hard of hearing · crowdsourcing · real-time captioning · communication accessibility

• Warping Time for More Effective Real-Time Crowdsourcing

  Walter S. Lasecki, Christopher D. Miller, Jeffrey P. Bigham · 2013 · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013)

  This paper introduces TimeWarp, a technique that manipulates audio playback speed to improve crowd workers performance on real-time speech captioning. The core problem is that non-expert typists cannot keep up with natural speaking rates of 150-225 words per minute, forcing them…

  real-time captioning · crowdsourcing · deaf and hard of hearing · human computation · speech accessibility

• Real-Time Captioning by Non-Experts with Legion Scribe

  Walter S. Lasecki, Christopher D. Miller, Raja Kushalnagar, Jeffrey P. Bigham · 2013 · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2013)

  This short paper introduces Legion Scribe (Scribe), a system that enables 3-5 non-expert typists to collectively caption speech in real time, achieving accuracy approaching that of a professional stenographer at 20-30% of the cost. The system addresses a critical accessibility…

  real-time captioning · deaf and hard of hearing · crowdsourcing · human computation · assistive technology

• Adaptive Time Windows for Real-Time Crowd Captioning

  Matthew J. Murphy, Christopher D. Miller, Walter S. Lasecki, Jeffrey P. Bigham · 2013 · CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

  This paper addresses a key barrier to real-time captioning access for deaf and hard of hearing people: the high cost of professional stenographers, who can charge up to $200 per hour. Building on the Legion:Scribe system, which demonstrated that groups of non-expert crowd…

  real-time captioning · crowdsourcing · deaf and hard of hearing · assistive technology · human computation

• Crowd Caption Correction (CCC)

  Rebecca Perkins Harrington, Gregg C. Vanderheiden · 2013 · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (Assets '13)

  This short paper presents Crowd Caption Correction (CCC), a feature that allows meeting participants or authorized third parties to correct errors in real-time captions during telecollaboration sessions. Captions are critical for deaf and hard of hearing people to participate in…

  captioning · deaf and hard of hearing · crowdsourcing · telecollaboration · real-time captioning

• A Readability Evaluation of Real-Time Crowd Captions in the Classroom

  Raja S. Kushalnagar, Walter S. Lasecki, Jeffrey P. Bigham · 2012 · Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2012)

  This paper evaluates the readability of real-time captions produced by three different approaches in a higher education classroom setting: professional CART (Communication Access Realtime Translation) captionists, automatic speech recognition (ASR), and a novel crowd captioning…

  real-time captioning · deaf and hard of hearing · crowdsourcing · classroom accessibility · higher education

• Real-Time Captioning by Groups of Non-Experts

  Walter Lasecki, Christopher Miller, Adam Sadilek, Andrew Abumoussa, Donato Borrello, Raja Kushalnagar, Jeffrey Bigham · 2012 · UIST '12: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology

  This paper presents Legion:Scribe, an end-to-end system that enables groups of non-expert typists to collectively produce real-time captions for deaf and hard of hearing (DHH) people, offering a cheaper and more available alternative to professional stenographers (CART, costing…

  real-time captioning · crowdsourcing · deaf and hard of hearing · human computation · text alignment

• Online Quality Control for Real-Time Crowd Captioning

  Walter S. Lasecki, Jeffrey P. Bigham · 2012 · Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2012)

  This paper addresses quality control in Legion:Scribe, a system that provides real-time captioning by having multiple non-expert crowd workers simultaneously type what they hear, then automatically merging their partial transcriptions into a single caption stream. Real-time…

  real-time captioning · crowdsourcing · deaf and hard of hearing · automatic speech recognition · human computation

• ClassInFocus: Enabling Improved Visual Attention Strategies for Deaf and Hard of Hearing Students

  Anna C. Cavender, Jeffrey P. Bigham, Richard E. Ladner · 2009 · Assets '09: Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility

  This paper presents ClassInFocus, a system designed to help deaf and hard of hearing (DHH) students manage the demanding visual attention requirements of modern multi-modal classrooms. DHH students face a unique challenge: while hearing students can simultaneously listen to the…

  deaf and hard of hearing · classroom accessibility · visual attention · notifications · eye tracking

• Speech Recognition in University Classrooms: Liberated Learning Project

  Keith Bain, Sara H. Basson, Mike Wald · 2002 · Proceedings of the Fifth International ACM Conference on Assistive Technologies (Assets 02)

  This paper describes the Liberated Learning Project (LLP), an international applied research initiative studying whether speech recognition technology can successfully convert live university lectures into real-time text displays, serving as an alternative to traditional…

  speech recognition · higher education · deaf accessibility · real-time captioning · universal design

25 results.