Reviews

Search results

• Towards More Robust Speech Interactions for Deaf and Hard of Hearing Users

  Raymond Fok, Harmanpreet Kaur, Skanda Palani, Martez E. Mott, Walter S. Lasecki · 2018 · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2018)

  This University of Michigan study addresses a largely overlooked accessibility gap: while much research has focused on providing deaf users access to spoken output (via captioning or sign language), almost no work has addressed improving deaf users' ability to provide speech…

  deaf and hard of hearing · automatic speech recognition · deaf speech · crowdsourcing · speech intelligibility

• Crowd-AI Camera Sensing in the Real World

  Anhong Guo, Anuraag Jain, Shomiron Ghose, Gierad Laput, Chris Harrison, Jeffrey P. Bigham · 2018 · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

  This paper presents Zensors++, a hybrid crowd-AI camera sensing system that allows users to point a networked camera at a scene, define a natural language question about it (such as "Is the coffee machine in use?" or "How many people are in the room?"), and receive continuous,…

  crowdsourcing · computer vision · human computation · machine learning · smart environments

• In-context Q&A to Support Blind People Using Smartphones

  André Rodrigues, Kyle Montague, Hugo Nicolau, João Guerreiro, Tiago Guerreiro · 2017 · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '17)

  This paper addresses a pervasive but understudied problem: the daily challenges blind people face when using smartphone applications go far beyond the touchscreen gesture difficulties that most accessibility research focuses on. The researchers conducted workshops with 42 blind…

  blindness · smartphone accessibility · screen readers · crowdsourcing · human computation

• Scribe: Deep Integration of Human and Machine Intelligence to Caption Speech in Real Time

  Walter S. Lasecki, Christopher D. Miller, Iftekhar Naim, Raja Kushalnagar, Adam Sadilek, Daniel Gildea, Jeffrey P. Bigham · 2017 · Communications of the ACM

  Scribe is a system that provides on-demand, real-time captioning of live speech for deaf and hard of hearing (DHH) people by combining groups of non-expert human captionists with machine intelligence. The system addresses a critical accessibility gap: professional CART…

  real-time captioning · deaf and hard of hearing · crowdsourcing · human computation · speech recognition

• WearMail: On-the-Go Access to Information in Your Email with a Privacy-Preserving Human Computation Workflow

  Saiganesh Swaminathan, Raymond Fok, Fanglin Chen, Ting-Hao (Kenneth) Huang, Irene Lin, Rohan Jadvani, Walter S. Lasecki, Jeffrey P. Bigham · 2017 · Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST 2017)

  WearMail is a conversational system that extracts specific information from a user's email via voice queries on wearable devices (such as smartwatches), using a novel privacy-preserving human computation workflow. The system addresses the challenge that email functions as…

  crowdsourcing · human computation · privacy · wearable technology · information extraction

• The Effects of Automatic Speech Recognition Quality on Human Transcription Latency

  Yashesh Gaur, Walter S. Lasecki, Florian Metze, Jeffrey P. Bigham · 2016 · Proceedings of the 13th International Web for All Conference (W4A)

  This paper from Carnegie Mellon University and the University of Michigan empirically investigates when automatic speech recognition (ASR) output helps or hinders human transcriptionists producing captions for deaf and hard of hearing people. Manual transcription remains…

  speech recognition · captioning · deaf and hard of hearing · crowdsourcing · human computation

• WearWrite: Crowd-Assisted Writing from Smartwatches

  Michael Nebeling, Alexandra To, Anhong Guo, Adrian A. de Freitas, Jaime Teevan, Steven P. Dow, Jeffrey P. Bigham · 2016 · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI 2016)

  WearWrite is a system that enables users to write documents from their smartwatches by leveraging crowd workers to translate ideas into text. The system addresses the fundamental limitation that smartwatches have severely constrained input/output — touch-based text input is…

  crowdsourcing · wearable technology · human computation · collaborative writing · mobile accessibility

• The Effects of Automatic Speech Recognition Quality on Human Transcription Latency

  Yashesh Gaur · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility

  This paper investigates a practical question for accessibility: when does providing automatic speech recognition (ASR) output help human captionists work faster, and when does it slow them down? Converting speech to text is fundamental for making audio content accessible to deaf…

  deaf · hard of hearing · automatic speech recognition · ASR · captioning

• Guiding Novice Web Workers in Making Image Descriptions Using Templates

  Valerie S. Morash, Yue-Ting Siu, Joshua A. Miele, Lucia Hasty, Steven Landau · 2015 · ACM Transactions on Accessible Computing (TACCESS)

  This study compares two approaches for using non-expert crowdworkers to create accessible descriptions of STEM images (charts, graphs, diagrams) for people who are blind or have print-reading disabilities. The researchers tested Free-Response Image Description (FRID), where…

  image description · alt text · crowdsourcing · human computation · STEM accessibility

• Zensors: Adaptive, Rapidly Deployable, Human-Intelligent Sensor Feeds

  Gierad Laput, Walter S. Lasecki, Jason Wiese, Robert Xiao, Jeffrey P. Bigham, Chris Harrison · 2015 · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI 2015)

  Zensors presents a novel sensing platform that combines real-time human intelligence from online crowd workers with machine learning to create adaptive, rapidly deployable intelligent sensors. The system addresses a fundamental gap in smart environment technology: traditional…

  crowdsourcing · human computation · computer vision · smart environments · machine learning

• Introducing Game Elements in Crowdsourced Video Captioning by Non-Experts

  Hernisa Kacorri, Kaoru Shinkawa, Shin Saito · 2014 · Proceedings of the 11th Web for All Conference (W4A)

  This paper from CUNY Graduate Center and IBM Research Tokyo presents a gamified crowdsourcing platform for video captioning that combines ASR output with non-expert human transcription to improve caption accuracy without monetary rewards. The system builds on the Collaborative…

  captioning · crowdsourcing · deaf and hard of hearing · gamification · automatic speech recognition

• Real-Time Captioning with the Crowd

  Walter S. Lasecki, Jeffrey P. Bigham · 2014 · Interactions

  This article presents Scribe, a crowdsourced real-time captioning system that allows groups of non-expert typists to collectively produce captions at the speed of natural speech — a task that normally requires highly trained professional stenographers. The authors motivate the…

  real-time captioning · crowdsourcing · deaf and hard of hearing · speech-to-text · human computation

• Legion Scribe: Real-Time Captioning by Non-Experts

  Walter S. Lasecki, Raja Kushalnagar, Jeffrey P. Bigham · 2014 · ACM SIGACCESS Conference on Computers and Accessibility

  This demonstration paper presents Legion:Scribe, a crowd-powered captioning system that enables groups of 3-5 non-expert typists to collectively produce real-time captions with less than 5 seconds of latency. The system addresses the prohibitive cost of professional…

  real-time captioning · crowdsourcing · deaf and hard of hearing · speech-to-text · human computation

• Legion Scribe: Real-Time Captioning by the Non-Experts

  Walter S. Lasecki, Christopher D. Miller, Raja Kushalnagar, Jeffrey P. Bigham · 2013 · Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility (W4A)

  This demo paper introduces Legion Scribe, a system that enables real-time captioning of speech by harnessing 3-5 ordinary typists working simultaneously, rather than relying on expensive professional stenographers. Real-time captioning provides text equivalents of spoken…

  captioning · deaf and hard of hearing · crowdsourcing · real-time captioning · communication accessibility

• Warping Time for More Effective Real-Time Crowdsourcing

  Walter S. Lasecki, Christopher D. Miller, Jeffrey P. Bigham · 2013 · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013)

  This paper introduces TimeWarp, a technique that manipulates audio playback speed to improve crowd workers performance on real-time speech captioning. The core problem is that non-expert typists cannot keep up with natural speaking rates of 150-225 words per minute, forcing them…

  real-time captioning · crowdsourcing · deaf and hard of hearing · human computation · speech accessibility

• Real-Time Captioning by Non-Experts with Legion Scribe

  Walter S. Lasecki, Christopher D. Miller, Raja Kushalnagar, Jeffrey P. Bigham · 2013 · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2013)

  This short paper introduces Legion Scribe (Scribe), a system that enables 3-5 non-expert typists to collectively caption speech in real time, achieving accuracy approaching that of a professional stenographer at 20-30% of the cost. The system addresses a critical accessibility…

  real-time captioning · deaf and hard of hearing · crowdsourcing · human computation · assistive technology

• Real-Time Crowd Labeling for Deployable Activity Recognition

  Walter S. Lasecki, Young Chol Song, Henry Kautz, Jeffrey P. Bigham · 2013 · Proceedings of the 2013 Conference on Computer Supported Cooperative Work (CSCW 2013)

  Legion:AR is a system that provides deployable activity recognition by combining real-time crowd labeling with automatic recognition using Hidden Markov Models (HMMs). The system addresses a critical limitation of current activity recognition: automated systems must be trained…

  activity recognition · crowdsourcing · human computation · machine learning · aging in place

• Answering Visual Questions with Conversational Crowd Assistants

  Walter S. Lasecki, Phyo Thiha, Yu Zhong, Erin Brady, Jeffrey P. Bigham · 2013 · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2013)

  This paper introduces Chorus:View, a system that enables blind users to get visual questions answered through continuous conversational interaction with multiple crowd workers viewing a live video stream from the user's mobile device. The system addresses key limitations of…

  blind and low vision · crowdsourcing · human computation · assistive technology · visual assistance

• Adaptive Time Windows for Real-Time Crowd Captioning

  Matthew J. Murphy, Christopher D. Miller, Walter S. Lasecki, Jeffrey P. Bigham · 2013 · CHI EA '13: CHI '13 Extended Abstracts on Human Factors in Computing Systems

  This paper addresses a key barrier to real-time captioning access for deaf and hard of hearing people: the high cost of professional stenographers, who can charge up to $200 per hour. Building on the Legion:Scribe system, which demonstrated that groups of non-expert crowd…

  real-time captioning · crowdsourcing · deaf and hard of hearing · assistive technology · human computation

• Chorus: A Crowd-Powered Conversational Assistant

  Walter S. Lasecki, Rachel Wesley, Jeffrey Nichols, Anand Kulkarni, James F. Allen, Jeffrey P. Bigham · 2013 · UIST '13: Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology

  This paper presents Chorus, a crowd-powered conversational assistant that enables users to hold natural, continuous conversations with what appears to be a single partner, but is actually backed by multiple crowd workers operating in real-time. The system addresses a fundamental…

  crowdsourcing · conversational assistants · human computation · dialog systems · real-time systems

• Using Real-time Feedback to Improve Visual Question Answering

  Yu Zhong, Phyo Thiha, Grant He, Walter Lasecki, Jeffrey Bigham · 2012 · CHI EA '12: CHI '12 Extended Abstracts on Human Factors in Computing Systems

  This work-in-progress paper introduces Legion:View, a system that extends the VizWiz model of crowd-powered visual question answering by adding a real-time feedback loop between blind users and crowd workers. The original VizWiz allowed blind users to take a still photograph,…

  visual question answering · crowdsourcing · blind users · real-time systems · assistive technology

• Real-Time Captioning by Groups of Non-Experts

  Walter Lasecki, Christopher Miller, Adam Sadilek, Andrew Abumoussa, Donato Borrello, Raja Kushalnagar, Jeffrey Bigham · 2012 · UIST '12: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology

  This paper presents Legion:Scribe, an end-to-end system that enables groups of non-expert typists to collectively produce real-time captions for deaf and hard of hearing (DHH) people, offering a cheaper and more available alternative to professional stenographers (CART, costing…

  real-time captioning · crowdsourcing · deaf and hard of hearing · human computation · text alignment

• Online Quality Control for Real-Time Crowd Captioning

  Walter S. Lasecki, Jeffrey P. Bigham · 2012 · Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2012)

  This paper addresses quality control in Legion:Scribe, a system that provides real-time captioning by having multiple non-expert crowd workers simultaneously type what they hear, then automatically merging their partial transcriptions into a single caption stream. Real-time…

  real-time captioning · crowdsourcing · deaf and hard of hearing · automatic speech recognition · human computation

• The Design of Human-Powered Access Technology

  Jeffrey P. Bigham, Richard E. Ladner, Yevgen Borodin · 2011 · Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2011)

  This paper presents a comprehensive framework of 13 design dimensions for evaluating and comparing human-powered access technology — systems that use human assistance, facilitated by technology, to overcome accessibility barriers that automation alone cannot solve. The authors…

  human computation · crowdsourcing · assistive technology · accessibility framework · design principles

• VizWiz: Nearly Real-Time Answers to Visual Questions

  Jeffrey P. Bigham, Chandrika Jayant, Hanjie Ji, Greg Little, Andrew Miller, Robert C. Miller, Aubrey Tatarowicz, Brandyn White, Samuel White, Tom Yeh · 2010 · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A)

  This paper introduces VizWiz, a pioneering mobile application that enables blind and low-vision people to get nearly real-time answers to visual questions by connecting their smartphone cameras to remote paid workers on Amazon Mechanical Turk. Users take a photo with their…

  blind and low vision · crowdsourcing · assistive technology · mobile accessibility · human computation