Reviews

Search results

• Case Study: In-the-Field Accessibility Information Collection Using Gamification

  Akihiro Miyata, Kazuki Okugawa, Yusaku Murayama, Akihiro Furuta, Keihiro Ochiai, Yuko Murayama · 2023 · Proceedings of the 20th International Web for All Conference (W4A '23)

  This study introduces and evaluates a crowdsourcing platform designed to collect real-world accessibility information for constructing accessibility maps that support people with mobility disabilities. Accessibility maps are critical for safe navigation by wheelchair users and…

  crowdsourcing · gamification · accessible maps · physical accessibility · pedestrian infrastructure

• A Demonstration of RASSAR: Room Accessibility and Safety Scanning in Augmented Reality

  Xia Su, Kaiming Cheng, Han Zhang, Jaewook Lee, Wyatt Olson, Jon E. Froehlich · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This demo paper introduces RASSAR (Room Accessibility and Safety Scanning in Augmented Reality), a mobile AR application that semi-automatically identifies, localizes, and visualizes indoor accessibility and safety issues using iPhone LiDAR sensors and real-time computer vision.…

  augmented reality · computer vision · indoor accessibility · object detection · LiDAR

• Understanding Personalized Accessibility through Teachable AI: Designing and Evaluating Find My Things for People who are Blind or Low Vision

  Cecily Morrison, Martin Grayson, Rita Faia Marques, Daniela Massiceti, Camilla Longden, Linda Wen, Edward Cutrell · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '23)

  This paper from Microsoft Research presents Find My Things, one of the first fully realized end-to-end teachable AI applications for accessibility. The app allows people who are blind or low vision to teach their phone to recognize personal objects — keys, earbuds, lip balm,…

  teachable AI · object recognition · blind and low vision · personalization · few-shot learning

• The Sem-Lex Benchmark: Modeling ASL Signs and their Phonemes

  Lee Kezar, Jesse Thomason, Naomi Caselli, Zed Sehyr, Elana Pontecorvo · 2023 · ASSETS '23: Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility

  This paper introduces the Sem-Lex Benchmark, the largest curated dataset of its kind for American Sign Language (ASL) isolated sign recognition, containing over 84,000 videos of isolated sign productions from 41 deaf ASL signers. The dataset addresses two critical barriers in…

  sign language recognition · American Sign Language · machine learning · phonology · dataset

• What do Blind and Low-Vision People Really Want from Assistive Smart Devices? Comparison of the Literature with a Focus Study

  Bhanuka Gamage, Thanh-Toan Do, Nicholas Seow Chiang Price, Arthur Lowery, Kim Marriott · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '23)

  This paper investigates whether the tasks and devices explored by researchers developing AI-based assistive smart devices actually align with what blind and low-vision (BLV) people want. The authors conducted a three-part study combining a scoping literature review with…

  blind and low vision · assistive technology · smart devices · computer vision · wearable technology

• TouchPilot: Designing a Guidance System that Assists Blind People in Learning Complex 3D Structures

  Xiyue Wang, Seita Kayukawa, Hironobu Takagi, Chieko Asakawa · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2023)

  This paper introduces TouchPilot, a step-by-step guidance system designed to help blind people independently learn complex 3D structures through interactive 3D printed models (I3Ms). While existing I3Ms allow blind users to trigger audio labels by pointing at specific elements,…

  3D printed models · tactile learning · blindness · computer vision · guidance systems

• VisPhoto: Photography for People with Visual Impairments via Post-Production of Omnidirectional Camera Imaging

  Naoki Hirabayashi, Masakazu Iwamura, Zheng Cheng, Kazunori Minatani, Koichi Kise · 2023 · Proceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This paper introduces VisPhoto, a novel photography system that fundamentally reimagines how people with visual impairments (PVI) take photographs. Rather than helping users aim a conventional camera at a target in real-time (the dominant approach in prior work), VisPhoto…

  visual impairment · blindness · photography · computer vision · object detection

• Machine Generation of Audio Description for Blind and Visually Impaired People

  Virgínia P. Campos, Tiago M. U. de Araújo, Guido L. de Souza Filho, Luiz M. G. Gonçalves · 2023 · ACM Transactions on Accessible Computing

  This paper presents an extension to CineAD, a system for automatically generating audio descriptions (AD) for videos. The authors address a critical accessibility gap: most videos, films, and cultural programming lack audio descriptions, leaving blind and visually impaired (BVI)…

  audio description · blind and visually impaired · computer vision · machine learning · video accessibility

• Never-ending Learning of User Interfaces

  Jason Wu, Rebecca Krosnick, Eldon Schoop, Amanda Swearngin, Jeffrey P. Bigham, Jeffrey Nichols · 2023 · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST)

  This paper introduces the Never-ending UI Learner, an automated system that continuously crawls real mobile applications to learn semantic properties of user interfaces. The system addresses a fundamental limitation of current approaches to UI understanding: most machine…

  machine learning · mobile accessibility · UI semantics · automated crawling · tappability

• WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics

  Jason Wu, Siyan Wang, Siman Shen, Yi-Hao Peng, Jeffrey Nichols, Jeffrey P. Bigham · 2023 · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

  This paper introduces WebUI, a large-scale dataset of approximately 400,000 web pages automatically crawled and paired with visual, semantic, and stylistic metadata extracted from the browser engine. The dataset addresses a critical bottleneck in UI understanding research:…

  machine learning · computer vision · UI modeling · web semantics · transfer learning

• AIDE: Automatic and Accessible Image Descriptions for Review Imagery in Online Retail

  Rachana Sreedhar, Nicole Tan, Jingyue Zhang, Kim Jin, Spencer Gregson, Eli Moreta-Feliz, Niveditha Samudrala, Shrenik Sadalgi · 2022 · Proceedings of the 19th International Web for All Conference (W4A)

  This paper from the Wayfair Next team presents AIDE (Automatic Image Description Engine), a multi-modal system that automatically generates alt-text for user-submitted review photos on e-commerce sites. While product images on retail sites sometimes have alt-text, customer…

  alternative text · image description · online shopping · blindness and low vision · computer vision

• FootUI: Designing and Detecting Foot Gestures to Assist People with Upper Body Motor Impairments to Use Smartphones on the Bed

  Xiaozhu Hu, Jiting Wang, Weiwei Gao, Yongquan Hu · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS)

  This paper presents FootUI, a vision-based interaction technique that enables people with upper body motor impairments but sound lower limbs to use smartphones through foot gestures while reclining on a bed. The system uses the phone's front-facing camera, mounted on a phone…

  motor accessibility · foot-based interaction · gesture recognition · smartphone accessibility · computer vision

• AAC with Automated Vocabulary from Photographs: Insights from School and Speech-Language Therapy Settings

  Mauricio Fontana de Vargas, Jiamin Dai, Karyn Moffatt · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)

  This paper presents Click AAC, a prototype mobile application that automatically generates situation-specific communication boards from photographs using computer vision and natural language processing. Traditional symbol-based AAC devices organize vocabulary hierarchically by…

  augmentative and alternative communication · AAC · autism · computer vision · just-in-time vocabulary

• The Future of Urban Accessibility for People with Disabilities: Data Collection, Analytics, Policy, and Tools

  Jon E. Froehlich, Yochai Eisenberg, Maryam Hosseini, Fabio Miranda, Marc Adams, Anat Caspi, Holger Dieterich, Heather Feldner, Aldo Gonzalez, Claudina de Gyves, Joy Hammel, Reuben Kirkham, Melanie Kneisel, Delphine Labbé, Steve J. Mooney, Victor Pineda, Cláudia Fonseca Pinhão, Ana Rodríguez, Manaswi Saha, Michael Saugstad, Judy Shanley, Ather Sharif, Qing Shen, Cláudio Silva, Maarten Sukel, Eric K. Tokuda, Sebastian Felix Zappe, Anna Zivarts · 2022 · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 22)

  This workshop paper frames urban accessibility as a "wicked problem" spanning transportation, urban planning, disability studies, public health, and human geography, and assembles a remarkably diverse team of 28 co-organizers from six countries across academia, government, NGOs,…

  urban accessibility · built environment · pedestrian infrastructure · sidewalk accessibility · data collection

• AIDE: An Automatic Image Description Engine for Review Imagery

  Rachana Sreedhar, Nicole Tan, Jingyue Zhang, Kim Jin, Spencer Gregson, Eli Moreta-Feliz, Niveditha Samudrala, Shrenik Sadalgi · 2022 · Proceedings of the 19th International Web for All Conference (W4A)

  This paper from Wayfair presents AIDE, a multimodal machine learning system that automatically generates contextual alt-text for user-submitted review images in e-commerce — a category of imagery that is particularly inaccessible because it is user-generated, unpredictable in…

  alt text · computer vision · blindness · visual impairment · screen readers

• Improving Mealtime Experiences of People with Visual Impairments

  SeungA Chung, Soobin Park, Sohyeon Park, Kyungyeon Lee, Uran Oh · 2021 · Proceedings of the 18th International Web for All Conference (W4A)

  This paper investigates the mealtime challenges faced by people with visual impairments (PVI) and explores how assistive technology can support more independent dining experiences. While most accessibility research for PVI has focused on navigation and object recognition, this…

  visual impairment · assistive technology · independent living · daily living · computer vision

• Slidecho: Flexible Non-Visual Exploration of Presentation Videos

  Yi-Hao Peng, Jeffrey P Bigham, Amy Pavel · 2021 · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21)

  This paper presents Slidecho, a system that makes recorded presentation videos accessible to blind and visually impaired learners by automatically extracting slide content and synchronizing it with the presenter's speech. The core problem is that most presentation videos —…

  video accessibility · blind and low vision · audio description · presentations · screen reader

• Designing Tools for High-Quality Alt Text Authoring

  Kelly Mack, Edward Cutrell, Bongshin Lee, Meredith Ringel Morris · 2021 · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21)

  This paper investigates how to improve the quality of alternative text through better authoring interfaces and feedback mechanisms for automatic alt text, focusing on Microsoft PowerPoint as the application context. The researchers built and tested four prototype interfaces: two…

  alt text · image accessibility · screen readers · authoring tools · automatic alt text

• Accessing Passersby Proxemic Signals through a Head-Worn Camera: Opportunities and Limitations for the Blind

  Kyungjun Lee, Daisuke Sato, Saki Asakawa, Chieko Asakawa, Hernisa Kacorri · 2021 · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '21)

  This paper explores the potential and limitations of using head-worn cameras with computer vision to help blind people access proxemic signals — the spatial behaviour of nearby people that sighted individuals perceive visually, such as someone's presence, distance, relative…

  blind and low vision · proxemics · wearable camera · smart glasses · pedestrian detection

• Going Beyond One-Size-Fits-All Image Descriptions to Satisfy the Information Wants of People Who are Blind or Have Low Vision

  Abigale Stangl, Nitin Verma, Kenneth R. Fleischmann, Meredith Ringel Morris, Danna Gurari · 2021 · ASSETS '21: The 23rd International ACM SIGACCESS Conference on Computers and Accessibility

  Current image description practices typically produce a single, one-size-fits-all description for each image, yet the same image can appear across vastly different contexts — news websites, e-commerce platforms, social media feeds, travel sites, and personal photo libraries —…

  image description · alternative text · blind · low vision · context-aware

• Deep Learning Methods for Sign Language Translation

  Tejaswini Ananthanarayana, Priyanshu Srivastava, Akash Chintha, Akhil Santha, Brian Landy, Joseph Panaro, Andre Webster, Nikunj Kotecha, Shagan Sah, Thomastine Sarchet, Raymond Ptucha, Ifeoma Nwogu · 2021 · ACM Transactions on Accessible Computing

  This comprehensive study evaluates deep learning methods for translating sign language video directly to spoken/written text—critically, without requiring the intermediate step of gloss-based recognition (manual sign-for-sign transcription). The researchers systematically…

  sign language · machine translation · deep learning · transformer · neural network

• Iterative Design of Sonification Techniques to Support People with Visual Impairments in Obstacle Avoidance

  Giorgio Presti, Dragan Ahmetovic, Mattia Ducci, Cristian Bernareggi, Luca A. Ludovico, Adriano Baratè, Federico Avanzini, Sergio Mascetti · 2021 · ACM Transactions on Accessible Computing

  This paper presents WatchOut, a sonification system that conveys real-time obstacle information to blind or visually impaired (BVI) people through non-verbal audio cues. While white canes detect obstacles only within about 1 meter and miss elevated hazards, computer vision on…

  visual impairment · blindness · sonification · obstacle avoidance · orientation and mobility

• Effect of Sign-recognition Performance on the Usability of Sign-language Dictionary Search

  Saad Hassan, Oliver Alonzo, Abraham Glasser, Matt Huenerfauth · 2021 · ACM Transactions on Accessible Computing

  This paper investigates how the performance of sign-language recognition technology affects user satisfaction with ASL dictionary search systems. Unlike written languages where users can type unfamiliar words to look them up, ASL learners who encounter an unfamiliar sign cannot…

  American Sign Language · ASL · sign language recognition · dictionary search · information retrieval

• Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots

  Jason Wu, Xiaoyi Zhang, Jeff Nichols, Jeffrey P. Bigham · 2021 · The 34th Annual ACM Symposium on User Interface Software and Technology (UIST)

  This paper introduces screen parsing, the task of predicting UI elements and their hierarchical relationships from a screenshot alone. While prior work could detect individual UI elements on a screen (element detection), those approaches produced flat lists of elements with no…

  screen readers · mobile accessibility · computer vision · UI semantics · machine learning

• Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels

  Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, Jeffrey P. Bigham · 2021 · CHI Conference on Human Factors in Computing Systems

  This paper from Apple introduces Screen Recognition, a system that automatically generates accessibility metadata for mobile apps by analyzing their visual pixels, enabling screen readers to work with apps that lack proper developer-provided accessibility information. The…

  mobile accessibility · screen readers · machine learning · object detection · VoiceOver