Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels
Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, Jeffrey P. Bigham · 2021 · CHI Conference on Human Factors in Computing Systems
This paper from Apple introduces Screen Recognition, a system that automatically generates accessibility metadata for mobile apps by analyzing their visual pixels, enabling screen readers to work with apps that lack proper developer-provided accessibility information. The…
mobile accessibility · screen readers · machine learning · object detection · VoiceOver