Never-ending Learning of User Interfaces
Jason Wu, Rebecca Krosnick, Eldon Schoop, Amanda Swearngin, Jeffrey P. Bigham, Jeffrey Nichols · 2023 · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST)
This paper introduces the Never-ending UI Learner, an automated system that continuously crawls real mobile applications to learn semantic properties of user interfaces. The system addresses a fundamental limitation of current approaches to UI understanding: most machine…
machine learning · mobile accessibility · UI semantics · automated crawling · tappability