Exploring Accessibility of Mobile Applications Through User Feedback: Insights from App Reviews in a Systematic Literature Review
Alberto Dumont Alves Oliveira, Marcelo Medeiros Eler · 2024 · Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems (IHC 2024) · doi:10.1145/3702038.3702094
Summary
This systematic literature review examines how user reviews from app stores have been used to study the accessibility of mobile applications. The authors searched five major academic databases (ACM, IEEE, ScienceDirect, Scopus, and Web of Science) and identified 638 papers, which were filtered through a rigorous multi-step process down to 16 key studies published between 2013 and early 2024. The review addresses four research questions: the characteristics and collection strategies of user reviews, the methodologies used to analyze them, the specific accessibility issues users report, and future research challenges. The studies analyzed varied enormously in scale, from 173 reviews of 25 apps to over 179 million reviews across 340 apps. The Google Play Store was the dominant data source, with Android apps receiving the most research attention. The authors identified four main strategies for collecting and analyzing accessibility reviews: automatic extraction using tools, manual inspection by researchers, hybrid approaches combining both, and reuse of existing curated datasets. Most research was predominantly quantitative, with machine learning classifiers like Extra Tree, Random Forest, and Support Vector Classification being used for automated review classification. The commonly referenced accessibility standards included BBC Mobile Accessibility Guidelines, Google Material Design, WCAG 2.1, and GAIA (Guidelines for Accessible Interfaces for people with Autism).
Key findings
The review identified nine categories of accessibility issues reported by users: barriers related to language and regionalism, compatibility and performance problems, customization limitations, color and contrast challenges, font and text issues, layout and navigation difficulties, media and dynamic content control problems, theme and color mode concerns, and zooming feature issues. Theme and color mode was the most commonly reported topic, appearing in 27% of accessibility reviews, followed by zooming features at 20%. Contrast issues were reported in 8% of reviews affecting 33% of apps studied. The research revealed a notable six-year gap in publications between 2013 and 2019, followed by a surge peaking at five papers in 2022. The authors synthesized 31 recommendations for future research, organized into ten areas including mining accessibility reviews, applying AI and machine learning, developing accessibility tools for development environments, and promoting community engagement. A significant finding is that accessibility issues affect not just users with disabilities but also the broader user population, with problems in navigation, reading, and access restrictions impacting all users.
Relevance
This paper provides a valuable roadmap for practitioners interested in leveraging app store reviews as a source of accessibility intelligence. For development teams, it highlights that user reviews are an underutilized but rich source of real-world accessibility feedback that can complement formal testing. The nine-category taxonomy of user-reported issues offers a practical checklist for mobile developers prioritizing accessibility improvements. The finding that themes, color modes, and zooming are the most frequently reported concerns suggests these should be early priorities in any mobile accessibility effort. The research agenda pointing toward AI-assisted review mining and integration of accessibility tools into development environments signals where the field is heading. A limitation is that the review is heavily weighted toward Android and Google Play Store data, with iOS accessibility research notably underrepresented.
Tags: mobile accessibility · app reviews · user feedback · systematic literature review · user reviews · Android · machine learning · accessibility barriers
Standards referenced: WCAG 2.1 · BBC Mobile Accessibility Guidelines · Google Material Design