More than One Step at a Time: Designing Procedural Feedback for Non-visual Makeup Routines
Franklin Mingzhe Li, Akihiko Oharazawa, Chloe Qingyu Zhu, Misty Fan, Daisuke Sato, Chieko Asakawa, Patrick Carrington · 2025 · ASSETS 2025: 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746366
Summary
This paper investigates how people with vision impairments navigate the complex, multi-step process of applying makeup—a domain that is deeply personal and culturally significant yet remains underexplored in assistive technology. The researchers conducted a two-phase study: a contextual inquiry with 15 visually impaired makeup users (5 totally blind, 10 legally blind, all female, ages 23-65) who applied makeup in real-time while verbalizing their thought processes via Zoom, followed by semi-structured interviews with 5 licensed professional makeup artists who reviewed participant videos and responded to participant-raised questions. Existing assistive tools address isolated tasks like color identification or product labeling but fail to support the procedural complexity of makeup routines—coordinating step sequences, managing product placement, blending, achieving symmetry, and assessing the final look without visual confirmation. Participants's makeup goals varied by context: daily routines favored simple, quick products for confidence-building; special occasions involved bolder choices with more sighted assistance; and professional settings demanded polished, self-reliant application. Learning pathways included mirrored side-by-side instruction from makeup artists, trial-and-error experimentation, professional consultations, and translating visual media tutorials into non-visual strategies.
Key findings
Participants developed rich embodied, tactile-first strategies that professional makeup artists validated as best practice rather than workarounds—finger-based application was praised as "the best way to warm up the product" and "already a great professional trick." Key challenges persisted around blending (the hardest quality to assess tactilely), facial symmetry, and obtaining reliable assessment feedback. Current AI tools were found promising but incomplete: apps like BeMyAI and Estée Lauder's voice-enabled assistant often gave generic or inaccurate feedback, couldn't address nuanced aesthetic questions about blending or contouring, and were frequently abandoned. The study produced a taxonomy of 11 feedback need categories grounded in real user questions, with step confirmation (N=10), color and shade (N=7), placement and location (N=6), symmetry and alignment (N=6), and quantity (N=6) being most common. Assessment occurred at three phases: during individual steps (real-time correction), after individual steps (step-level evaluation), and after the full routine (closure and confidence). Participants unanimously wanted conversational, voice-based interfaces with hands-free interaction, emphasizing honest, direct, actionable feedback that balanced emotional sensitivity with truthfulness. Professional makeup artists recommended cream-based products over powders for better tactile control, smaller ergonomic tools for precision, and strategic procedural sequencing to prevent errors.
Relevance
This research redefines accessible beauty from deficit compensation to strategic adaptation and embodied expertise. The finding that tactile-first techniques are professional best practices—not workarounds—challenges ableist assumptions embedded in assistive technology design. For practitioners building AI-powered feedback systems, the taxonomy of feedback needs provides a concrete framework for moving beyond binary validation ("looks good/bad") toward nuanced, context-aware, procedural guidance. The paper's concept of procedural feedback systems—intelligent co-pilots that monitor progress, anticipate challenges, and provide actionable guidance throughout multi-step processes—has implications well beyond makeup, applicable to any complex embodied task like cooking, crafting, or personal grooming. Design principles include supporting fluid step transitions, enabling granular on-demand guidance, responding to user-specific routines, and positioning the system as a collaborative partner rather than an evaluator.
Tags: blind and low vision · makeup accessibility · contextual inquiry · procedural feedback · tactile strategies · beauty accessibility · assistive technology design · self-expression