A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking Instructions
Franklin Mingzhe Li, Ashley Wang, Patrick Carrington, Shaun K. Kane · 2024 · Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2024) · doi:10.1145/3663548.3675662
Summary
This paper investigates how people with vision impairments access and follow cooking recipes, identifying practices, challenges, and design opportunities for more accessible kitchen technologies. The researchers conducted semi-structured interviews with 20 blind cooks (ranging from 24 to 70 years old, with 2 to 58 years of cooking experience) and four adaptive cooking instructors at a vision rehabilitation centre (SAVVI in Arizona). The study also included a recipe review and editing activity where participants gave feedback on actual recipes for dishes they wanted to learn. The research identified three distinct cooking phases with unique accessibility challenges: (1) searching for and editing recipes before cooking, where participants copy recipes into plain text files, manually transcribe audio/video content, and edit for personal preferences; (2) preparing workspace and tools, involving challenges with identifying objects via vision apps, managing measurements with inaccessible measuring tools, and organizing ingredients; and (3) following recipes while cooking, where participants navigate recipes via screen readers or audio players while dealing with dirty/wet hands, device protection needs, and the cognitive burden of memorizing steps to reduce device interactions. A major finding was that recipes are fundamentally designed around visual information — instructions like "cook until golden brown" are meaningless to blind cooks, and images containing critical information about size, colour, and technique have no text alternatives.
Key findings
The study revealed several critical accessibility barriers and user strategies. Participants strongly preferred non-visual alternatives to visual descriptors, favouring texture descriptions ("crispy," "firm"), sound descriptions ("sizzling"), and smell descriptions ("caramelized onion smells sweet") over colour-based doneness indicators. They wanted recipes broken into discrete steps with one action per step rather than combined instructions, with summaries and estimated times at the beginning of each step. Participants desired ingredients listed in order of use or grouped by category, with consistent naming between the ingredients list and instructions. The hand-washing burden was a major pain point — participants must repeatedly wash and dry hands to interact with touchscreen devices, leading many to wrap laptops in cling film or use dedicated audio players with physical buttons. Participants wanted hands-free interaction through voice commands or simplified touch interfaces, and multiple interaction modalities available simultaneously. The study also found important differences between experienced and novice blind cooks: those who cooked before losing vision needed less detail, while those learning to cook non-visually for the first time preferred comprehensive descriptions with links to external resources. Cooking instructors emphasised "Structured Discovery" — encouraging students to solve problems independently rather than through hand-over-hand teaching. The paper proposed creating a non-visual database of common cooking references and substitutions, mapping visual descriptors to non-visual equivalents.
Relevance
This research has direct practical implications for anyone developing cooking applications, recipe websites, or smart kitchen technologies. The findings provide concrete design guidelines: replace visual descriptors with multi-sensory alternatives, break complex steps into atomic actions, provide recipe summaries upfront, support non-linear navigation for parallel cooking tasks, and enable hands-free interaction. The proposed non-visual cooking reference database could transform recipe accessibility at scale, potentially leveraging LLMs to automatically convert visual recipe descriptions into non-visual equivalents. For web developers, the study underscores that recipe websites remain deeply inaccessible — auto-playing videos interfere with screen readers, ads disrupt navigation, and critical information is trapped in images. The finding that participants use NFC tags on spice jars and other kitchen items demonstrates grassroots accessibility innovation worth supporting in commercial products. The instructors' emphasis on customising instruction based on individual cooking experience and vision level is relevant for any adaptive learning system.
Tags: blind · visual impairment · cooking accessibility · recipe access · assistive technology · non-visual interaction · screen readers · hands-free interaction · kitchen accessibility · multimodal interaction · vision rehabilitation