Using In-Situ Projection to Support Cognitively Impaired Workers at the Workplace
Markus Funk, Sven Mayer, Albrecht Schmidt · 2015 · ASSETS '15: Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility · doi:10.1145/2700648.2809853
Summary
This research investigates whether projecting assembly instructions directly onto a workspace (in-situ projection) can help cognitively impaired workers complete more complex tasks compared to traditional pictorial instructions. The work addresses a significant barrier to workplace inclusion: sheltered work organizations typically limit cognitively impaired workers to simple tasks averaging only 5.25 steps because more complex assembly requires continuous guidance that pictorial instructions cannot adequately provide. The researchers built an assistive system using a Microsoft Kinect v2 depth camera and projector mounted 1.4 meters above the workspace. The system provides two types of context-sensitive feedback: it highlights which parts box to pick from using a green light, and projects the contour of the picked part at the exact position where it should be placed. The system automatically detects correct assembly by comparing depth data to reference images, advancing to the next instruction only when the current step is completed correctly—providing implicit quality control. The study recruited 15 cognitively impaired workers from a German sheltered work organization, stratified into three Performance Index groups (5-10%, 15-35%, and 40%+, indicating their work capability compared to non-disabled workers). Participants assembled Lego Duplo constructions at five complexity levels (3, 6, 12, 24, and 48 bricks, corresponding to 6-96 working steps) under both in-situ projection and pictorial instruction conditions. The pictorial instructions, displayed on a 28-inch screen, showed which brick to pick, the assembled state after placement, and a red arrow indicating placement position—representing state-of-the-art practice in sheltered work organizations.
Key findings
The in-situ projection system produced dramatic improvements across all metrics. Workers completed tasks up to 1.6 times faster (statistically significant at all complexity levels) and made up to 3 times fewer errors (significant for 12+ brick constructions). Most importantly, while performance with pictorial instructions degraded significantly as task complexity increased, performance with in-situ projection remained essentially constant regardless of complexity. This finding directly challenges the assumption that cognitively impaired workers can only handle simple tasks. For the most complex 48-brick task, workers averaged 7.40 seconds per brick with in-situ projection versus 14.21 seconds with pictorial instructions. Error rates showed even more dramatic differences: 0.02 errors per brick with projection versus 0.26 with pictorial instructions at 48 bricks. The large effect sizes (η² ranging from .289 to .781 for time, .523 to .685 for errors) indicate these are not marginal improvements but fundamental changes in capability. Qualitative feedback was overwhelmingly positive. Participants described the system as "magic light that helps performing the task" and appreciated that it "exactly shows where to put the next brick." Supervisors reported that workers asked to use "the workplace with the lights" for their regular tasks after the study concluded. The system was also exhibited at a German vocational rehabilitation trade fair where over 400 impaired persons tried it, consistently praising its ease of use.
Relevance
This research has significant implications for workplace inclusion of people with cognitive disabilities. The finding that in-situ projection maintains constant performance across complexity levels suggests that the traditional practice of limiting cognitively impaired workers to simple tasks may be a limitation of instruction methods rather than worker capability. With appropriate assistive technology, these workers could potentially take on more complex, higher-value assembly work. For practitioners designing accessible workplaces, the key insight is that reducing the cognitive load of matching instructions to physical locations—by projecting guidance directly where actions should occur—eliminates a major barrier. The system also provides automatic quality control through depth sensing, reducing supervisor burden while ensuring correct assembly. Limitations include the need for workers to remove their hands from assembled parts for detection, Kinect sensor warm-up requirements (45 minutes), and the abstract nature of Lego Duplo as a proxy for real assembly tasks. The researchers plan long-term studies to assess whether benefits persist over months of use. The technology has clear potential for extension to assisted living contexts beyond the workplace.
Tags: cognitive disability · workplace accessibility · augmented reality · in-situ projection · sheltered work · assistive technology · industrial accessibility · employment · assembly tasks
Standards referenced: UN Convention on the Rights of Persons with Disabilities