Iterative Crowdsourcing

A human computation workflow in which multiple rounds of crowd workers build iteratively upon each other's responses to collectively achieve higher quality results than any individual worker could produce alone. In each iteration, workers are shown the previous round's outputs alongside the original task, allowing them to identify correct elements, reject errors, and refine the collective response. This approach has been applied to tasks like deciphering poor handwriting, transcribing difficult audio, and translating low-resource languages. For accessibility, iterative crowdsourcing has been explored as a method for transcribing deaf speech, where individual workers produce transcriptions with high error rates but groups can collectively reduce word error rates by over 50% through iterative refinement. A key limitation is the "priming" effect, where workers may converge on an incorrect transcription from earlier rounds rather than generating a fresh interpretation.