User Interaction with Word Prediction: The Effects of Prediction Quality
Keith Trnka, John McCaw, Debra Yarrington, Kathleen F. McCoy, Christopher Pennington · 2009 · ACM Transactions on Accessible Computing (TACCESS) · doi:10.1145/1497302.1497307
Summary
This study investigates how the quality of word prediction systems affects both communication speed and user behavior in augmentative and alternative communication (AAC). The researchers conducted a controlled experiment with 33 adult participants who used a simulated AAC interface with an artificially imposed 1.5-second delay per keystroke, mimicking the input rate of users who rely on alternative access methods like switches or head pointers. Participants completed conversational phrase transcription tasks under three conditions: no prediction, basic prediction using a simple recency-based model, and advanced prediction using a trigram language model with backoff to lower-order n-grams. The research addresses a critical gap in AAC evaluation. Prior studies often measured only keystroke savings—the percentage of keystrokes eliminated by accepting predictions—without examining whether users actually utilize predictions or how prediction quality affects real-world communication rates. The authors developed a comprehensive mathematical model relating keystroke savings, prediction utilization, and the time cost of scanning prediction lists to calculate theoretical speedup factors. This framework allows researchers and developers to predict performance improvements based on measurable system characteristics.
Key findings
The advanced trigram prediction system yielded dramatically better results than basic prediction. Users achieved 8.40 words per minute with advanced prediction versus 5.73 wpm with basic and 5.21 wpm without prediction—a 58.6% improvement over no prediction and 45.8% improvement over basic prediction. Task completion times showed similar patterns: 20:54 with advanced, 29:36 with basic, and 32:24 without prediction. Critically, prediction utilization—the percentage of opportunities where users accepted predictions rather than continuing to type—was significantly higher with the advanced system (94.4%) compared to basic prediction (79.9%). This demonstrates that users develop trust-based relationships with prediction systems; when predictions are consistently accurate, users rely on them more heavily. The basic system's lower-quality suggestions actually trained users to ignore the prediction list, reducing its effectiveness. Post-task questionnaires revealed that participants reported less fatigue with advanced prediction and rated its suggestions as more helpful. The mathematical speedup model was validated against experimental results, providing a tool for estimating real-world performance gains from keystroke savings metrics.
Relevance
This research has direct implications for AAC device selection and development. The finding that prediction quality affects user behavior—not just objective keystroke savings—means that poorly-performing prediction systems may actually harm communication by training users to ignore helpful features. Practitioners should prioritize high-quality language models over simpler alternatives, even if the simpler systems show reasonable keystroke savings in automated testing. The mathematical framework provided enables clinicians and developers to estimate realistic communication rate improvements when evaluating AAC systems. The study also highlights that laboratory studies using unconstrained typing speeds may not reflect the experience of actual AAC users, whose slower input rates amplify both the benefits of good prediction and the costs of scanning unhelpful suggestions. For organizations developing or selecting AAC technology, investing in sophisticated prediction algorithms yields compounding returns through increased user trust and utilization.
Tags: AAC · word prediction · language models · communication rate · user study · assistive technology