Perplexity

A standard metric for evaluating language models that measures how well the model predicts a sample of text. Mathematically, perplexity is the inverse probability of the test set, normalised by the number of words — a lower perplexity indicates that the model assigns higher probability to the actual text and is therefore a better predictor. In accessibility contexts, perplexity is used to evaluate language models that underlie word prediction systems for AAC devices. However, perplexity does not directly translate to practical metrics like keystroke savings because it weighs all words equally, while keystroke savings depends on how many characters are saved by correctly predicting common words. A model with lower perplexity may not always produce higher keystroke savings, making it important to evaluate AAC prediction systems using task-relevant metrics rather than relying solely on perplexity.