Two models can share an accuracy and differ in stability
A classifier may change its answer when the task is rephrased, or make the same mistake repeatedly on a group of examples. Federico Errica, Giuseppe Siracusano, Davide Sanvito, and Roberto Bifulco formalize those as different measurements in What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering.
They define sensitivity from the entropy of a model’s predictions across meaning-preserving prompt rewrites, and consistency from how similarly the model treats examples with the same true class. Their demonstration uses Llama 3 70B Instruct, Mixtral 8x7B Instruct, GPT-3.5 Turbo, and GPT-4o on five English classification datasets: TREC, CB, RTE, DBPedia, and WoS. For each task, the authors compare simple, more detailed, and one-shot prompt strategies; an LLM generates the paraphrases.
The measures did not collapse into one score
Across their non-degenerate cases, sensitivity and consistency had a Pearson correlation of −0.07. A low value for one measure therefore did not tell developers what the other would be. Accuracy remains necessary too: consistent predictions can still be wrong, while varied predictions may include correct answers.
This is a demonstration of diagnostic metrics, not an extensive comparison of models. The definitions apply to classification, the paraphrase sampler may introduce its own bias, and estimating the measures costs additional model calls. The paper leaves open how to extend the approach to free-form generation, where correctness and consistency are harder to define.
Bibliography
- Federico Errica, Giuseppe Siracusano, Davide Sanvito, and Roberto Bifulco (2024). What Did I Do Wrong? Quantifying LLMs’ Sensitivity and Consistency to Prompt Engineering. arXiv:2406.12334v4. Full paper (PDF).