The test asks whether a rule survives rewording

A moral prompt rarely has a universally accepted answer key, so a model's answer can be difficult to score as simply right or wrong. Vamshi Krishna Bonagiri, Sreeram Vennam, Priyanshul Govil, Ponnurangam Kumaraguru, and Manas Gaur approach a different question in SaGE: Evaluating Moral Consistency in Large Language Models: does the model apply a similar rule when the scenario is paraphrased?

Their Moral Consistency Corpus starts with 10,000 moral questions, expands them with five paraphrases, and collects answers and generated “Rules of Thumb” (RoTs) from 11 language models. The authors embed those short rules, build a semantic graph, and define Semantic Graph Entropy (SaGE) to summarize how consistently a model invokes similar principles across the paraphrases. Higher SaGE indicates more consistency under this particular measure.

Consistency and accuracy separate

Among the tested models, the highest reported answer-based SaGE score was 0.681; GPT-3.5 Turbo scored higher than GPT-4 on this corpus. The team also reports that temperature sampling made little difference to consistency across paraphrases. On separate TruthfulQA and HellaSwag evaluations, their consistency metric did not track task accuracy. Adding a specified human-annotated rule to a prompt raised consistency on a selected sample, a result that tests a simple intervention rather than a general remedy.

SaGE does not decide which rule is morally correct. It measures agreement between rules inferred from answers to similar wording, which is why its result is descriptive rather than a moral verdict.

The measurement pipeline has its own assumptions

The dataset covers 11 models and five generated paraphrases per question. The pipeline relies on other language models to create paraphrases and rules, as well as on a sentence-embedding model to compare those rules. The authors acknowledge these dependencies and the limited sample. If any stage distorts a scenario or its rule, the resulting consistency score can reflect that error too.

Bibliography

  • Vamshi Krishna Bonagiri, Sreeram Vennam, Priyanshul Govil, Ponnurangam Kumaraguru, and Manas Gaur (2024). SaGE: Evaluating Moral Consistency in Large Language Models. arXiv:2402.13709v2 [cs.CL]. Full paper (PDF).