When GPT-4 judges chatbot answers, its agreement depends on the matchup

Two routes to a human preference signal

In Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, Zheng and colleagues test whether strong language models can evaluate chatbot responses in open-ended exchanges. The study introduces MT-Bench, a fixed set of multi-turn questions with expert ratings, and analyzes Chatbot Arena, where users compare two anonymously presented chatbots. The released benchmark data includes 3,000 expert votes and 30,000 crowd conversations.

GPT-4's judgments agreed with human votes at rates above 80% in the authors' evaluations. They also found that GPT-4 agreement with human raters rose as the win-rate difference between compared models grew. Single-answer grading tracked pairwise and human preferences well in their tests, giving the authors a second evaluation format to examine.

Where agreement thins

The aggregate rate sits beside documented position, verbosity, and self-enhancement biases. The agreement gradient across model matchups means that the overall percentage conceals weaker performance on closer comparisons. The work evaluates helpfulness; its authors say safety is largely outside scope, and their score combines dimensions such as accuracy, relevance, and creativity. For MT-Bench and Arena, the relevant unresolved measurement is how reliably judges separate close contenders on each of those dimensions.

Bibliography

• Zheng, L., Chiang, W., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., Zhang, H., Gonzalez, J. E., & Stoica, I. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. arXiv:2306.05685v4 [cs.CL]. https://arxiv.org/pdf/2306.05685