The explanation condition changed the choices

With a chain-of-thought conjoint prompt, GPT-4 chose the larger-later reward 34.5% of the time; in standard conjoint, the rates were 22% for GPT-3.5 and 16% for GPT-4. Ali Goli and Amandeep Singh report these results in Can LLMs Capture Human Preferences?, a multilingual conjoint experiment comparing smaller-sooner with larger-later rewards across 22 languages, delay conditions, and interest rates.

The authors compare model choices with established human intertemporal-choice findings. GPT-3.5 showed a lexicographic preference for sooner rewards, and GPT-4's estimated discount rates remained much higher than the human benchmarks. Language rankings also shifted with elicitation method; the authors note that asking for an explanation may prime the response.

Carry the prompt with the estimate

This experiment measures generated choices in one preference domain. Translation and prompt framing are part of its setup. The authors propose chain-of-thought conjoint as a way to generate hypotheses about factors behind preference differences. Any estimate from this task should travel with its model, language, and elicitation condition: changing the prompt changed the result.

Bibliography