The buyer's first offer can decide the outcome

A buyer who opens just below a spending limit has little room left to negotiate. Tian Xia, Zhiwei He, Tong Ren, Yibo Miao, Zhuosheng Zhang, Yang Yang, and Rui Wang study that asymmetry in Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method. Their benchmark puts a language model in a buyer or seller role and measures profit, valid agreements, and deals across simulated price negotiations.

The authors assembled product information and historical prices from Amazon. For each of 930 products, a model received the listing information plus a private budget or cost, then exchanged structured offers and dialogue with ChatGPT in the opposing role. The paper treats the lowest observed historical price as a proxy for seller cost and uses a budget set to 80% of the historical high price. These are benchmark values, not verified transaction costs.

A role reversal in performance

Every tested buyer model had negative aggregate normalized profit: when it made deals, its offers often left it paying too much relative to its budget. GPT-4 lost the least among buyers. Sellers, by contrast, generally earned positive normalized profit, and GPT-4 was the strongest seller in the reported comparison. The authors therefore find that buying is harder than selling in this setup; simply increasing model size did not reliably repair buyer performance.

They also propose OG-Narrator, which separates choosing an offer price from phrasing that offer in natural language. That intervention improved buyer deal rates in the experiments, suggesting that offer selection and conversational narration need not be delegated to the same model behavior.

A benchmark, not a marketplace

The task uses one product per negotiation, historical prices as proxies for costs, fixed prompting, and older model versions. ChatGPT is the common opponent, and the paper's modeled negotiations are not a measure of live consumer bargaining or actual profits. The useful next check is whether the buyer disadvantage persists with independently varied counterparties and real, verified reservation prices.

Bibliography

  • Tian Xia, Zhiwei He, Tong Ren, Yibo Miao, Zhuosheng Zhang, Yang Yang, and Rui Wang (2024). Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method. arXiv:2402.15813v3 [cs.CL]. Full paper (PDF).