衡量LLM的议价能力：一个基准与一种买方增强方法
Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method
AI总结：
本文将议价形式化为非对称不完全信息博弈，构建了AmazonHistoryPrice数据集评估LLM智能体，发现买方角色难度更高且增大模型规模无效，提出OG-Narrator方法显著提升买方交易率和利润。
摘要：
议价是人类谈判中重要且独特的一部分。随着LLM驱动的智能体学会像真实人类一样进行谈判和行动，如何评估智能体的议价能力仍是一个开放问题。我们首次将议价任务正式描述为非对称不完全信息博弈，定义了买方和卖方在多个议价过程中的收益。这使我们能够定量评估智能体在议价任务中的表现。我们收集了一个真实的产品价格数据集AmazonHistoryPrice，并对各种LLM智能体的议价能力进行了评估。我们发现，扮演买方比卖方困难得多，并且增加模型规模并不能有效提升买方的表现。为了应对这一挑战，我们提出了一种名为OG-Narrator的新方法，该方法集成了一个确定性的Offer Generator来控制买方报价的价格范围，以及一个LLM Narrator为生成的报价创建自然语言句子。实验结果表明，OG-Narrator将买方的交易率从26.67%提高到88.88%，并在所有基线上带来了十倍的利润增长，即使是未对齐的模型也是如此。
英文摘要：
Bargaining is an important and unique part of negotiation between humans. As LLM-driven agents learn to negotiate and act like real humans, how to evaluate agents' bargaining abilities remains an open problem. For the first time, we formally described the Bargaining task as an asymmetric incomplete information game, defining the gains of the Buyer and Seller in multiple bargaining processes. It allows us to quantitatively assess an agent's performance in the Bargain task. We collected a real product price dataset, AmazonHistoryPrice, and conducted evaluations of various LLM agents' bargaining abilities. We find that playing a Buyer is much harder than a Seller, and increasing model size can not effectively improve the Buyer's performance. To address the challenge, we propose a novel approach called OG-Narrator that integrates a deterministic Offer Generator to control the price range of Buyer's offers, and an LLM Narrator to create natural language sentences for generated offers. Experimental results show that OG-Narrator improves the buyer's deal rates from 26.67% to 88.88% and brings a ten times multiplication of profits on all baselines, even a model that has not been aligned.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
