PieArena：在真实谈判场景中对语言智能体进行排名与画像
PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios
AI总结：
本文提出PieArena基准，通过多智能体交互评估LLM的谈判能力，并开发排名模型与行为画像，发现联合意图框架对中低端模型提升显著，前沿模型（如GPT-5）在谈判中达到或超过人类基线。
摘要：
我们深入评估了LLM的谈判能力，这是一项需要战略推理、心理理论和经济价值创造的核心商业任务。为此，我们引入了PieArena，一个基于精英商学院MBA谈判课程中真实场景的多智能体交互的大规模谈判基准。我们在三种配对模式下评估语言智能体：镜像博弈、交叉博弈和人与语言模型博弈。我们开发了一个用于连续谈判收益的排名模型，该模型生成顺序不变、不确定性量化的排行榜，同时纠正系统性的实验不对称性。我们进一步研究了联合意图智能体框架的效果，发现其收益不对称：对中低端语言模型有大幅提升，而对前沿语言模型的边际收益递减。作为校准锚点，我们收集了受过训练的商学院学生之间以及学生与语言模型之间的谈判数据，发现代表性前沿语言智能体（GPT-5）在我们的评估设置中达到或超过了这一人类基线。除了交易结果，PieArena还提供了多维行为画像，揭示了指令遵从性、计算准确性以及法官评估的欺骗性和声誉方面的跨模型异质性，说明了超越仅基于结果的排行榜的评估价值。
英文摘要：
We present an in-depth evaluation of LLMs' ability to negotiate, a central business task requiring strategic reasoning, theory of mind, and economic value creation. To do so, we introduce PieArena, a large-scale negotiation benchmark grounded in multi-agent interactions over realistic scenarios adapted from MBA negotiation courses at an elite business school. We evaluate language agents across three pairing regimes: mirror-play, cross-play, and human-LM play. We develop a ranking model for continuous negotiation payoffs that yields order-invariant, uncertainty-quantified leaderboards while correcting for systematic experimental asymmetries. We further study the effects of joint-intentionality agentic scaffolding and find asymmetric gains, with large improvements for mid- and lower-tier LMs and diminishing returns for frontier LMs. As calibration anchors, we collect human-human and human-LM negotiation data from trained business school students, finding that a representative frontier language agent (GPT-5) matches or exceeds this human baseline in our evaluation settings. Beyond deal outcomes, PieArena provides a multi-dimensional behavioral profile that reveals cross-model heterogeneity in instruction compliance, computation accuracy, as well as judge-assessed deception and reputation, illustrating the value of evaluation beyond outcome-only leaderboards.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
