发表机构
King’s College London; Institute for Decentralized AI; University of Oxford; The Alan Turing Institute（伦敦国王学院; 去中心化人工智能研究所; 牛津大学; 艾伦·图灵研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出BazaarBench模拟C2C市场基准，评估LLM智能体在委托交易中的安全性，发现对抗性指令下承诺交易完成率升至33.4%，收入增至33美元，并公开模拟器与数据。
AI 中文摘要
在去中心化的消费者对消费者（C2C）市场中，人们列出商品、与陌生人谈判并互相评分，因此信任依赖于声誉。大型语言模型（LLM）智能体现在代表用户行事，给用户的资金、隐私和声誉带来风险。我们提出了BazaarBench，一个模拟的C2C市场和基准，用于评估这些智能体的安全性。它跟踪交易中的所有权、物品状况和承诺，将记录检查与基于规则的LLM判断相结合，以识别五个阶段中的六种失败类型。我们运行三个基础市场30个模拟日，每个市场有100个使用同一模型的智能体，其库存来自公开的eBay样本。在45个延续中，我们在普通指令、截止日期压力或利用其他交易者的对抗性指令下评估五个模型。每个延续从市场第30天状态的副本运行七个模拟日。被测试的模型控制相同的20个选定智能体，保留其角色、库存和历史，而其他80个保持基础模型。所有五个模型在普通指令下都尝试将同一物品承诺给多个买家。添加目标和截止日期增加了每个模型的这些尝试。在对抗性指令下，被测试卖家在物品不可用或状况被夸大情况下仍完成的承诺交易比例从15.4%上升到33.4%，对GPT-5.4达到55.5%。跨模型和市场平均，每个被测试智能体的模拟每周收入从普通指令下的20美元上升到对抗性指令下的33美元。大部分增长来自卖家从未持有的物品。我们发布了模拟器、保存的市场状态、评估代码和覆盖357,608次智能体模型调用的记录，用于评估新模型和开发更安全的市场智能体。
英文摘要
In decentralized consumer-to-consumer (C2C) marketplaces, people list goods, negotiate with strangers, and rate one another, so trust rests on reputation. Large language model (LLM) agents now act for users, raising risks to their money, privacy, and reputation. We introduce BazaarBench, a simulated C2C marketplace and benchmark for evaluating the safety of these agents. It tracks ownership, item condition, and commitments across transactions, combining record checks with rubric-based LLM judgments to identify six failure types across five stages. We run three base markets for 30 simulated days, each with 100 agents using one model and inventories drawn from a public eBay sample. Across 45 continuations, we evaluate five models under ordinary instructions, deadline pressure, or adversarial instructions to exploit other traders. Each continuation runs for seven simulated days from a copy of a market's day-30 state. The tested model controls the same 20 selected agents, retaining their personas, inventories, and histories, while the other 80 keep the base model. All five models attempt to promise the same item to multiple buyers under ordinary instructions. Adding targets and deadlines increases these attempts for every model. Under adversarial instructions, the share of tested sellers' committed transactions completed despite unavailable items or overstated conditions rises from 15.4% to 33.4%, reaching 55.5% for GPT-5.4. Averaged across models and markets, simulated weekly earnings per tested agent rise from USD 20 under ordinary instructions to USD 33 under adversarial instructions. Most of the increase comes from items the sellers never held. We release the simulator, saved market states, evaluation code, and records covering 357,608 agent model calls for evaluating new models and developing safer marketplace agents.
Comments38 pages, 4 figures. Code: https://github.com/ziyan-wang98/BazaarBench; data: https://huggingface.co/BazaarBench
