SmartSalesBench：用于评估大语言模型销售智能体的抗干扰多轮基准测试
SmartSalesBench: A Resistance-Aware Multi-Turn Benchmark for Evaluating LLM Sales Agents
AI总结：
SmartSalesBench是首个针对LLM销售智能体的多轮对话基准，以抗拒性买家角色为评估对象，含多类配置与实验，采用RAG生成，设三类评估指标，结合自动与人工评估验证有效性。
摘要：
我们提出了SmartSalesBench，这是首个专门针对大语言模型（LLM）销售智能体进行评估的多轮对话基准测试，其评估依据是基于心理学原理构建的具有抗拒性的买家角色。现有的面向任务的对话基准测试均假设用户是合作的，并以任务完成为评估目标；而SmartSalesBench则颠覆了这一假设，让智能体与被编程为在最多二十轮的对话中进行抗拒和质疑的角色展开对抗。该基准测试包含五种智能体配置、十二个买家角色、六个价格层级的十五款智能手机产品，以及四项受控实验，共3轮迭代，总计13680个会话。销售智能体和买家角色的生成均基于检索增强生成（RAG）技术。我们引入了九个评估指标，分为三个层级：主要结果指标（CR、BRS、RIR）、次要质量指标（SMS、PFS、ETI），以及计划后续报告的探索性指标（ITS、ARI、SPI）。我们的评估结合了自动化的LLM作为评判者的评分与内部人工评估（样本量n=50，Cohen’s kappa值为0.71，与BRS的皮尔逊相关系数r=0.81）。
英文摘要：
We present SmartSalesBench, the first multi-turn conversational benchmark explicitly designed to evaluate large language model (LLM) sales agents against psychologically grounded, resistant buyer personas. Existing task-oriented dialogue benchmarks assume cooperative users and measure task completion; SmartSalesBench inverts this assumption, placing agents against personas programmed to resist and challenge across conversations of up to twenty turns. The benchmark includes five agent profiles, twelve buyer personas, fifteen smartphone products across six price tiers, and four controlled experiments with 3 iterations totaling 13,680 sessions. Both sales-agent and buyer persona generations are grounded through retrieval-augmented generation (RAG). We introduce nine evaluation metrics grouped into three tiers: primary outcome metrics (CR, BRS, RIR), secondary quality metrics (SMS, PFS, ETI), and exploratory metrics planned for future reporting (ITS, ARI, SPI). Our evaluation combines automated LLM-as-judge scoring with internal human assessment (n = 50, \k{appa} = 0.71, Pearson r = 0.81 with BRS).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
