arXiv 2505.141062025-05-25cs.CLcs.AI

个性化对话基准：迈向模拟个性化对话

A Personalized Conversational Benchmark: Towards Simulating Personalized Conversations

University of Southern California（南加州大学）；Adobe Research（Adobe研究院）；University of California, San Diego（加州大学圣地亚哥分校）；Intel AI Research（英特尔人工智能研究）；University of Maryland, College Park（马里兰大学学院市分校）；Vanderbilt University（范德比大学）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院和州立大学）；University of Notre Dame（诺特丹大学）；University of Oregon（俄勒冈大学）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Arizona State University（亚利桑那州立大学）

Li Li, Peilin Cai, Ryan A. Rossi, Franck Dernoncourt, Branislav Kveton, Junda Wu, Tong Yu, Linxin Song, Tiankai Yang, Yuehan Qin, Nesreen K. Ahmed, Samyadeep Ba… 展开作者

Li Li, Peilin Cai, Ryan A. Rossi, Franck Dernoncourt, Branislav Kveton, Junda Wu, Tong Yu, Linxin Song, Tiankai Yang, Yuehan Qin, Nesreen K. Ahmed, Samyadeep Basu, Subhojyoti Mukherjee, Ruiyi Zhang, Zhengmian Hu, Bo Ni, Yuxiao Zhou, Zichao Wang, Yue Huang, Yu Wang, Xiangliang Zhang, Philip S. Yu, Xiyang Hu, Yue Zhao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出PersonaConvBench基准，整合个性化与对话结构，通过三个核心任务评估LLM在个性化多轮对话中的推理与生成能力，实验显示融入个性化历史可带来显著性能提升。

摘要：

我们提出了PersonaConvBench，一个用于评估大语言模型（LLMs）在多轮对话中个性化推理与生成能力的大规模基准。与现有仅关注个性化或对话结构某一方面的研究不同，PersonaConvBench将两者整合，提供三个核心任务：句子分类、影响回归以及面向用户的文本生成，覆盖十个基于Reddit的多样化领域。该设计使得在真实多用户场景中系统分析个性化对话上下文如何影响LLM输出成为可能。我们在统一的提示设置下对多个商业和开源LLM进行了基准测试，观察到融入个性化历史带来了显著的性能提升，包括在情感分类中相对于最佳非对话基线获得了198%的相对增益。通过发布PersonaConvBench及其评估和代码，我们旨在支持关于适应个体风格、跟踪长期上下文并生成上下文丰富且引人入胜的响应的LLM研究。

英文摘要：

We present PersonaConvBench, a large-scale benchmark for evaluating personalized reasoning and generation in multi-turn conversations with large language models (LLMs). Unlike existing work that focuses on either personalization or conversational structure in isolation, PersonaConvBench integrates both, offering three core tasks: sentence classification, impact regression, and user-centric text generation across ten diverse Reddit-based domains. This design enables systematic analysis of how personalized conversational context shapes LLM outputs in realistic multi-user scenarios. We benchmark several commercial and open-source LLMs under a unified prompting setup and observe that incorporating personalized history yields substantial performance improvements, including a 198 percent relative gain over the best non-conversational baseline in sentiment classification. By releasing PersonaConvBench with evaluations and code, we aim to support research on LLMs that adapt to individual styles, track long-term context, and produce contextually rich, engaging responses.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。