ProactiveEval：面向主动对话智能体的统一评估框架
ProactiveEval: A Unified Evaluation Framework for Proactive Dialogue Agents
AI总结：
本文提出统一评估框架 ProactiveEval，将主动对话分解为目标规划与对话引导，构建跨 6 个领域 328 个环境并实验 22 种 LLMs，揭示推理能力对主动行为的影响。
摘要：
主动对话已成为推进大语言模型（LLMs）研究中一个关键且具有挑战性的问题。现有工作主要聚焦于特定领域或任务导向场景，这导致评估碎片化，并限制了对模型主动对话能力的全面探索。在本工作中，我们提出 ProactiveEval，一个旨在评估 LLMs 主动对话能力的统一框架。该框架将主动对话分解为目标规划与对话引导，并在多个领域建立评估指标。此外，它还能自动生成多样且具有挑战性的评估数据。基于所提出的框架，我们构建了涵盖 6 个不同领域的 328 个评估环境。通过对 22 种不同类型 LLMs 的实验，我们表明 DeepSeek-R1 和 Claude-3.7-Sonnet 分别在目标规划与对话引导任务上表现出卓越性能。最后，我们研究了推理能力如何影响主动行为，并讨论了其对未来模型开发的启示。
英文摘要：
Proactive dialogue has emerged as a critical and challenging research problem in advancing large language models (LLMs). Existing works predominantly focus on domain-specific or task-oriented scenarios, which leads to fragmented evaluations and limits the comprehensive exploration of models' proactive conversation abilities. In this work, we propose ProactiveEval, a unified framework designed for evaluating proactive dialogue capabilities of LLMs. This framework decomposes proactive dialogue into target planning and dialogue guidance, establishing evaluation metrics across various domains. Moreover, it also enables the automatic generation of diverse and challenging evaluation data. Based on the proposed framework, we develop 328 evaluation environments spanning 6 distinct domains. Through experiments with 22 different types of LLMs, we show that DeepSeek-R1 and Claude-3.7-Sonnet exhibit exceptional performance on target planning and dialogue guidance tasks, respectively. Finally, we investigate how reasoning capabilities influence proactive behaviors and discuss their implications for future model development.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
