CoReflect：通过共进化模拟与反思性评分表细化进行对话评估
CoReflect: A Reflective Co-Evolution Framework for Improving Conversational Evaluation
AI总结：
CoReflect通过共进化模拟与反思性评分表细化，实现对话系统的自适应评估，提升测试用例复杂性和评分表精度。
摘要：
在多轮对话场景中评估对话系统仍然是一个基本挑战。传统流程通常依赖于人工定义的评分表和固定的对话上下文——一种静态方法，限制了覆盖范围，并无法捕捉对话模型多样且涌现的行为。为了解决这一问题，我们引入了CoReflect（通过共进化模拟与反思性评分表细化进行对话评估），将对话模拟和评估统一成一个适应性、迭代的过程。CoReflect采用一个对话规划器，生成结构化模板以指导用户模拟器进行多样、目标导向的对话。随后，一个反思分析器处理这些对话，以识别系统性行为模式并自动细化评分表。关键的是，对话分析的见解会反馈到规划器中，以更新对话模板以供后续迭代。这种共进化循环确保了测试用例的复杂性和评分表的诊断精度同步提升。通过最小化人工干预，CoReflect提供了一种可扩展且自我完善的方法，使评估协议能够随着对话模型快速发展的能力而适应。
英文摘要：
Evaluating conversational systems in multi-turn settings remains a fundamental challenge. Conventional pipelines typically rely on manually defined rubrics and fixed conversational context$-$a static approach that limits coverage and fails to capture the diverse, emergent behaviors of dialogue models. To address this, we introduce CoReflect (A Reflective Co-Evolution Framework for Improving Conversational Evaluation), which unifies dialogue simulation and evaluation into an adaptive, iterative process. CoReflect employs a conversation planner that generates structured templates to guide a user simulator through diverse, goal-directed dialogues. Subsequently, a reflective analyzer processes these dialogues to identify systematic behavioral patterns and automatically refine the evaluation rubrics. Crucially, the insights from the conversation analysis are fed back into the planner to update conversation templates for subsequent iterations. This co-evolution loop ensures that the complexity of test cases and the diagnostic precision of rubrics improve in tandem. By minimizing human intervention, CoReflect provides a scalable and self-refining methodology that allows evaluation protocols to adapt alongside the rapidly advancing capabilities of dialogue models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
