PersoDPO: 通过多LLM评估实现可扩展的偏好优化
PersoDPO: Scalable Preference Optimization for Instruction-Adherent, Persona-Grounded Dialogue via Multi-LLM Evaluation
AI总结：
PersoDPO通过多LLM评估实现可扩展的偏好优化，提升对话系统的人格导向和上下文连贯性。
摘要：
个性化和上下文连贯性是构建有效的人格导向对话系统的关键组成部分。这些方面在提高用户参与度和确保响应更加相关且与用户身份一致方面起着至关重要的作用。然而，最近的研究表明，尽管开源大型语言模型（LLMs）在流畅性和自然性等通用对话能力方面表现出色，但仍难以生成同时具有上下文基础和与人格线索一致的响应。我们提出了PersoDPO，一种可扩展的偏好优化框架，该框架利用来自闭源和开源LLM生成响应的自动评估的监督信号来微调对话模型。该框架整合了针对连贯性和个性化的目标评估指标，以及一个长度格式合规性功能以促进指令遵循。这些信号被结合以自动构建高质量的偏好对，无需人工标注，从而实现可扩展且可重复的训练流程。在FoCus数据集上的实验表明，经过PersoDPO框架微调的开源语言模型在多个评估维度上均优于强大的开源基线和标准的直接偏好优化（DPO）变体。
英文摘要：
Personalization and contextual coherence are two essential components in building effective persona-grounded dialogue systems. These aspects play a crucial role in enhancing user engagement and ensuring responses are more relevant and consistent with user identity. However, recent studies indicate that open-source large language models (LLMs) continue to struggle to generate responses that are both contextually grounded and aligned with persona cues, despite exhibiting strong general conversational abilities like fluency and naturalness. We present PersoDPO, a scalable preference optimisation framework that uses supervision signals from automatic evaluations of responses generated by both closed-source and open-source LLMs to fine-tune dialogue models. The framework integrates evaluation metrics targeting coherence and personalization, along with a length-format compliance feature to promote instruction adherence. These signals are combined to automatically construct high-quality preference pairs without manual annotation, enabling a scalable and reproducible training pipeline. Experiments on the FoCus dataset show that an open-source language model fine-tuned with the PersoDPO framework consistently outperforms strong open-source baselines and a standard Direct Preference Optimization (DPO) variant across multiple evaluation dimensions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
