解耦ESC：通过策略-回应解耦的偏好优化增强情感支持生成
DecoupledESC: Enhancing Emotional Support Generation via Strategy-Response Decoupled Preference Optimization
AI总结：
本文提出了解耦ESC框架，通过策略-回应解耦的偏好优化方法，提升情感支持生成的质量与准确性。
摘要：
最近在情感支持对话（ESC）方面的进展通过监督微调（SFT）对大型语言模型（LLMs）进行微调，从而提高了情感支持生成的能力。然而，常见的心理错误仍然存在。虽然直接偏好优化（DPO）通过成对偏好学习显示出减少这些错误的潜力，但在ESC任务中其有效性受到两个关键挑战的限制：（1）纠缠的数据结构：现有的ESC数据本质上将心理策略和回应内容纠缠在一起，使得难以构建高质量的偏好对；（2）优化模糊性：将普通的DPO应用于这种纠缠的成对数据会导致训练目标模糊。为了解决这些问题，我们引入了推断偏好挖掘（IPM）来构建高质量的偏好数据，形成IPM-PrefDial数据集。在此基础上，我们提出了一种受Gross扩展情绪调节过程模型启发的解耦ESC框架，该框架将ESC任务分解为两个顺序子任务：策略规划和共情回应生成。每个子任务通过SFT进行训练，随后通过DPO进一步优化，以符合心理偏好。广泛的实验表明，我们的解耦ESC框架优于联合优化基线，减少了偏好偏差并提高了回应质量。
英文摘要：
Recent advances in Emotional Support Conversation (ESC) have improved emotional support generation by fine-tuning Large Language Models (LLMs) via Supervised Fine-Tuning (SFT). However, common psychological errors still persist. While Direct Preference Optimization (DPO) shows promise in reducing such errors through pairwise preference learning, its effectiveness in ESC tasks is limited by two key challenges: (1) Entangled data structure: Existing ESC data inherently entangles psychological strategies and response content, making it difficult to construct high-quality preference pairs; and (2) Optimization ambiguity: Applying vanilla DPO to such entangled pairwise data leads to ambiguous training objectives. To address these issues, we introduce Inferential Preference Mining (IPM) to construct high-quality preference data, forming the IPM-PrefDial dataset. Building upon this data, we propose a Decoupled ESC framework inspired by Gross's Extended Process Model of Emotion Regulation, which decomposes the ESC task into two sequential subtasks: strategy planning and empathic response generation. Each was trained via SFT and subsequently enhanced by DPO to align with the psychological preference. Extensive experiments demonstrate that our Decoupled ESC framework outperforms joint optimization baselines, reducing preference bias and improving response quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
