PRIDE: 特权信息增强的共情对话生成蒸馏方法
PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation
AI总结：
提出PRIDE方法，利用训练时可用但推理时不可用的特权信息（如心理标注或未来事件摘要），通过共情推理提示、多源注意力机制和双对齐损失，将大模型的共情推理能力蒸馏到小模型，在资源受限场景下保持性能。
摘要：
大型语言模型在生成多样化且上下文感知的共情对话响应方面展现了显著能力。然而，它们的计算需求严重限制了其在资源受限环境中的部署。虽然知识蒸馏提供了一种有前景的压缩方案，但它通常无法传递共情所需的细微理解，因为它忽略了指导人类连接的隐含上下文线索。为弥补这一差距，我们提出了一种用于共情对话生成的特权信息增强知识蒸馏方法（PRIDE）。我们的方法利用特权信息，例如专家心理标注或未来事件摘要，这些信息仅在训练时可用，但在推理时不可用。这使得我们能够将教师模型的共情推理转移到较小的模型，而无需在部署时依赖额外输入。具体来说，PRIDE包含三个关键组件：（1）一个共情推理提示，引导教师将共情过程显式分解为逐步理解感受和分析情境；（2）一个多源注意力机制，指导学生有效整合特权信息；（3）一个双对齐损失，结合反向Kullback-Leibler散度和最大均值差异，确保在logit和特征层面上的稳健知识转移。在多模态和纯文本数据集上的实验表明，我们的方法取得了有竞争力的性能，并且在某些情况下，在准确性和语义相关性方面匹配甚至超越了更大的教师模型。
英文摘要：
Large language models have demonstrated significant capabilities in generating diverse and context-aware responses for empathetic dialogue. However, their computational demands severely limit their deployment in resource-constrained environments. While knowledge distillation offers a promising compression solution, it often fails to transfer the nuanced understanding essential for empathy, as it overlooks the implicit contextual cues that guide human connection. To bridge this gap, we propose a \textbf{pr}ivileged \textbf{i}nformation-enhanced knowledge \textbf{d}istillation method for \textbf{e}mpathetic dialogue generation (PRIDE). Our method leverages privileged information, such as expert psychological annotations or future event summaries, which is available exclusively during training but unavailable at inference time. This allows us to transfer the teacher model's empathetic reasoning to smaller models without relying on extra inputs during deployment. Specifically, PRIDE has three key components: (1) An empathy-reasoning prompt that guides the teacher to explicitly decompose the empathetic process into understanding feelings and analyzing situations step-by-step; (2) A multi-source attention mechanism that directs the student to effectively integrate privileged information; (3) A dual-alignment loss that combines reversed Kullback-Leibler divergence and maximum mean discrepancy to ensure robust knowledge transfer at both logit and feature levels. Experiments on multi-modal and text-only datasets demonstrate that our method achieves competitive performance, and in some cases matches or even surpasses larger teacher models in terms of accuracy and semantic relevance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
