对话系统中基于偏好自适应强化学习的少数群体感知满意度估计
Minority-Aware Satisfaction Estimation in Dialogue Systems via Preference-Adaptive Reinforcement Learning
AI总结：
本文针对对话系统中少数群体满意度估计被忽视的问题，提出融合个体与群体偏好的统一框架，通过CoPeR推理链、M2PC聚类和PAda-PPO强化学习实现偏好自适应对齐，在情感支持对话数据集上显著提升满意度估计性能。
摘要：
对话系统中的用户满意度本质上是主观的。当相同的响应策略被应用于不同用户时，由于个体意图和偏好的差异，少数群体用户可能会给出与多数群体用户不同的满意度评分。然而，现有的对齐方法通常训练“一刀切”的模型，旨在追求广泛共识，往往忽视了少数群体的视角和用户特定的适应性调整。我们提出了一个统一框架，同时建模个体层面和群体层面的偏好以进行用户满意度估计。首先，我们引入个性化推理链（Chain-of-Personalized-Reasoning，CoPeR），通过可解释的推理链捕捉个体偏好。其次，我们提出一种基于期望最大化的多数-少数群体偏好感知聚类（Majority-Minority Preference-Aware Clustering，M2PC）算法，以无监督方式发现不同的用户群体，从而学习群体层面的偏好。最后，我们将这些组件整合到一个偏好自适应强化学习框架（PAda-PPO）中，联合优化与个体偏好和群体偏好的对齐。在情感支持对话（Emotional Support Conversation）数据集上的实验表明，用户满意度估计得到了一致的提升，尤其对代表性不足的用户群体效果更为显著。
英文摘要：
User satisfaction in dialogue systems is inherently subjective. When the same response strategy is applied across users, minority users may assign different satisfaction ratings than majority users due to variations in individual intents and preferences. However, existing alignment methods typically train one-size-fits-all models that aim for broad consensus, often overlooking minority perspectives and user-specific adaptation. We propose a unified framework that models both individual- and group-level preferences for user satisfaction estimation. First, we introduce Chain-of-Personalized-Reasoning (CoPeR) to capture individual preferences through interpretable reasoning chains. Second, we propose an expectation-maximization-based Majority-Minority Preference-Aware Clustering (M2PC) algorithm that discovers distinct user groups in an unsupervised manner to learn group-level preferences. Finally, we integrate these components into a preference-adaptive reinforcement learning framework (PAda-PPO) that jointly optimizes alignment with both individual and group preferences. Experiments on the Emotional Support Conversation dataset demonstrate consistent improvements in user satisfaction estimation, particularly for underrepresented user groups.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
