通过不确定性估计缓解情感支持对话中的策略偏好偏差
Mitigating Strategy Preference Bias in Emotional Support Conversation via Uncertainty Estimations
AI总结：
本文通过识别LLMs在策略规划中的知识边界揭示偏好偏差根源，提出基于双重奖励函数的强化学习方法，利用准确率和基于熵的置信度优化策略规划，有效缓解了情感支持对话中的策略偏好偏差。
摘要：
情感支持对话（ESC）旨在通过共情对话缓解痛苦，但由于策略规划准确率低，大型语言模型（LLMs）在提供有效ESC方面面临持续挑战。此外，模型对特定策略存在相当大的偏好偏差。先前使用微调策略规划器的方法在减少此类偏差方面显示出潜力，但LLMs中偏好偏差的根本原因尚未得到充分研究。为了解决这些问题，我们首先通过识别LLMs在策略规划中的知识边界来揭示偏差的根本原因。然后，我们提出了一种通过具有双重奖励函数的强化学习来缓解偏差的方法，该方法根据知识边界，通过准确率和基于熵的置信度来优化各个区域的策略规划。在ESCov和ExTES数据集上使用多个LLM骨干网络的实验表明，我们的方法优于基线，证实了我们方法的有效性。
英文摘要：
Emotional support conversation (ESC) aims to alleviate distress through empathetic dialogue, yet large language models (LLMs) face persistent challenges in delivering effective ESC due to low accuracy in strategy planning. Moreover, there is a considerable preference bias towards specific strategies. Prior methods using fine-tuned strategy planners have shown potential in reducing such bias, while the underlying causes of the preference bias in LLMs have not well been studied. To address these issues, we first reveal the fundamental causes of the bias by identifying the knowledge boundaries of LLMs in strategy planning. Then, we propose an approach to mitigate the bias by reinforcement learning with a dual reward function, which optimizes strategy planning via both accuracy and entropy-based confidence for each region according to the knowledge boundaries. Experiments on the ESCov and ExTES datasets with multiple LLM backbones show that our approach outperforms the baselines, confirming the effectiveness of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
