arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.00446cs.CL
收藏

利用社会感知对比学习提升对话安全性

Improving Dialog Safety using Socially Aware Contrastive Learning

University at Buffalo(布法罗大学)

Souvik Das, Rohini K. Srihari

AI总结:

针对现有对话AI易生成不安全内容、难识别细微不安全场景的问题,提出基于社会感知n对对比损失的双步微调方法,结合多数据集训练融合亲社会行为的模型,有效提升了对话回复的社会适宜性。

摘要:

当前最先进的对话AI系统因存在生成不安全、有毒、不道德或危险内容的潜在风险而引发担忧。已有研究构建了相关数据集,用以教会对话智能体恰当的社会行为范式,使其能有效应对专门设计的有害内容。然而,在这些对抗性数据集上训练的模型,仍难以识别对话中自然出现的细微不安全场景,或是在随意语境下给出不当回复。为明确这一问题的严重程度,我们研究了对抗性和随意对话语境下的亲社会性,并审计了通用语言模型在生成不安全内容倾向方面的回复质量。我们提出了一种双步微调流程,通过社会感知n对对比损失来解决上述问题。随后,我们利用Moral Integrity Corpus(MIC,道德诚信语料库)和ProsocialDialog等数据集训练了一个融合亲社会行为的基础模型。在多个对话数据集上的实验结果表明,我们的方法在生成符合社会规范的回复方面具备有效性。

英文摘要:

State-of-the-art conversational AI systems raise concerns due to their potential risks of generating unsafe, toxic, unethical, or dangerous content. Previous works have developed datasets to teach conversational agents the appropriate social paradigms to respond effectively to specifically designed hazardous content. However, models trained on these adversarial datasets still struggle to recognize subtle unsafe situations that appear naturally in conversations or introduce an inappropriate response in a casual context. To understand the extent of this problem, we study prosociality in both adversarial and casual dialog contexts and audit the response quality of general-purpose language models in terms of propensity to produce unsafe content. We propose a dual-step fine-tuning process to address these issues using a socially aware n-pair contrastive loss. Subsequently, we train a base model that integrates prosocial behavior by leveraging datasets like Moral Integrity Corpus (MIC) and ProsocialDialog. Experimental results on several dialog datasets demonstrate the effectiveness of our approach in generating socially appropriate responses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑