停止过滤:多视图属性增强对话学习
Stop Filtering: Multi-View Attribute-Enhanced Dialogue Learning
摘要:
通过过滤原始对话语料库来提高模型的会话能力越来越受到人们的关注。以前的过滤策略通常依赖于评分方法来从一个角度评估和丢弃样本,从而使模型能够增强相应的对话属性(例如,一致性)更容易。然而,丢弃的样本可能在其他方面获得高分,并且可以对模型学习提供正则化效果,这导致性能改善对过滤比率敏感。在这项工作中,我们提出了一个多视图属性增强的对话学习框架,加强属性相关的功能更强大,更全面。我们的框架不是过滤原始数据集来训练模型,而是首先在原始数据集上对模型进行预训练,然后通过选定子集上的适配器对其进行微调,这也增强了响应的某些属性,但不会出现上述问题。考虑到对话属性的多样性,我们进一步设计了多视图增强机制,包括多视图选择和视图间融合。它从多个角度分别对高质量样本进行分组,并使用相应的样本集和适配器增强响应的不同属性,保持知识独立并允许灵活集成。实验结果和分析表明,我们的框架可以提高性能显着增强对话属性和融合的观点特定的知识。
英文摘要:
There is a growing interest in improving the conversational ability of models by filtering the raw dialogue corpora. Previous filtering strategies usually rely on a scoring method to assess and discard samples from one perspective, enabling the model to enhance the corresponding dialogue attributes (e.g., consistency) more easily. However, the discarded samples may obtain high scores in other perspectives and can provide regularization effects on the model learning, which causes the performance improvement to be sensitive to the filtering ratio. In this work, we propose a multi-view attribute-enhanced dialogue learning framework that strengthens the attribute-related features more robustly and comprehensively. Instead of filtering the raw dataset to train the model, our framework first pre-trains the model on the raw dataset and then fine-tunes it through adapters on the selected sub-sets, which also enhances certain attributes of responses but without suffering from the problems mentioned above. Considering the variety of the dialogue attribute, we further design a multi-view enhancement mechanism, including multi-view selection and inter-view fusion. It groups the high-quality samples from multiple perspectives, respectively, and enhances different attributes of responses with the corresponding sample sets and adapters, keeping knowledge independent and allowing flexible integration. Empirical results and analysis show that our framework can improve the performance significantly in terms of enhancing dialogue attributes and fusing view-specific knowledge.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。