arXiv 2305.093602023-07-18cs.CL

GIFT：面向多方对话理解的图诱导微调

GIFT: Graph-Induced Fine-Tuning for Multi-Party Conversation Understanding

University of Science and Technology of China（中国科学技术大学）；iFLYTEK Research（讯飞研究院）

Jia-Chen Gu, Zhen-Hua Ling, Quan Liu, Cong Liu, Guoping Hu

备注：: Accepted by ACL 2023. arXiv admin note: substantial text overlap with arXiv:2106.01541

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有多方对话理解方法对图结构利用不足的问题，提出即插即用的轻量图诱导微调方法GIFT，通过四类边将图信号融入注意力优化预训练语言模型，仅增少量参数就在多个任务上取得SOTA性能。

摘要：

解决多方对话（MPCs）中“谁对谁说了什么”的问题近年来吸引了大量研究关注。然而，现有的多方对话理解方法通常将对话参与者和话语嵌入到序列信息流中，或是仅利用了多方对话中固有图结构的表层信息。为此，我们提出了一种即插即用的轻量级方法，名为图诱导微调（graph-induced fine-tuning，GIFT），该方法可适配多种基于Transformer的预训练语言模型（PLMs），用于通用多方对话理解。具体而言，常规Transformer中话语之间的完全对等连接忽略了多方对话中一个话语对另一个话语稀疏但具有区分性的依赖关系。为区分话语之间的不同关系，我们设计了四种类型的边，将图诱导信号整合到注意力机制中，以优化原本为处理序列文本设计的预训练语言模型。我们将GIFT实现在三个预训练语言模型中进行评估，并在三个下游任务上测试性能，包括受话人识别、说话人识别和回复选择。实验结果表明，GIFT仅在每个编码层增加4个额外参数，就能显著提升三个预训练语言模型在三个下游任务和两个基准上的性能，在多方对话理解任务上达到了新的state-of-the-art（最优）性能。

英文摘要：

Addressing the issues of who saying what to whom in multi-party conversations (MPCs) has recently attracted a lot of research attention. However, existing methods on MPC understanding typically embed interlocutors and utterances into sequential information flows, or utilize only the superficial of inherent graph structures in MPCs. To this end, we present a plug-and-play and lightweight method named graph-induced fine-tuning (GIFT) which can adapt various Transformer-based pre-trained language models (PLMs) for universal MPC understanding. In detail, the full and equivalent connections among utterances in regular Transformer ignore the sparse but distinctive dependency of an utterance on another in MPCs. To distinguish different relationships between utterances, four types of edges are designed to integrate graph-induced signals into attention mechanisms to refine PLMs originally designed for processing sequential texts. We evaluate GIFT by implementing it into three PLMs, and test the performance on three downstream tasks including addressee recognition, speaker identification and response selection. Experimental results show that GIFT can significantly improve the performance of three PLMs on three downstream tasks and two benchmarks with only 4 additional parameters per encoding layer, achieving new state-of-the-art performance on MPC understanding.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。