arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.04158cs.CLcs.AI
收藏

DA-Net:一种用于多源跨语言迁移学习的解耦自适应网络

DA-Net: A Disentangled and Adaptive Network for Multi-Source Cross-Lingual Transfer Learning

Beihang University(北京航空航天大学);Zhongguancun Laboratory(中关村实验室);National Computer Network Emergency Response Technical Team / Coordination Center of China(国家计算机网络应急技术处理协调中心)

Ling Ge, Chunming Hu, Guanghui Ma, Jihong Liu, Hong Zhang

AI总结:

针对多源跨语言迁移学习中编码器表征混杂干扰分类器、语言鸿沟导致预测不准的问题,提出DA-Net,通过反馈引导协同解耦和类别感知并行适配优化,在38种语言的三类任务上验证了有效性。

摘要:

多源跨语言迁移学习旨在应对语言偏移场景下,将任务知识从多个有标注源语言迁移到无标注目标语言的问题。现有方法通常侧重于对由共享编码器下不同源语言的特定语言分类器生成的预测结果进行加权。然而,所有源语言共用同一个由所有这些语言更新的编码器,提取的表征不可避免地会包含不同源语言的信息,这可能会干扰特定语言分类器的学习。此外,由于语言鸿沟,使用源语言标签训练的特定语言分类器无法对目标语言做出准确预测,这两个问题都会损害模型性能。为解决这些挑战,我们提出了解耦自适应网络(DA-Net)。首先,我们设计了一种反馈引导的协同解耦方法,旨在纯化分类器的输入表征,从而缓解多源之间的相互干扰。其次,我们提出了一种类别感知的并行适配方法,为每个源-目标语言对对齐类别级分布,从而缓解语言对之间的语言鸿沟。在涉及38种语言的三个不同任务上的实验结果验证了我们方法的有效性。

英文摘要:

Multi-Source cross-lingual transfer learning deals with the transfer of task knowledge from multiple labelled source languages to an unlabeled target language under the language shift. Existing methods typically focus on weighting the predictions produced by language-specific classifiers of different sources that follow a shared encoder. However, all source languages share the same encoder, which is updated by all these languages. The extracted representations inevitably contain different source languages' information, which may disturb the learning of the language-specific classifiers. Additionally, due to the language gap, language-specific classifiers trained with source labels are unable to make accurate predictions for the target language. Both facts impair the model's performance. To address these challenges, we propose a Disentangled and Adaptive Network (DA-Net). Firstly, we devise a feedback-guided collaborative disentanglement method that seeks to purify input representations of classifiers, thereby mitigating mutual interference from multiple sources. Secondly, we propose a class-aware parallel adaptation method that aligns class-level distributions for each source-target language pair, thereby alleviating the language pairs' language gap. Experimental results on three different tasks involving 38 languages validate the effectiveness of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑