融合词汇与句法知识的无监督跨语言迁移
Incorporating Lexical and Syntactic Knowledge for Unsupervised Cross-Lingual Transfer
AI总结：
针对无监督跨语言迁移中词汇与句法知识未能结合的问题，提出融合两者的 Lexicon-Syntax Enhanced Multilingual BERT 框架，通过代码切换和句法图注意力网络增强 mBERT，在多项零样本任务上持续优于基线。
摘要：
无监督跨语言迁移涉及在没有显式监督的情况下在语言之间传递知识。尽管已有许多研究通过关注跨语言知识（特别是词汇和句法知识）来提升此类任务的性能，但当前方法存在局限性，因为它们仅结合了句法或词汇信息。由于每种类型的信息都提供了独特的优势，且先前没有尝试将两者结合，我们试图探索这种方法的潜力。在本文中，我们提出了一个名为“Lexicon-Syntax Enhanced Multilingual BERT”的新颖框架，它结合了词汇和句法知识。具体来说，我们使用 Multilingual BERT (mBERT) 作为基础模型，并采用两种技术来增强其学习能力。代码切换技术用于隐式地教导模型词汇对齐信息，而基于句法的图注意力网络被设计来帮助模型编码句法结构。为了整合这两种类型的知识，我们将代码切换序列同时输入到句法模块和 mBERT 基础模型中。我们广泛的实验结果表明，该框架在零样本跨语言迁移中能够持续优于所有基线，在文本分类、命名实体识别（ner）和语义解析任务上获得了 1.0~3.7 分的提升。关键词：跨语言迁移，词汇，句法，代码切换，图注意力网络
英文摘要：
Unsupervised cross-lingual transfer involves transferring knowledge between languages without explicit supervision. Although numerous studies have been conducted to improve performance in such tasks by focusing on cross-lingual knowledge, particularly lexical and syntactic knowledge, current approaches are limited as they only incorporate syntactic or lexical information. Since each type of information offers unique advantages and no previous attempts have combined both, we attempt to explore the potential of this approach. In this paper, we present a novel framework called "Lexicon-Syntax Enhanced Multilingual BERT" that combines both lexical and syntactic knowledge. Specifically, we use Multilingual BERT (mBERT) as the base model and employ two techniques to enhance its learning capabilities. The code-switching technique is used to implicitly teach the model lexical alignment information, while a syntactic-based graph attention network is designed to help the model encode syntactic structure. To integrate both types of knowledge, we input code-switched sequences into both the syntactic module and the mBERT base model simultaneously. Our extensive experimental results demonstrate this framework can consistently outperform all baselines of zero-shot cross-lingual transfer, with the gains of 1.0~3.7 points on text classification, named entity recognition (ner), and semantic parsing tasks. Keywords:cross-lingual transfer, lexicon, syntax, code-switching, graph attention network
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
