arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1911.06111cs.CLcs.IRcs.LGstat.ML
收藏

基于实例的多语种深度检索迁移学习

Instance-based Transfer Learning for Multilingual Deep Retrieval

Google Research(谷歌研究院);AWS AI(亚马逊云科技人工智能)

Andrew O. Arnold, William W. Cohen

摘要:

我们专注于在多语言环境中的搜索问题。通过对下一句预测和反向完形填空问题的研究,我们发现,在大规模测试中,基于实例的迁移学习在多语言环境中的效果令人惊讶,在所有35种目标语言和两个测试任务上都产生了正迁移。我们分析了这种改进,并认为最自然的解释,即语言之间的直接词汇重叠,只是部分解释了性能的提高:事实上,我们证明了目标语言的改进可以发生在添加辅助语言的数据后,即使没有词汇与目标相同。这一令人惊讶的结果是由于辅助语言和目标语言之间的传递性词汇重叠的影响。

英文摘要:

We focus on the problem of search in the multilingual setting. Examining the problems of next-sentence prediction and inverse cloze, we show that at large scale, instance-based transfer learning is surprisingly effective in the multilingual setting, leading to positive transfer on all of the 35 target languages and two tasks tested. We analyze this improvement and argue that the most natural explanation, namely direct vocabulary overlap between languages, only partially explains the performance gains: in fact, we demonstrate target-language improvement can occur after adding data from an auxiliary language even with no vocabulary in common with the target. This surprising result is due to the effect of transitive vocabulary overlaps between pairs of auxiliary and target languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑