新兴领域开放式检索问答系统中的跨语言问题研究
Addressing Issues of Cross-Linguality in Open-Retrieval Question Answering Systems For Emergent Domains
摘要：
开放式检索问答系统通常在成熟领域的大型数据集上进行训练和测试。然而，低资源设置，如新的和新兴的领域将特别受益于可靠的问答系统。此外，新兴领域的多语言和跨语言资源稀缺，导致很少或根本没有这样的系统。在本文中，我们展示了一个跨语言的开放检索问答系统的新冠肺炎的紧急领域。我们的系统采用了语料库的科学文章，以确保检索到的文件是可靠的。为了解决紧急领域中跨语言训练数据的稀缺性，我们提出了一种利用自动翻译、对齐和过滤来产生英语到所有数据集的方法。我们表明，深度语义检索器从我们的英语到所有数据的训练中受益匪浅，并且在跨语言设置中显着优于BM 25基线。我们举例说明我们的系统的能力，并释放所有必要的代码训练和部署这样一个系统。
英文摘要：
Open-retrieval question answering systems are generally trained and tested on large datasets in well-established domains. However, low-resource settings such as new and emerging domains would especially benefit from reliable question answering systems. Furthermore, multilingual and cross-lingual resources in emergent domains are scarce, leading to few or no such systems. In this paper, we demonstrate a cross-lingual open-retrieval question answering system for the emergent domain of COVID-19. Our system adopts a corpus of scientific articles to ensure that retrieved documents are reliable. To address the scarcity of cross-lingual training data in emergent domains, we present a method utilizing automatic translation, alignment, and filtering to produce English-to-all datasets. We show that a deep semantic retriever greatly benefits from training on our English-to-all data and significantly outperforms a BM25 baseline in the cross-lingual setting. We illustrate the capabilities of our system with examples and release all code necessary to train and deploy such a system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
