训练数据比你想象的更有价值--一种简单有效的从训练数据中检索的方法
Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
摘要：
基于检索的方法已被证明是有效的NLP任务，通过引入外部知识。然而，大规模语料库的索引和检索带来了相当大的计算成本。令人惊讶的是，我们发现，从训练数据中提取（REINA）只能在多个NLG和NLU任务上获得显着收益。我们检索与输入文本最相似的标记训练实例，然后将它们与输入连接起来，以馈送到模型中以生成输出。实验结果表明，这种简单的方法可以实现各种NLU和NLG任务，包括摘要，机器翻译，语言建模和问答任务的性能显着更好。例如，我们提出的方法在XSum，BigPatent和CommonsenseQA上取得了最先进的结果。我们的代码已发布，https://github.com/microsoft/REINA。
英文摘要：
Retrieval-based methods have been shown to be effective in NLP tasks via introducing external knowledge. However, the indexing and retrieving of large-scale corpora bring considerable computational cost. Surprisingly, we found that REtrieving from the traINing datA (REINA) only can lead to significant gains on multiple NLG and NLU tasks. We retrieve the labeled training instances most similar to the input text and then concatenate them with the input to feed into the model to generate the output. Experimental results show that this simple method can achieve significantly better performance on a variety of NLU and NLG tasks, including summarization, machine translation, language modeling, and question answering tasks. For instance, our proposed method achieved state-of-the-art results on XSum, BigPatent, and CommonsenseQA. Our code is released, https://github.com/microsoft/REINA .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
