MuLD:多任务长文档基准
MuLD: The Multitask Long Document Benchmark
AI总结:
本文提出MuLD,一个仅含超万标记文档的多任务长文档基准,评估现有模型表现,发现其挑战性高于短文档基准,并证明更长上下文长度的Transformer模型更有效,同时发布数据与代码以促进研究。
摘要:
自然语言处理技术的显著进步得益于多任务基准(如GLUE和SuperGLUE)的发展。尽管这些基准主要关注包含一到两个输入句子的任务,但已有令人振奋的研究致力于设计处理更长输入的高效技术。在本文中,我们提出了MuLD:一个新的长文档基准,仅包含超过10,000个标记的文档。通过修改现有的自然语言处理任务,我们创建了一个多样化的基准,要求模型成功建模文本中的长期依赖关系。我们评估了现有模型的表现,发现我们的基准比其对应的“短文档”基准更具挑战性。此外,通过评估常规和高效的Transformer模型,我们表明具有更长上下文长度的模型能够更好地解决所提出的任务,这表明未来对这些模型的改进对于解决类似的长文档问题至关重要。我们发布了数据和基线代码,以鼓励对高效自然语言处理模型的进一步研究。
英文摘要:
The impressive progress in NLP techniques has been driven by the development of multi-task benchmarks such as GLUE and SuperGLUE. While these benchmarks focus on tasks for one or two input sentences, there has been exciting work in designing efficient techniques for processing much longer inputs. In this paper, we present MuLD: a new long document benchmark consisting of only documents over 10,000 tokens. By modifying existing NLP tasks, we create a diverse benchmark which requires models to successfully model long-term dependencies in the text. We evaluate how existing models perform, and find that our benchmark is much more challenging than their `short document' equivalents. Furthermore, by evaluating both regular and efficient transformers, we show that models with increased context length are better able to solve the tasks presented, suggesting that future improvements in these models are vital for solving similar long document problems. We release the data and code for baselines to encourage further research on efficient NLP models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。