arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.04792cs.CL
收藏

瞄准基准:当前自然语言处理研究的方法论

Targeting the Benchmark: On Methodology in Current Natural Language Processing Research

University of Potsdam(波茨坦大学)

David Schlangen

摘要:

它已经成为我们领域的一种常见模式:一组引入了一项语言任务,以数据集为例,他们认为这具有足够的挑战性,可以作为基准。他们还提供了一个基线模型,然后很快被其他团体改进。通常,研究工作然后继续,模式重复。典型地留下隐含的是关于为什么这构成进步以及朝着什么方向进步的论证。在这篇文章中,我们试着从这个模式中退一步,找出可能的论点及其部分。

英文摘要:

It has become a common pattern in our field: One group introduces a language task, exemplified by a dataset, which they argue is challenging enough to serve as a benchmark. They also provide a baseline model for it, which then soon is improved upon by other groups. Often, research efforts then move on, and the pattern repeats itself. What is typically left implicit is the argumentation for why this constitutes progress, and progress towards what. In this paper, we try to step back for a moment from this pattern and work out possible argumentations and their parts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑