SimpleQuestions几乎解决了：一种新的上限和基线方法
SimpleQuestions Nearly Solved: A New Upperbound and Baseline Approach
摘要：
SimpleQuestions数据集是研究单关系类事实问题最常用的基准之一。在本文中，我们提出了新的证据表明，这个基准几乎可以解决的标准方法。首先，我们表明，在这个基准测试中，数据边界的模糊性表现为83.4%;通常有多个答案不能单独从语言信号中消除歧义。其次，我们引入了一个基线，将新的最先进的性能水平设定为78.1%的准确率，尽管使用标准方法。最后，我们报告的实证分析表明，上限是松散的，大约三分之一的剩余错误也无法解决的语言信号。总之，这些结果表明SimpleQuestions数据集几乎已经解决。
英文摘要：
The SimpleQuestions dataset is one of the most commonly used benchmarks for studying single-relation factoid questions. In this paper, we present new evidence that this benchmark can be nearly solved by standard methods. First we show that ambiguity in the data bounds performance on this benchmark at 83.4%; there are often multiple answers that cannot be disambiguated from the linguistic signal alone. Second we introduce a baseline that sets a new state-of-the-art performance level at 78.1% accuracy, despite using standard methods. Finally, we report an empirical analysis showing that the upperbound is loose; roughly a third of the remaining errors are also not resolvable from the linguistic signal. Together, these results suggest that the SimpleQuestions dataset is nearly solved.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
