NLP模型真的能够解决简单的数学应用题吗？
Are NLP Models really able to Solve Simple Math Word Problems?
摘要：
设计数学应用题（MWP）的NLP求解器的问题已经看到了持续的研究活动和测试准确性的稳定增长。由于现有的求解器实现高性能的基准数据集上的初级水平MWPs包含一个未知的算术字的问题，这样的问题往往被认为是“解决”的大部分研究的注意力转移到更复杂的MWPs。在本文中，我们限制我们的注意力在四年级及以下的英语MWPs教学。我们提供了强有力的证据表明，现有的MWP求解器依赖于浅层算法来实现高性能的基准数据集。为此，我们表明，MWP求解器，没有访问的问题，在MWP仍然可以解决大部分的MWP。类似地，将MWP视为词袋的模型也可以实现令人惊讶的高准确性。此外，我们引入了一个挑战数据集，SVAMP，通过应用精心选择的变化从现有的数据集采样的例子。最先进的模型所达到的最佳精度在SVAMP上要低得多，因此即使是最简单的MWP也有很多工作要做。
英文摘要：
The problem of designing NLP solvers for math word problems (MWP) has seen sustained research activity and steady gains in the test accuracy. Since existing solvers achieve high performance on the benchmark datasets for elementary level MWPs containing one-unknown arithmetic word problems, such problems are often considered "solved" with the bulk of research attention moving to more complex MWPs. In this paper, we restrict our attention to English MWPs taught in grades four and lower. We provide strong evidence that the existing MWP solvers rely on shallow heuristics to achieve high performance on the benchmark datasets. To this end, we show that MWP solvers that do not have access to the question asked in the MWP can still solve a large fraction of MWPs. Similarly, models that treat MWPs as bag-of-words can also achieve surprisingly high accuracy. Further, we introduce a challenge dataset, SVAMP, created by applying carefully chosen variations over examples sampled from existing datasets. The best accuracy achieved by state-of-the-art models is substantially lower on SVAMP, thus showing that much remains to be done even for the simplest of the MWPs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
