arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.13435cs.AI
收藏

MWP-BERT:数学应用题解决的数字增强预训练

MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving

University of Notre Dame(圣母大学);Hong Kong University of Science and Technology(香港科技大学);Singapore Management University(新加坡管理大学);Hefei University of Technology(合肥工业大学);East China Normal University(华东师范大学);University of Electronic Science and Technology of China(电子科技大学)

Zhenwen Liang, Jipeng Zhang, Lei Wang, Wei Qin, Yunshi Lan, Jie Shao, Xiangliang Zhang

摘要:

数学应用题解决在数字表示学习中面临着一个困境。为了避免数字表示问题和减少可行解的搜索空间,现有的工作努力MWP求解通常取代真正的数字与符号占位符,专注于逻辑推理。然而,与程序合成和知识图谱推理等常见的符号推理任务不同,MWP求解对数值推理有额外的要求。换句话说,在数值推理中更重要的是可重复使用的数值属性,而不是数值本身。因此,我们认为,注入数字属性的符号占位符与语境化的表示学习图式可以提供一个出路的困境,在数字表征问题。在这项工作中,我们将这一思想引入到流行的预训练语言模型(PLM)技术中,并建立了MWP-BERT,一个有效的上下文数字表示PLM。我们证明了我们的MWP-BERT的有效性MWP解决和几个MWP特定的理解任务的英语和汉语基准。

英文摘要:

Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, different from common symbolic reasoning tasks like program synthesis and knowledge graph reasoning, MWP solving has extra requirements in numerical reasoning. In other words, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema can provide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM. We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑