arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.08464cs.CL
收藏

寻找模式,而不仅仅是简化程序:解决数学应用题的对比学习

Seeking Patterns, Not just Memorizing Procedures: Contrastive Learning for Solving Math Word Problems

Tencent Cloud Xiaowei(腾讯云小微);Peking University(北京大学)

Zhongli Li, Wenxuan Zhang, Chao Yan, Qingyu Zhou, Chao Li, Hongzhi Liu, Yunbo Cao

摘要:

数学应用题解决需要发现自然语言叙述中的数量关系。最近的工作表明,现有的模型从上下文中记忆过程,并依赖于浅层算法来解决MWPs。在本文中,我们看看这个问题,并认为,原因是缺乏全面的了解MWP模式。我们首先研究了神经网络如何只从语义上理解模式,并观察到,如果原型方程是相同的,大多数问题得到更接近的表示和那些表示除了他们或接近其他原型往往会产生错误的解决方案。受此启发,我们提出了一种对比学习方法,其中神经网络感知模式的分歧。我们通过将原型方程转化为树并寻找相似的树结构来收集对比实例。求解模型在收集的示例上使用辅助目标进行训练,从而使具有相似原型的问题的表示更加接近。我们在中文数据集Math23k和英文数据集MathQA上进行了实验。我们的方法大大提高了单语和多语言设置的性能。

英文摘要:

Math Word Problem (MWP) solving needs to discover the quantitative relationships over natural language narratives. Recent work shows that existing models memorize procedures from context and rely on shallow heuristics to solve MWPs. In this paper, we look at this issue and argue that the cause is a lack of overall understanding of MWP patterns. We first investigate how a neural network understands patterns only from semantics, and observe that, if the prototype equations are the same, most problems get closer representations and those representations apart from them or close to other prototypes tend to produce wrong solutions. Inspired by it, we propose a contrastive learning approach, where the neural network perceives the divergence of patterns. We collect contrastive examples by converting the prototype equation into a tree and seeking similar tree structures. The solving model is trained with an auxiliary objective on the collected examples, resulting in the representations of problems with similar prototypes being pulled closer. We conduct experiments on the Chinese dataset Math23k and the English dataset MathQA. Our method greatly improves the performance in monolingual and multilingual settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑