表达式语法信息瓶颈用于数学文字问题
Expression Syntax Information Bottleneck for Math Word Problems
AI总结：
本文提出ESIB方法，通过信息瓶颈理论提取数学文字问题的表达式语法本质特征，通过多模型相互学习和自蒸馏损失提升模型泛化能力与解题多样性。
摘要：
数学文字问题（MWP）旨在自动解决文本中给出的数学问题。以往的研究倾向于设计复杂的模型以捕捉原始文本中的额外信息，以便使模型获得更全面的特征。在本文中，我们转向相反的方向，研究如何丢弃包含虚假相关性的冗余特征。为此，我们设计了一种基于变分信息瓶颈的表达式语法信息瓶颈方法用于MWP（称为ESIB），该方法在提取表达式语法树的本质特征的同时过滤掉包含语法无关特征的潜在特定冗余。ESIB的核心思想是鼓励多个模型通过相互学习来预测相同表达式语法树，以不同问题表示形式的同一问题，从而捕捉表达式语法树的一致信息并丢弃潜在特定冗余。为了提高模型的泛化能力并生成更多样化的表达，我们设计了一个自我蒸馏损失，以促使模型更依赖于潜在空间中的表达式语法信息。在两个大规模基准测试上的实验结果表明，我们的模型不仅实现了最先进的结果，还生成了更多样化的解决方案。代码可在https://github.com/menik1126/math_ESIB上获得。
英文摘要：
Math Word Problems (MWP) aims to automatically solve mathematical questions given in texts. Previous studies tend to design complex models to capture additional information in the original text so as to enable the model to gain more comprehensive features. In this paper, we turn our attention in the opposite direction, and work on how to discard redundant features containing spurious correlations for MWP. To this end, we design an Expression Syntax Information Bottleneck method for MWP (called ESIB) based on variational information bottleneck, which extracts essential features of expression syntax tree while filtering latent-specific redundancy containing syntax-irrelevant features. The key idea of ESIB is to encourage multiple models to predict the same expression syntax tree for different problem representations of the same problem by mutual learning so as to capture consistent information of expression syntax tree and discard latent-specific redundancy. To improve the generalization ability of the model and generate more diverse expressions, we design a self-distillation loss to encourage the model to rely more on the expression syntax information in the latent space. Experimental results on two large-scale benchmarks show that our model not only achieves state-of-the-art results but also generates more diverse solutions. The code is available in https://github.com/menik1126/math_ESIB.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
