一种具备从图中解析文本子句能力的多模态神经几何求解器
A Multi-Modal Neural Geometric Solver with Textual Clauses Parsed from Diagram
AI总结:
针对几何问题求解中神经求解器的图表表征与模态融合短板,提出将图表转文本子句的PGPSNet,结合预训练等技术提升推理能力,构建PGPS9K数据集,实验证明其性能优于现有最优求解器。
摘要:
几何问题求解(GPS)是一项需要多模态融合与几何知识应用能力的高级数学推理任务。近年来,神经求解器在GPS领域展现出巨大潜力,但在图表表征与模态融合方面仍存在不足。在本研究中,我们将图表转换为基础文本子句以有效描述图表特征,并提出一种名为PGPSNet的新型神经求解器,用于高效融合多模态信息。通过结合结构与语义预训练、数据增强以及自限制解码,PGPSNet被赋予了丰富的几何定理知识与几何表征能力,从而提升了几何理解与推理水平。此外,为推动GPS领域的研究,我们构建了一个全新的大规模、精细标注的GPS数据集,命名为PGPS9K,该数据集同时标注了细粒度图表注释与可解释求解程序。在PGPS9K与现有数据集Geometry3K上的实验验证了,我们的方法相较于当前最优的神经求解器具有更优的性能。我们的代码、数据集与附录材料可在https://github.com/mingliangzhang2018/PGPS 获取。
英文摘要:
Geometry problem solving (GPS) is a high-level mathematical reasoning requiring the capacities of multi-modal fusion and geometric knowledge application. Recently, neural solvers have shown great potential in GPS but still be short in diagram presentation and modal fusion. In this work, we convert diagrams into basic textual clauses to describe diagram features effectively, and propose a new neural solver called PGPSNet to fuse multi-modal information efficiently. Combining structural and semantic pre-training, data augmentation and self-limited decoding, PGPSNet is endowed with rich knowledge of geometry theorems and geometric representation, and therefore promotes geometric understanding and reasoning. In addition, to facilitate the research of GPS, we build a new large-scale and fine-annotated GPS dataset named PGPS9K, labeled with both fine-grained diagram annotation and interpretable solution program. Experiments on PGPS9K and an existing dataset Geometry3K validate the superiority of our method over the state-of-the-art neural solvers. Our code, dataset and appendix material are available at \url{https://github.com/mingliangzhang2018/PGPS}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。