图形形式化增强的多模态几何问题求解器
Diagram Formalization Enhanced Multi-Modal Geometry Problem Solver
AI总结:
针对MLLM难以理解几何图形的问题,本文提出DFE-GPS框架,融合视觉特征、几何形式语言与自然语言,并构建SynthGeo228K数据集增强视觉编码器,提升几何推理及开放式任务表现。
摘要:
数学推理对AI模型而言仍是一项持续挑战,尤其是对于同时需要语言和视觉信号的几何问题。由于大多数MLLM的视觉编码器是在自然场景上训练的,它们往往难以理解几何图形,在几何问题求解中的表现并不比仅处理文本的LLM更好。缺乏表示几何关系的有效方法进一步放大了这一局限。为解决这些问题,我们提出图形形式化增强几何问题求解器(DFE-GPS),这是一个整合视觉特征、几何形式语言和自然语言表示的新框架。我们提出一种新颖的合成数据方法,并创建了大规模几何数据集SynthGeo228K,该数据集同时标注有形式语言和自然语言描述,旨在增强视觉编码器以更好地理解几何结构。我们的框架提升了MLLM处理几何图形的能力,并将其应用扩展到formalgeo7k数据集上的开放式任务。
英文摘要:
Mathematical reasoning remains an ongoing challenge for AI models, especially for geometry problems that require both linguistic and visual signals. As the vision encoders of most MLLMs are trained on natural scenes, they often struggle to understand geometric diagrams, performing no better in geometry problem solving than LLMs that only process text. This limitation is amplified by the lack of effective methods for representing geometric relationships. To address these issues, we introduce the Diagram Formalization Enhanced Geometry Problem Solver (DFE-GPS), a new framework that integrates visual features, geometric formal language, and natural language representations. We propose a novel synthetic data approach and create a large-scale geometric dataset, SynthGeo228K, annotated with both formal and natural language captions, designed to enhance the vision encoder for a better understanding of geometric structures. Our framework improves MLLMs' ability to process geometric diagrams and extends their application to open-ended tasks on the formalgeo7k dataset.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。