迈向人工通用教师：基于视觉-语言模型的程序几何数据生成与视觉语义
Toward an Artificial General Teacher: Procedural Geometry Data Generation and Visual Grounding with Vision-Language Models
AI总结：
本文研究几何教育中的视觉解释问题，提出通过程序生成几何图示数据，结合视觉-语言模型进行领域特定微调，提升几何元素分割性能，并引入几何感知的评估指标。
摘要：
我们研究几何教育中的视觉解释问题，将其作为指代图像分割（RIS）任务：给定一个图表和自然语言描述，任务是生成所指几何元素的像素级掩码。然而，现有基于自然图像基准（如RefCOCO）训练的RIS模型在几何图表上表现糟糕，因为摄影场景与抽象、无纹理的图表之间存在根本性的领域转移。为解决缺乏合适训练数据的问题，我们提出一个完全自动化的程序数据引擎，生成超过200,000个合成几何图表，具有像素级分割掩码和语言多样性的指代表达，无需人工标注。我们进一步提出对视觉-语言模型（VLMs）的领域特定微调，证明经过微调的Florence-2在IoU上达到49%和85%的Buffered IoU（BIoU），相比零样本设置下的<1% IoU。我们引入Buffered IoU，一种考虑薄结构定位的几何感知评估指标，并展示其比标准IoU更能反映真实的分割质量。我们的结果为构建能够提供视觉基础、逐步解释几何问题的人工通用教师（AGTs）奠定了基础。
英文摘要：
We study visual explanation in geometry education as a Referring Image Segmentation (RIS) problem: given a diagram and a natural language description, the task is to produce a pixel-level mask for the referred geometric element. However, existing RIS models trained on natural image benchmarks such as RefCOCO fail catastrophically on geometric diagrams due to the fundamental domain shift between photographic scenes and abstract, textureless schematics. To address the absence of suitable training data, we present a fully automated procedural data engine that generates over 200,000 synthetic geometry diagrams with pixel-perfect segmentation masks and linguistically diverse referring expressions, requiring zero manual annotation. We further propose domain-specific fine-tuning of vision-language models (VLMs), demonstrating that a fine-tuned Florence-2 achieves 49% IoU and 85% Buffered IoU (BIoU), compared to <1% IoU in zero-shot settings. We introduce Buffered IoU, a geometry-aware evaluation metric that accounts for thin-structure localization, and show that it better reflects true segmentation quality than standard IoU. Our results establish a foundation for building Artificial General Teachers (AGTs) capable of providing visually grounded, step-by-step explanations of geometry problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
