v1: 为多模态 grounded 推理学习指向视觉标记
v1: Learning to Point Visual Tokens for Multimodal Grounded Reasoning
AI总结：
v1通过点对复制机制实现轻量级的主动视觉引用，提升多模态推理中视觉证据与推理空间的对齐性，优于现有基线模型。
摘要：
当人们思考图像时，很少依赖单一视角：他们在推理过程中会反复查看视觉证据。相比之下，大多数多模态语言模型将图像编码一次到键值缓存中，然后仅在文本中推理，这使得难以重新定位中间步骤。我们通过实验证实：随着推理链长度增加，模型逐渐失去对相关区域的关注。我们引入v1，一种通过点对复制实现主动视觉引用的轻量级扩展：模型选择相关图像块并将它们的嵌入回传到推理流中。关键的是，我们的点对复制机制通过语义表示作为键来检索块，确保感知证据与推理空间保持一致。为了训练这种行为，我们构建了v1g数据集，包含300,000个多模态推理轨迹及其交错的 grounding 注释。在多模态数学推理基准测试中，v1始终优于现有基线模型。
英文摘要：
When thinking with images, humans rarely rely on a single glance: they revisit visual evidence while reasoning. In contrast, most Multimodal Language Models encode an image once to key-value cache and then reason purely in text, making it hard to re-ground intermediate steps. We empirically confirm this: as reasoning chains lengthen, models progressively lose focus on relevant regions. We introduce v1, a lightweight extension for active visual referencing via point-and-copy: the model selects relevant image patches and copies their embeddings back into the reasoning stream. Crucially, our point-and-copy mechanism retrieves patches using their semantic representations as keys, ensuring perceptual evidence remains aligned with the reasoning space. To train this behavior, we build v1g, a dataset of 300K multimodal reasoning traces with interleaved grounding annotations. Across multimodal mathematical reasoning benchmarks, v1 consistently outperforms comparable baselines. We release our code, model, and data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
