RECODE: 通过代码生成进行视觉问答的推理
RECODE: Reasoning Through Code Generation for Visual Question Answering
AI总结：
RECODE通过代码生成实现视觉问答的可验证推理，优于传统方法。
摘要：
多模态大语言模型（MLLMs）在处理结构化视觉如图表和图表时难以进行精确推理，因为基于像素的感知缺乏验证机制。为了解决这个问题，我们提出利用去渲染——将视觉反向工程为可执行代码——作为一种新的可验证视觉推理模式。具体而言，我们提出了RECODE，一个代理框架，首先生成多个候选程序以重现输入图像。然后使用批评者选择最忠实的重建，并迭代地优化代码。这一过程不仅将模糊的感知任务转化为可验证的符号问题，还使后续的精确计算和逻辑推理成为可能。在各种视觉推理基准，如CharXiv、ChartQA和Geometry3K上，RECODE显著优于不利用代码或仅用代码绘制辅助线或裁剪的方法。我们的工作证明，将视觉感知 grounding 在可执行代码中为更准确和可验证的多模态推理提供了一条新路径。
英文摘要：
Multimodal Large Language Models (MLLMs) struggle with precise reasoning for structured visuals like charts and diagrams, as pixel-based perception lacks a mechanism for verification. To address this, we propose to leverage derendering -- the process of reverse-engineering visuals into executable code -- as a new modality for verifiable visual reasoning. Specifically, we propose RECODE, an agentic framework that first generates multiple candidate programs to reproduce the input image. It then uses a critic to select the most faithful reconstruction and iteratively refines the code. This process not only transforms an ambiguous perceptual task into a verifiable, symbolic problem, but also enables precise calculations and logical inferences later on. On various visual reasoning benchmarks such as CharXiv, ChartQA, and Geometry3K, RECODE significantly outperforms methods that do not leverage code or only use code for drawing auxiliary lines or cropping. Our work demonstrates that grounding visual perception in executable code provides a new path toward more accurate and verifiable multimodal reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
