多模态思维模型中的实时视觉归因流
Real-Time Visual Attribution Streaming in Thinking Model
AI总结：
本文提出了一种用于多模态思维模型实时视觉归因流的近似框架，通过学习从注意力特征中直接估计语义区域的因果效应，实现轻量级学习而非暴力计算的高保真归因。
摘要：
我们提出了一种用于多模态思维模型中实时视觉归因流的近似框架。当这些模型从截图生成代码或从图像解决数学问题时，其长推理轨迹应基于视觉证据。然而，验证这种依赖关系具有挑战性：忠实的因果方法需要成本高昂的重复反向传递或扰动，而原始注意力图提供即时访问，但缺乏因果有效性。为了解决这一问题，我们引入了一种近似方法，该方法直接从注意力特征中丰富的信号中学习估计语义区域的因果效应。在五个多样化的基准和四个思维模型上，我们的方法在保真度上与穷尽因果方法相当，同时实现了视觉归因流，用户在模型推理过程中观察到归因证据，而非事后。我们的结果表明，通过轻量级学习而非暴力计算，多模态思维模型中的实时忠实归因是可行的。
英文摘要：
We present an amortized framework for real-time visual attribution streaming in multimodal thinking models. When these models generate code from a screenshot or solve math problems from images, their long reasoning traces should be grounded in visual evidence. However, verifying this reliance is challenging: faithful causal methods require costly repeated backward passes or perturbations, while raw attention maps offer instant access, they lack causal validity. To resolve this, we introduce an amortized approach that learns to estimate the causal effects of semantic regions directly from the rich signals encoded in attention features. Across five diverse benchmarks and four thinking models, our approach achieves faithfulness comparable to exhaustive causal methods while enabling visual attribution streaming, where users observe grounding evidence as the model reasons, not after. Our results demonstrate that real-time, faithful attribution in multimodal thinking models is achievable through lightweight learning, not brute-force computation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
