一次前向胜过两次：InnerZoom 实现准确高效的 GUI 定位
One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
AI总结：
提出 InnerZoom 框架，通过跨层证据桥接在单次前向传播中实现精确 GUI 元素定位，无需额外裁剪重跑，在六个基准上达到最优性能，同时降低延迟和计算量。
摘要：
基于 MLLM 的 GUI 定位方法通常将目标定位建模为自回归坐标生成，使模型能够利用 MLLM 强大的指令遵循和语义理解能力。然而，这种公式要求模型在解码坐标令牌时保留区域级目标证据，以满足 GUI 点击所需的空间精度。我们的诊断分析表明，目标区域感知出现在中间解码器层，但既未保留也未转化为最终的坐标预测。现有的 ZoomIn 风格方法通过外部裁剪重跑过程解决此问题，这提高了定位精度，但增加了端到端延迟和计算成本。为了在不增加额外成本的情况下保留两遍缩放方法的精度优势，我们提出了 InnerZoom，一种用于跨层证据桥接的单次前向框架。InnerZoom 将原始前向传播中的目标相关线索转换为紧凑的跨层证据状态，然后在后续解码层中保留、精炼并重新注入该状态以指导坐标预测。大量实验结果表明，InnerZoom-4B 在所有六个 GUI 定位基准上均达到了最先进的性能，在 OSWorld-G 上获得 64.7，在 UI-Vision 上获得 40.2，在 OSWorld-GR 上获得 73.1，在 MMBench-GUI 上获得 87.6，分别比之前的最佳结果高出 4.1、3.2、2.9 和 2.3 分。在受控的 4B 设置下，InnerZoom 将相同的 SFT+RL 基线平均提高了 5.3 分，并平均优于两遍 ZoomIn 1.3 分，同时将端到端延迟降低高达 31.8%，TFLOPs 降低约 29%。代码和模型将公开提供。
英文摘要：
MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage the strong instruction-following and semantic understanding capabilities of MLLMs. However, this formulation requires the model to retain region-level target evidence while decoding coordinate tokens with the spatial precision demanded by GUI clicking. Our diagnostic analysis reveals that target-region awareness emerges in intermediate decoder layers but is neither retained nor translated into the final coordinate prediction. Existing ZoomIn-style methods address this issue through an external crop-and-rerun pass, which improves localization but increases end-to-end latency and computational cost. To retain the accuracy benefits of two-pass zooming without this extra cost, we propose InnerZoom, a single-forward framework for cross-layer evidence bridging. InnerZoom transforms target-related cues from the original forward pass into a compact cross-layer evidence state, then preserves, refines, and reinjects this state throughout later decoding layers to guide coordinate prediction. Extensive experimental results suggest that InnerZoom-4B achieves state-of-the-art performance on all six GUI grounding benchmarks, obtaining 64.7 on OSWorld-G, 40.2 on UI-Vision, 73.1 on OSWorld-GR, and 87.6 on MMBench-GUI, surpassing the previous best results by 4.1, 3.2, 2.9, and 2.3 points, respectively. Under a controlled 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass ZoomIn by 1.3 points on average, while reducing end-to-end latency by up to 31.8% and TFLOPs by about 29%. Code and models will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
