理解GUI智能体定位偏差：基于Logit锐度分析
Understanding GUI Agent Localization Biases through Logit Sharpness
AI总结：
本研究提出细粒度评估框架和峰值锐度分数（PSS）指标，并引入无需训练的上下文感知裁剪技术，以识别和缓解多模态大语言模型在GUI智能体中的系统性定位幻觉，提升其可解释性与鲁棒性。
摘要：
多模态大语言模型（MLLMs）使GUI智能体能够通过将语言转化为空间动作来与操作系统交互。尽管其性能表现令人期待，但这些模型经常表现出幻觉——系统性的定位错误，这损害了其可靠性。我们提出了一种细粒度的评估框架，将模型预测分为四种不同类型，揭示了超越传统准确率指标的细微失败模式。为了更好地量化模型不确定性，我们引入了峰值锐度分数（PSS），该指标评估坐标预测中语义连续性与logits分布之间的对齐程度。基于这一见解，我们进一步提出了上下文感知裁剪（Context-Aware Cropping），这是一种无需训练的技术，通过自适应地细化输入上下文来提升模型性能。大量实验表明，我们的框架和方法提供了可操作的见解，并增强了GUI智能体行为的可解释性和鲁棒性。
英文摘要：
Multimodal large language models (MLLMs) have enabled GUI agents to interact with operating systems by grounding language into spatial actions. Despite their promising performance, these models frequently exhibit hallucinations-systematic localization errors that compromise reliability. We propose a fine-grained evaluation framework that categorizes model predictions into four distinct types, revealing nuanced failure modes beyond traditional accuracy metrics. To better quantify model uncertainty, we introduce the Peak Sharpness Score (PSS), a metric that evaluates the alignment between semantic continuity and logits distribution in coordinate prediction. Building on this insight, we further propose Context-Aware Cropping, a training-free technique that improves model performance by adaptively refining input context. Extensive experiments demonstrate that our framework and methods provide actionable insights and enhance the interpretability and robustness of GUI agent behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
