一个数字值多少像素？用于GUI智能体置信度估计的位置感知坐标熵
How Many Pixels Is a Digit Worth? Place-Aware Coordinate Entropy for GUI Agent Confidence Estimation
浏览论文内容
中文总结 AI 辅助
针对GUI智能体点击坐标置信度估计，提出位置感知坐标熵（PACE），按位值加权数字熵，在ScreenSpot基准上以单次前向传播超越K样本方法。
中文摘要 AI 辅助
GUI智能体将点击坐标预测为数字令牌序列，但标准的文本大语言模型置信度估计方法只能弱地区分正确点击与错误点击。针对GUI的替代方法使用K个样本或新的监督信号，但仍留有改进空间。我们将部分原因归结为位值不对称性：边界框的正确性往往使得高位数字比低位数字更重要，因此统一聚合削弱了决定正确性的信号。解决方案是按位值对每个数字的香农熵进行加权。我们将其称为位置感知坐标熵（PACE）。在ScreenSpot-Pro和ScreenSpot-v2上，针对固定规模智能体，PACE在单次前向传播中在所有主要比较中同时赢得AUROC和选择性准确率，以一小部分成本匹配或超越K样本基线。PACE提供了每次点击的置信度估计，将坐标令牌内部信息转化为实用的置信度信号，用于GUI智能体的部署。
英文摘要
GUI agents predict click coordinates as digit-token sequences, but standard text-LLM confidence estimation methods rank correct clicks from wrong ones only weakly. GUI-specific alternatives use K samples or new supervision, but still leave room for improvement. We trace part of this to place-value asymmetry: bounding-box correctness often makes higher-place digits more important than lower-place digits, so uniform aggregation weakens the signal that determines correctness. The fix is to weight each digit's Shannon entropy by its place value. We call this Place-Aware Coordinate Entropy (PACE). Across fixed-scale agents on ScreenSpot-Pro and ScreenSpot-v2, PACE wins both AUROC and selective accuracy on all primary comparisons in a single forward pass, matching or outperforming K-sample baselines at a fraction of the cost. PACE provides a per-click confidence estimate that turns coordinate-token internals into a practical confidence signal for GUI agent deployment.
发表机构
- The Chinese University of Hong Kong（香港中文大学）
机构由 AI 辅助整理，请以论文原文为准。
