计算机使用代理的不确定性量化：跨视觉语言模型和GUI基础数据集的基准测试
Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets
AI总结：
提出跨机制基准Argus，评估27种后验不确定性量化方法在4个VLM代理和4个数据集上的表现，发现UQ排名在固定模型内跨数据集稳定，但跨模型类别和接口时下降，隐藏状态和密度方法最稳定。
摘要：
计算机使用代理将视觉语言模型（VLM）的预测转化为可执行的GUI点击，因此可靠的不确定性估计对于拒绝、校准、错误严重性排序和空间安全区域至关重要。然而，关于这些代理的后验不确定性量化（UQ）的证据分散在孤立的模型和数据集对中，尚不清楚当代理、基准或可观察接口发生变化时，UQ排名是否保持稳定。我们提出了Argus，一个用于单步可执行GUI基础中后验UQ的跨机制基准：一个包含27种方法的开放权重矩阵，涵盖4个VLM代理和4个数据集，以及一个包含8种方法的闭源矩阵，涵盖3个前沿供应商，其中logits、隐藏状态和注意力图不可用。评估的方法包括基于logits的分数、采样和一致性度量、隐藏状态和密度估计器（Mahalanobis、SAPLMA）、基于注意力的分数、P(True)和口头化置信度提示，以及分裂共形预测。主要发现是选择性迁移：对于固定模型，UQ排名跨数据集稳定，但跨模型类别和可观察接口时下降。隐藏状态和密度方法是开放权重家族中最稳定的，而CoCoA-1MCA、Focus、基于采样的分数和口头化自我评估在特定机制中获胜。模型内排名迁移很强（Spearman rho高达0.969），但跨层级迁移到闭源供应商平均仅为+0.08，因此闭源UQ应在目标上重新排名而非外推。共形点击区域显示，分数级别的区分度不足以部署：当插件UQ经过校准时，局部加权磁盘的半径缩小40-60%，但在校准-测试或接口不匹配下覆盖率下降。我们发布逐项记录、校准/测试分割、UQ分数和分析脚本，用于GUI代理中机制感知的UQ选择。
英文摘要：
Computer-use agents turn vision-language model (VLM) predictions into executable GUI clicks, so reliable uncertainty estimates are essential for rejection, calibration, miss-severity ranking, and spatial safety regions. Yet evidence on post-hoc uncertainty quantification (UQ) for these agents is fragmented across isolated model and dataset pairs, leaving it unclear whether UQ rankings stay stable when the agent, benchmark, or observable interface changes. We present Argus, a cross-regime benchmark for post-hoc UQ in single-step executable GUI grounding: a 27-method open-weight matrix over 4 VLM agents and 4 datasets, plus an 8-method closed-source matrix across 3 frontier vendors where logits, hidden states, and attention maps are unavailable. Evaluated methods span logit-based scores, sampling and consistency measures, hidden-state and density estimators (Mahalanobis, SAPLMA), attention-based scores, P(True) and verbalised-confidence prompting, and split-conformal prediction. The main finding is selective transfer: UQ rankings are stable across datasets for a fixed model, but degrade across model classes and observable interfaces. Hidden-state and density methods are the most stable open-weight family, while CoCoA-1MCA, Focus, sampling-based scores, and verbalised self-assessment win in specific regimes. Within-model ranking transfer is strong (Spearman rho up to 0.969), but cross-tier transfer to closed-source vendors averages only +0.08, so closed-source UQ should be reranked on the target rather than extrapolated. Conformal click regions show score-level discrimination is not enough for deployment: locally weighted disks shrink radii by 40-60% when the plug-in UQ is calibrated, but coverage degrades under calibration-test or interface mismatch. We release per-item records, calibration/test splits, UQ scores, and analysis scripts for regime-aware UQ selection in GUI agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
