SPECTRA：面向自主边缘-云GUI接地（GUI Grounding）的端侧认知扰动与轨迹分析
SPECTRA: On-Device Cognitive Perturbation and Trajectory Analysis for Autonomous Edge-Cloud GUI Grounding
浏览论文内容
中文总结 AI 辅助
针对边缘代理在GUI接地中因过度自信幻觉导致请求不可靠的问题，提出SPECTRA框架，通过显著性引导扰动和认知轨迹分析实现无需解码的云端请求评估，在保持高比例云端性能的同时显著降低请求率。
中文摘要 AI 辅助
边缘-云协作在GUI接地（GUI Grounding）中的有效性依赖于自主请求机制，即边缘代理有选择地将复杂任务卸载至强大的云端。然而，在视觉密集场景中，轻量级边缘代理常表现出过度自信的幻觉，导致置信度与准确性之间出现错位，从而阻碍了可靠的自主请求。为解决此问题，我们利用一个观察：由于决策边界陡峭，代理的认知不稳定性会在微小扰动下引发显著的潜在漂移。我们提出SPECTRA，一种用于边缘-云GUI接地的轻量级自主请求框架，包含：（1）显著性引导的目标扰动（Saliency-Guided Targeted Perturbation）和（2）高效认知轨迹分析（Efficient Cognitive Trajectory Analysis）。SPECTRA通过向关键视觉锚点注入掩码进行视觉认知压力测试，并量化代理在前填充（prefill）阶段高维认知轨迹的拓扑发散，从而避免低效的输出解码。实验表明，SPECTRA无需自回归解码即可进行云端请求评估。我们的GTA1-32B+InfiGUI-G1-3B和GTA1-32B+Holo1.5-3B分别保持了云端性能的93.44%和95.60%，平均请求率分别为37.58%和39.24%。
英文摘要
The effectiveness of edge-cloud collaboration for GUI grounding depends on autonomous requesting, where the edge agent selectively offloads complex tasks to the powerful cloud. However, in visually dense scenarios, lightweight edge agents often exhibit overconfident hallucinations, leading to a misalignment between confidence and accuracy that hinders reliable autonomous requesting. To address this, we leverage the observation that an agent's cognitive instability leads to significant latent drift under minute perturbations due to steep decision boundaries. We propose SPECTRA, a lightweight autonomous request framework for edge-cloud GUI grounding, comprising (1) Saliency-Guided Targeted Perturbation and (2) Efficient Cognitive Trajectory Analysis. SPECTRA conducts a visual cognitive stress test by injecting masks into critical visual anchors and quantifies the topological divergence of the agent's high-dimensional cognitive trajectories during the prefill phase, avoiding inefficient output decoding. Experiments demonstrate that SPECTRA performs cloud request assessment without autoregressive decoding. Our GTA1-32B+InfiGUI-G1-3B and GTA1-32B+Holo1.5-3B maintain 93.44% and 95.60% of cloud-only performance with average request rates of 37.58% and 39.24%, respectively.
发表机构
- Zhejiang University（浙江大学）
- Huawei Technologies Co., Ltd.（华为技术有限公司）
机构由 AI 辅助整理，请以论文原文为准。
