SCA：GUI代理强化学习的空间信用分配
SCA: Spatial Credit Assignment for Reinforcement Learning of GUI Agents
- Southeast University（东南大学）
- Kuaishou Technology（快手科技）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对GUI代理强化学习中失败点击缺乏相对信号的问题，提出利用点击坐标的空间信用分配（SCA），通过预测残差或距离排序细化信用，提升接地与动作预测性能。
AI中文摘要：
GUI代理通过将语言指令锚定在视觉界面中，来自动化数字设备上的任务。现有的组相对强化学习通过比较从同一GUI状态采样的多个响应的奖励，来改进GUI动作预测。然而，二元评估将空间上不同的失败点击视为相同，并且在所有采样点击都失败时不提供相对信号。为解决这些局限，我们提出空间信用分配（SCA），它利用采样点击的屏幕坐标来细化组相对信用。具体而言，SCA在包含成功和失败的组中，利用其他响应预测每个留出响应的奖励，然后使用预测残差调整信用。当所有采样点击都失败时，SCA则根据它们与标注目标的距离进行排序。这些空间参考仅用于构建训练更新；部署的策略保持不变。我们通过在有控制的合成研究中，将其误差和方向对齐与精确回报梯度进行比较，来评估该修正是否改进了策略更新本身。在GUI接地和离线动作预测基准上，SCA提升了跨专业领域的接地性能，并在大多数动作预测指标上取得了强化微调模型中最强的结果，在报告的GUI套件中保持一致增益。
英文摘要：
GUI agents automate tasks on digital devices by grounding language instructions in visual interfaces. Existing group-relative reinforcement learning improves GUI action prediction by comparing the rewards of multiple responses sampled from the same GUI state. However, binary evaluation treats spatially different failed clicks as identical and provides no relative signal when all sampled clicks fail. To address these limitations, we propose Spatial Credit Assignment (SCA), which uses the screen coordinates of sampled clicks to refine group-relative credit. Specifically, SCA predicts each held-out response's reward from the other responses in groups containing both successes and failures, then uses the prediction residual to adjust credit. When all sampled clicks fail, SCA instead orders them by distance to the annotated target. These spatial references are used only to construct the training update; the deployed policy remains unchanged. We evaluate whether this correction improves the policy update itself by comparing its error and directional alignment with the exact return gradient in a controlled synthetic study. Across GUI grounding and offline action-prediction benchmarks, SCA improves grounding across professional domains and achieves the strongest results among reinforcement-fine-tuned models on most action-prediction metrics, with consistent gains across the reported GUI suites.
