GUI不是状态：诊断GUI世界模型中的状态别名问题
The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models
浏览论文内容
中文总结 AI 辅助
本研究提出StateAliasBench基准诊断GUI世界模型的状态别名问题，并通过预测状态恢复方法增强模型，显著提升状态敏感预测与智能体下游性能。
中文摘要 AI 辅助
GUI世界模型（GUI-WMs）越来越多地被用于预测智能体规划和模拟的未来状态，然而现有的大多数公式仅基于当前的GUI观察和动作进行条件设定。我们识别出状态别名问题，即可见界面遗漏了与转换相关的环境状态，导致相同的可观察条件可能对应不同的有效未来。为了诊断这一失败模式，我们引入了StateAliasBench，一个通过严格配对显式隔离此类歧义的诊断基准。我们进一步提出了轻量级预测状态恢复方法，该方法从历史中推断结构化状态，并通过确定性状态接口增强原本冻结的GUI-WMs。针对不同状态类型的家族特定专家提供状态恢复，多教师蒸馏将其整合为单一统一估计器。实验表明，现有的GUI-WMs在仅观察条件下表现出系统性失败，而预测状态增强在评估的WMs中显著恢复了状态敏感预测，保持了生成保真度，并提高了GUI智能体在AndroidWorld上的下游性能。这些结果表明，可靠的GUI世界建模不仅应考虑可见内容，还应考虑决定下一步发生的隐藏转换状态。
英文摘要
GUI World Models (GUI-WMs) are increasingly used to predict future states for agent planning and simulation, yet most existing formulations condition only on the current GUI observation and action. We identify state aliasing, where the vis- ible interface omits transition-relevant environment state, so identical observable conditions can correspond to different valid futures. To diagnose this failure mode, we introduce StateAliasBench, a diagnostic benchmark that explicitly isolates such ambiguities via strict pairing. We further propose lightweight predictive- state recovery that infers structured state from history and augments otherwise frozen GUI-WMs through a deterministic state interface. Family-specific special- ists provide state recovery across heterogeneous state types, and multi-teacher dis- tillation consolidates them into a single unified estimator. Experiments show that existing GUI-WMs exhibit systematic failures under observation-only condition- ing, while predictive-state augmentation substantially restores state-sensitive pre- diction across evaluated WMs, preserves generative fidelity, and improves down- stream performance of GUI agents on AndroidWorld. These results suggest that reliable GUI world modeling should account not only for what is visible, but also for the hidden transition state that determines what happens next.
