动作条件双模拟用于GUI智能体记忆

中文摘要 AI 辅助

一个记住自己在网页上做了什么操作的智能体，必须决定何时两个页面算作相同。基于观测相似性的记忆会将看起来相似但行为不同的页面合并，而图形用户界面（GUI）中充满了这样的页面：一个小部件的两个标签页或一个菜单的两行，对相同的点击会给出不同的响应。我们将合并规则定义为动作条件下的双模拟关系，作用于一个冻结的智能体在行动过程中填充的经验预测状态图。仅当两个状态的共享动作导致一致的结果和基于可供性标签的后继块时，它们才会合并。观测相似性从不进入该规则，且无需任何训练。该规则取代了现有结果值记忆中的合并规则，因此闭环比较可以将其隔离出来。在MiniWoB++上，它相对于无记忆智能体提高了成功率，而采用相同探索性绕行的对照组、先前的后继表示合并以及无动作条件的相同准则均未带来任何改变。

英文摘要

An agent that remembers what it did on a web page must decide when two pages count as the same. Memories built on observation similarity merge pages that look alike but behave differently, and GUIs are full of such pages: two tabs of one widget or two rows of one menu answer the same click differently. We define the merge rule as an action-conditioned bisimulation over the empirical predictive state graph a frozen agent fills as it acts. Two states merge only when their shared actions lead to agreeing outcomes and successor blocks under an affordance label. Observation similarity never enters the rule, and nothing is trained. It replaces the merge rule of an existing outcome-value memory, so a closed-loop comparison isolates it. On MiniWoB++ it raises success rate over a memoryless agent, while a control taking identical exploratory detours, the prior successor-representation merge, and the same criterion without action conditioning change nothing.

发表机构

• Peking University（北京大学）
• East China Normal University（华东师范大学）
• University of Southern California（南加州大学）

机构由 AI 辅助整理，请以论文原文为准。