arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.01985cs.LGcs.AIcs.RO
收藏

World Action Verifier: 通过前向-反向不对称性自我改进世界模型

World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry

Stanford University(斯坦福大学);UC San Diego(加州大学圣地亚哥分校);Carnegie Mellon University(卡内基梅隆大学);Google DeepMind(谷歌DeepMind);Harvard University(哈佛大学)

Yuejiang Liu, Fan Feng, Lingjing Kong, Weifeng Lu, Jinzhou Tang, Kun Zhang, Kevin Murphy, Chelsea Finn, Yilun Du

AI总结:

提出World Action Verifier (WAV)框架,利用状态合理性和动作可达性的独立验证以及前向-反向不对称性,通过视频语料库的多样子目标生成器和稀疏逆模型实现循环一致性,从而在欠探索区域自我改进世界模型,在多个任务中样本效率提升2倍且下游策略性能提升22%以上。

摘要:

通用世界模型有望实现可扩展的策略评估、优化和规划,但达到所需的鲁棒性仍然具有挑战性。与主要关注最优动作的策略学习不同,世界模型需要在大量次优动作的空间中保持可靠,而这些动作在带有动作标签的机器人交互中往往代表性不足。为了解决这一挑战,我们提出了World Action Verifier (WAV)框架,该框架使世界模型能够识别自身的预测错误并进行自我改进。关键思想是将动作条件的状态预测分解为两个独立可验证的因素:状态合理性和动作可达性。我们证明,由于两个潜在的不对称性——更广泛的无动作数据的可用性和动作相关特征的更低维度——验证这些因素比直接前向预测更容易处理。利用这些不对称性,我们通过(i)从视频语料库中获得的多样子目标生成器和(ii)从状态特征子集推断动作的稀疏逆模型来增强世界模型。通过强制提议的子目标、推断的动作和前向展开之间的循环一致性,WAV在现有方法常常失败的欠探索区域提供了一种有效的验证机制。在涵盖MiniGrid、RoboMimic和ManiSkill的九个任务中,我们的方法实现了2倍的样本效率提升,同时将下游策略性能提高了22%以上。

英文摘要:

General-purpose world models promise scalable policy evaluation, optimization, and planning, yet achieving the required level of robustness remains challenging. Unlike policy learning which primarily focuses on optimal actions, a world model needs to be reliable over a vast space of suboptimal actions, which are often underrepresented in action-labeled robot interactions. To address this challenge, we propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve. The key idea is to decompose action-conditioned state prediction into two independently verifiable factors: state plausibility and action reachability. We show that verifying these factors is significantly more tractable than direct forward prediction due to two underlying asymmetries: the broader availability of action-free data and the lower dimensionality of action-relevant features. Leveraging these asymmetries, we augment a world model with (i) a diverse subgoal generator obtained from video corpora and (ii) a sparse inverse model that infers actions from a subset of state features. By enforcing cycle consistency among proposed subgoals, inferred actions, and forward rollouts, WAV provides an effective verification mechanism in under-explored regimes, where existing methods often fail. Across nine tasks spanning MiniGrid, RoboMimic, and ManiSkill, our method achieves 2x higher sample efficiency while improving downstream policy performance by over 22%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑