arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2611.26366cs.AI
收藏

世界模型更偏好学习到的先验而非视觉证据

World Models Favor Learned Priors over Visual Evidence

Xiamen University(厦门大学);Northeastern University(东北大学);Tongji University(同济大学)

Han Xiao, Zhiyong Bao, Xiaotong Liu, Yanhao Chen, Qingqiang Wu

AI总结:

该研究针对世界模型跨环境预测的长尾训练分布偏差问题,提出RIFT方法,引入TailWorld基准验证其可提升罕见动态预测的准确性与物理一致性。

摘要:

世界模型越来越多地用于模拟物理交互,但为每个环境单独训练一个模型会限制其可扩展性。跨环境预测需要识别当前的物理规则,仅外观可能无法揭示该规则。因此,我们研究视觉校准:利用短视频中的物体运动和交互作为直接证据,推断当前规则并从不同初始状态预测该规则下的运动。然而,不同物理条件下的数据可用性差异会导致长尾训练分布,使模型偏向常见规则,即使证据支持罕见规则。为研究这一挑战,我们引入TailWorld,该基准评估已演示规则在不同初始状态下的迁移,同时独立控制训练不平衡和校准信息性。它通过比较同一预测初始状态下的两个参考未来(一个遵循已演示规则,另一个遵循常见规则)来测量预测对常见动态的偏差。我们进一步提出迁移规则推理(RIFT),该方法学习不同规则对物体运动的影响,从校准交互中推断当前规则,并将其应用于新初始状态的预测。RIFT明确控制训练频率对规则推理的影响,使足够信息的观测证据能支持罕见规则。在TailWorld上的实验表明,RIFT在不同训练不平衡和分布偏移下均提高了预测准确性和物理一致性,尤其在预测罕见动态时增益显著。

英文摘要:

World models are increasingly used to simulate physical interactions, but training a separate model for each environment limits scalability. Cross-environment prediction requires identifying the current physical rule, which appearance alone may not reveal. We therefore study visual calibration: using object motion and interactions in a short video as direct evidence to infer the current rule and predict motion under that rule from a different initial state. However, differences in data availability across physical conditions can yield a long-tailed training distribution, biasing models toward common rules even when evidence supports rare ones. To study this challenge, we introduce TailWorld, which evaluates the transfer of demonstrated rules across initial states while independently controlling training imbalance and calibration informativeness. The benchmark compares predictions with two reference futures from the same prediction initial state, one following the demonstrated rule and the other a common rule, to measure prediction bias toward common dynamics. We further propose Rule Inference for Transfer (RIFT), which learns how different rules affect object motion, infers the current rule from calibration interactions, and applies it to prediction from a new initial state. RIFT explicitly controls the influence of training frequency on rule inference, allowing sufficiently informative observational evidence to support rare rules. Experiments on TailWorld show that RIFT improves prediction accuracy and physical consistency across varying training imbalances and distribution shifts, with gains in predicting rare dynamics.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑