arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.08025cs.CVcs.AI
收藏

MIND:世界模型中内存一致性与动作控制的基准测试

MIND: Benchmarking Memory Consistency and Action Control in World Models

CSU-JPG, Central South University(中南大学);National University of Singapore(新加坡国立大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Nanyang Technological University(南洋理工大学)

Yixuan Ye, Xuanyu Lu, Yuxin Jiang, Yuchao Gu, Rui Zhao, Qiwei Liang, Jiachun Pan, Fengda Zhang, Weijia Wu, Alex Jinpeng Wang

AI总结:

MIND提出首个开放域闭环基准,评估世界模型的内存一致性和动作控制能力,揭示当前模型在长期记忆保持和动作空间泛化上的挑战。

摘要:

世界模型旨在理解和记忆动态视觉环境,但缺乏统一的评估其基本能力的基准。为解决这一差距,我们引入MIND,首个开放域闭环回访基准,用于评估世界模型中的内存一致性与动作控制。MIND包含250个高质量视频,分辨率为1080p,24帧每秒,包括100个(第一人称)+ 100个(第三人称)视频片段,在共享动作空间下以及在不同动作空间中覆盖八个多样化场景。我们设计了一个高效的评估框架,以测量两个核心能力:内存一致性和动作控制,捕捉不同视角下的时间稳定性和上下文一致性。此外,我们设计了各种动作空间,包括不同的角色移动速度和摄像机旋转角度,以评估在共享场景下不同动作空间中的动作泛化能力。为了促进未来在MIND上的性能基准测试,我们引入了MIND-World,一个新颖的视频到世界基线。广泛的实验展示了MIND的完整性,并揭示了当前世界模型中的关键挑战,包括维持长期内存一致性和在不同动作空间中泛化的能力。代码:https://github.com/CSU-JPG/MIND。

英文摘要:

World models aim to understand, remember, and predict dynamic visual environments, yet a unified benchmark for evaluating their fundamental abilities remains lacking. To address this gap, we introduce MIND, the first open-domain closed-loop revisited benchmark for evaluating Memory consIstency and action coNtrol in worlD models. MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 (first-person) + 100 (third-person) video clips under a shared action space and 25 + 25 clips across varied action spaces covering eight diverse scenes. We design an efficient evaluation framework to measure two core abilities: memory consistency and action control, capturing temporal stability and contextual coherence across viewpoints. Furthermore, we design various action spaces, including different character movement speeds and camera rotation angles, to evaluate the action generalization capability across different action spaces under shared scenes. To facilitate future performance benchmarking on MIND, we introduce MIND-World, a novel interactive Video-to-World baseline. Extensive experiments demonstrate the completeness of MIND and reveal key challenges in current world models, including the difficulty of maintaining long-term memory consistency and generalizing across action spaces. Code: https://github.com/CSU-JPG/MIND.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑