arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.02697cs.CVcs.AI
收藏

ShareVerse:面向共享世界建模的多智能体一致视频生成

ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modeling

Shanghai Jiao Tong University China(上海交通大学中国);Fudan University China(复旦大学中国);StepFun China(StepFun中国)

Jiayi Zhu, Jianing Zhang, Yiying Yang, Wei Cheng, Xiaoyun Yuan

AI总结:

提出ShareVerse框架,通过构建多智能体交互数据集、空间拼接策略和跨智能体注意力机制,实现多智能体共享世界的一致视频生成。

摘要:

本文提出ShareVerse,一个视频生成框架,支持多智能体共享世界建模,解决了现有工作缺乏统一共享世界构建和多智能体交互支持的问题。ShareVerse利用大型视频模型的生成能力,并整合了三个关键创新:1)在CARLA仿真平台上构建了大规模多智能体交互世界建模数据集,包含多样场景、天气条件和交互轨迹,以及配对的每智能体四视角视频(前/后/左/右视图)和相机数据。2)我们提出了一种针对独立智能体四视角视频的空间拼接策略,以建模更广泛的环境并确保内部多视角几何一致性。3)我们将跨智能体注意力模块集成到预训练视频模型中,实现跨智能体时空信息的交互传递,保证重叠区域的共享世界一致性和非重叠区域的合理生成。支持49帧大规模视频生成的ShareVerse能够准确感知动态智能体的位置,实现一致的共享世界建模。

英文摘要:

This paper presents ShareVerse, a video generation framework enabling multi-agent shared world modeling, addressing the gap in existing works that lack support for unified shared world construction with multi-agent interaction. ShareVerse leverages the generation capability of large video models and integrates three key innovations: 1) A dataset for large-scale multi-agent interactive world modeling is built on the CARLA simulation platform, featuring diverse scenes, weather conditions, and interactive trajectories with paired multi-view videos (front/ rear/ left/ right views per agent) and camera data. 2) We propose a spatial concatenation strategy for four-view videos of independent agents to model a broader environment and to ensure internal multi-view geometric consistency. 3) We integrate cross-agent attention blocks into the pretrained video model, which enable interactive transmission of spatial-temporal information across agents, guaranteeing shared world consistency in overlapping regions and reasonable generation in non-overlapping regions. ShareVerse, which supports 49-frame large-scale video generation, accurately perceives the position of dynamic agents and achieves consistent shared world modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑