世界是你的画布:通过参考图像、轨迹和文本绘画可提示事件
The World is Your Canvas: Painting Promptable Events with Reference Images, Trajectories, and Text
AI总结:
WorldCanvas通过结合文本、轨迹和参考图像,实现可提示的多代理交互事件生成,提升世界模型的交互性和可控性。
摘要:
我们提出了WorldCanvas,一个可提示世界事件的框架,通过结合文本、轨迹和参考图像,实现丰富的用户主导模拟。与仅依赖文本的方法和现有的基于轨迹的图像到视频方法不同,我们的多模态方法将轨迹——编码运动、时间和可见性——与自然语言结合,用于语义意图,以及参考图像用于对象身份的视觉定位,从而生成连贯且可控的事件,包括多代理交互、对象进入/退出、参考引导的外观和反直觉事件。生成的视频不仅在时间上保持一致,还表现出涌现的一致性,在对象身份和场景暂时消失的情况下仍得以保持。通过支持富有表现力的世界事件生成,WorldCanvas将世界模型从被动预测器推进到交互式、用户塑造的模拟器。我们的项目页面可在:https://worldcanvas.github.io/上找到。
英文摘要:
We present WorldCanvas, a framework for promptable world events that enables rich, user-directed simulation by combining text, trajectories, and reference images. Unlike text-only approaches and existing trajectory-controlled image-to-video methods, our multimodal approach combines trajectories -- encoding motion, timing, and visibility -- with natural language for semantic intent and reference images for visual grounding of object identity, enabling the generation of coherent, controllable events that include multi-agent interactions, object entry/exit, reference-guided appearance and counterintuitive events. The resulting videos demonstrate not only temporal coherence but also emergent consistency, preserving object identity and scene despite temporary disappearance. By supporting expressive world events generation, WorldCanvas advances world models from passive predictors to interactive, user-shaped simulators. Our project page is available at: https://worldcanvas.github.io/.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。