Look-Before-Move：动态3D故事世界中的叙事驱动世界视觉注意力
Look-Before-Move: Narrative-Grounded World Visual Attention in Dynamic 3D Story Worlds
AI总结：
提出Look-Before-Move框架，通过语义观察合约、蒙特卡洛视点搜索和语义轨迹接地，在动态3D故事世界中实现叙事驱动的视觉注意力规划，提升主体感知、意图一致性和轨迹质量。
摘要：
随着具身AI和世界模型越来越多地在动态3D环境中运行，视觉感知必须超越被动解释给定观察，转向主动决定观察什么。我们通过动态3D故事世界中的相机规划来研究这一问题，其中相机不仅要生成平滑运动，还要在移动前决定应获取哪些视觉证据。我们将这种能力定义为叙事驱动世界视觉注意力，其中相机作为一个具身观察者，决定观察什么、如何构图观察，以及在叙事意图和物理3D约束下如何随时间转移注意力。为实现这一能力，我们提出Look-Before-Move，一个将观察规范与运动执行分离的相机规划框架。它首先构建语义观察合约，将导演意图转化为可执行的视觉约束；然后执行蒙特卡洛视点搜索，寻找符合叙事且几何可行的视点；最后应用语义轨迹接地，将选定视点连接成连续、碰撞感知且时间一致的相机运动。我们进一步基于StoryBlender构建了动态3D故事世界基准，涵盖50个故事、457个场景和1585个镜头，包含动画角色、语义场景配置和可执行的3D环境。实验表明，我们的框架在主体感知、意图一致性和轨迹质量上优于代表性基线，证明了在生成相机运动前组织视觉注意力的重要性。
英文摘要：
As embodied AI and world models increasingly operate in dynamic 3D environments, visual perception must move beyond passively interpreting given observations toward actively deciding what to observe. We study this problem through camera planning in dynamic 3D story worlds, where the camera must not only generate smooth motion, but also decide what visual evidence should be acquired before it moves. We formulate this capability as Narrative-Grounded World Visual Attention, where the camera acts as an embodied observer that determines what to observe, how to compose the observation, and how to shift attention over time under narrative intent and physical 3D constraints. To realize this capability, we propose Look-Before-Move, a camera planning framework that separates observation specification from motion execution. It first builds a Semantic Observation Contract to convert directorial intent into executable visual constraints, then performs Monte Carlo Viewpoint Search to find narrative-compliant and geometrically feasible viewpoints, and finally applies Semantic Trajectory Grounding to connect selected viewpoints into continuous, collision-aware, and temporally coherent camera motion. We further construct a dynamic 3D Story World Benchmark based on StoryBlender, covering 50 stories, 457 scenes, and 1585 shots with animated characters, semantic scene configurations, and executable 3D environments. Experiments show that our framework improves subject perception, intent consistency, and trajectory quality over representative baselines, demonstrating the importance of organizing visual attention before generating camera motion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
