PartiCam：基于奖励引导的相机控制视频生成

中文摘要 AI 辅助

我们提出了PartiCam，一种基于粒子滤波的无训练方法，用于改进相机控制的视频生成。对于大型视频扩散模型而言，生成遵循精确指定相机轨迹的视频仍然具有挑战性。无训练方法不依赖于特定骨干网络，并通过在测试时引导预训练模型朝向所需的相机运动，避免了构建大型相机标注数据集的需求。这使得能够生成相机控制的视频数据，这些数据随后可用于训练相机条件视频扩散模型。现有的基于采样的引导方法常常遭受不稳定轨迹的困扰：它们要么探索过于广泛而无法遵循目标相机运动，要么过早坍缩并随时间失去视觉多样性。我们引入了一种用于扩散奖励引导的全局-局部细化框架，能够在视频生成过程中实现准确且一致的相机控制。我们的方法建立在序贯蒙特卡洛（SMC）引导之上，但引入了一个基于粒子滤波重采样的局部细化阶段。实验表明，在相机轨迹遵循性、减少漂移和更好的视觉质量方面有显著改进，且无需重新训练模型。

英文摘要

We present PartiCam, a training-free Particle filtering rooted method for improved Camera controlled video generation. Generating videos that follow a precisely specified camera trajectory remains challenging for large video diffusion models. Training-free approaches are backbone-agnostic and avoid the need to construct large camera-annotated datasets by steering pretrained models toward the desired camera motion at test time. This enables the generation of camera-controlled video data that can subsequently be used to train camera-conditioned video diffusion models. Existing sampling-based guidance approaches often suffer from unstable trajectories: they either explore too broadly and fail to respect the target camera motion or collapse early and lose visual diversity over time. We introduce a global-local refinement framework for diffusion reward guidance, enabling accurate and consistent camera control during video generation. Our method builds on Sequential Monte-Carlo (SMC) guidance, but introduces a local refinement stage based on particle filtered resampling. Experiments show large improvements in camera trajectory adherence, reduced drift, and better visual quality, without requiring model retraining.

发表机构

• INRIA, Univ. Rennes, CNRS, IRISA（法国国家信息与自动化研究所，雷恩大学，法国国家科学研究中心，IRISA）
• University of Oxford（牛津大学）
• Meta

机构由 AI 辅助整理，请以论文原文为准。