Control-DINO：用于可控图像到视频扩散的特征空间条件
Control-DINO: Feature Space Conditioning for Controllable Image-to-Video Diffusion
AI总结：
本文提出Control-DINO，通过解耦外观与其他特征，实现对视频扩散模型的可控生成，提升生成渲染的可控性。
摘要：
视频模型近年来在内容生成、新视角合成和更广泛的虚拟世界模拟中得到了成功应用。许多生成和转移应用依赖于对这些模型的条件化，通常通过感知、几何或简单的语义信号进行。同时，从大规模自监督学习中获得的高维特征越来越多地被用作视觉模型的通用接口。两者之间的联系已被探索用于特定主题的编辑、对齐和训练视频扩散模型，但尚未在作为预训练视频扩散模型的更通用条件信号角色中得到探索。通过自监督学习获得的特征，如DINO，包含大量关于场景风格、照明和语义的信息。这使它们在重建任务中表现出色，但限制了生成能力。本文展示了如何利用这些特征进行视频域迁移和从3D生成视频。我们介绍了一种轻量级架构和训练策略，将外观与其他希望保留的特征解耦，从而实现对外观变化的鲁棒控制，如风格化和重新照明。此外，我们还表明，低空间分辨率可以通过更高的特征维度性来补偿，从而在从显式空间表示中生成时提高可控性。
英文摘要：
Video diffusion models have recently been applied with success to problems in content generation, novel view synthesis, and, more broadly, world simulation. Many applications in generation and transfer rely on conditioning these models, typically through perceptual, geometric, or simple semantic signals, fundamentally using them as generative renderers. At the same time, high-dimensional features obtained from large-scale self-supervised learning on images or point clouds are increasingly used as a general-purpose interface for vision models. The connection between the two has been explored for subject specific editing, aligning and training video diffusion models, but not in the role of a dense conditioning signal for pretrained video diffusion models. Features obtained through self-supervised learning like DINOv3, contain a lot of entangled information about style, lighting and semantics of the scene. This makes them great at reconstruction tasks but limits their generative capabilities. In this paper, we show how we can use the features for tasks such as video domain transfer and video-from-3D generation. We introduce a lightweight control architecture and training strategy that decouples appearance from other features that we wish to preserve, enabling robust control for appearance changes such as stylization and relighting. Furthermore, we show that low spatial resolution can be compensated by higher feature dimensionality, improving controllability in generative rendering from explicit spatial representations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
