FlexAM: 一种灵活的外观-运动分解方法用于多功能视频生成控制
FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Control
AI总结:
FlexAM通过引入新型3D控制信号,实现外观与运动的解耦,提升视频生成任务的灵活性和生成质量。
摘要:
在视频生成中实现有效且可推广的控制仍然是一个重大挑战。尽管许多方法依赖于模糊或任务特定的信号,我们认为对“外观”和“运动”的根本解耦提供了更稳健和可扩展的路径。我们提出了FlexAM,一种基于新型3D控制信号的统一框架。该信号将视频动态表示为点云,引入了三个关键增强:多频位置编码以区分细粒度运动、深度感知的位置编码,以及一个灵活的控制信号以在精度和生成质量之间取得平衡。这种表示方法使FlexAM能够有效解耦外观和运动,使包括I2V/V2V编辑、摄像机控制和空间物体编辑在内的广泛任务成为可能。大量实验表明,FlexAM在所有评估任务中均实现了优越的性能。
英文摘要:
Effective and generalizable control in video generation remains a significant challenge. While many methods rely on ambiguous or task-specific signals, we argue that a fundamental disentanglement of "appearance" and "motion" provides a more robust and scalable pathway. We propose FlexAM, a unified framework built upon a novel 3D control signal. This signal represents video dynamics as a point cloud, introducing three key enhancements: multi-frequency positional encoding to distinguish fine-grained motion, depth-aware encoding, and a flexible control signal for balancing precision and generalization. This representation allows FlexAM to effectively disentangle appearance and motion, enabling a wide range of tasks including I2V/V2V editing, camera control, and spatial object editing. Extensive experiments demonstrate that FlexAM achieves superior performance across all evaluated tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。