Disco-LoRA：内容、风格和运动的解耦组合用于多概念视频定制
Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization
AI总结：
提出Disco-LoRA框架，通过两阶段解耦和重组内容、风格和运动概念，实现多概念视频定制，并采用Z-score正则化协调LoRA权重分布。
摘要：
基于文本到视频（T2V）模型的视频定制旨在从参考数据中学习特定特征以生成可控视频。尽管在图像风格化和视频运动定制方面取得了显著进展，但同时控制多个概念（如内容、风格和运动）仍然是一个重大挑战。在这项工作中，我们系统地定义了多概念视频定制任务，该任务需要联合控制内容、风格和运动。为了促进该领域的研究，我们构建了一个全面的基准，并提出了Disco-LoRA，这是一个统一的框架，通过两阶段解耦和灵活重组不同概念来解决该问题：（1）我们将目标分解为两个子任务：内容-风格和内容-运动。每个子任务都使用我们的迭代双LoRA解耦框架来处理，该框架有效地解耦了数据中的不同概念。（2）我们确定逐层权重趋势对于LoRA身份至关重要，而权重幅度决定了可组合性。为了协调这些尺度，我们提出了一种基于Z分数的统计正则化方法，该方法对齐权重分布，在保留逐层趋势的同时最小化不同LoRA之间的干扰。大量实验表明，Disco-LoRA在多概念视频定制方面表现出色，有效保留了外观、风格和运动，实现了可控的文本到视频生成。
英文摘要：
Video customization based on Text-to-Video (T2V) models aims to learn specific features from reference data to generate controllable videos. While significant strides have been made in image stylization and video motion customization, simultaneously controlling multiple concepts, such as content, style, and motion, remains a major challenge. In this work, we systematically define the task of multi-concept video customization, which requires the joint control of content, style, and motion. To facilitate research in this area, we construct a comprehensive benchmark and propose Disco-LoRA, a unified framework designed to tackle this problem by disentangling and flexibly recombining different concepts in two stages: (1) We decompose the objective into two sub-tasks: Content-Style and Content-Motion. Each sub-task is addressed using our Iterative Dual-LoRA Disentanglement Framework, which effectively disentangles distinct concepts within the data. (2) We identify layer-wise weight trends as crucial for LoRA identity, while weight magnitudes dictate composability. To harmonize these scales, we propose a Z-score-based statistical regularization that aligns weight distributions, preserving layer-wise trends while minimizing interference between different LoRAs. Extensive experiments show that Disco-LoRA excels in multi-concept video customization, effectively preserving appearance, style, and motion for controllable text-to-video generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
