IC-Effect: 通过上下文学习实现精确高效的视频特效编辑
IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learning
AI总结：
IC-Effect通过上下文学习实现高效视频特效编辑，能精准保留背景并合成复杂特效，提升视频创作的可能性。
摘要：
我们提出IC-Effect，一种基于指令的DiT框架，用于少样本视频VFX编辑，能够合成复杂的特效（例如火焰、粒子和卡通角色），同时严格保持空间和时间一致性。视频VFX编辑极具挑战性，因为注入的特效必须与背景无缝融合，背景必须完全不变，且特效模式必须从有限的配对数据中高效学习。然而，现有的视频编辑模型无法满足这些要求。IC-Effect利用源视频作为干净的上下文条件，利用DiT模型的上下文学习能力，实现精确的背景保留和自然的特效注入。一个两阶段的训练策略，包括通用编辑适应，随后通过Effect-LoRA进行特效特定学习，确保了强大的指令遵循和稳健的特效建模。为了进一步提高效率，我们引入了时空稀疏令牌化，实现了高保真度的同时显著减少了计算量。我们还发布了一个覆盖15种高质量视觉风格的配对VFX编辑数据集。广泛的实验表明，IC-Effect能够实现高质量、可控且时间一致的VFX编辑，为视频创作开辟了新的可能性。
英文摘要：
We propose \textbf{IC-Effect}, an instruction-guided, DiT-based framework for few-shot video VFX editing that synthesizes complex effects (\eg flames, particles and cartoon characters) while strictly preserving spatial and temporal consistency. Video VFX editing is highly challenging because injected effects must blend seamlessly with the background, the background must remain entirely unchanged, and effect patterns must be learned efficiently from limited paired data. However, existing video editing models fail to satisfy these requirements. IC-Effect leverages the source video as clean contextual conditions, exploiting the contextual learning capability of DiT models to achieve precise background preservation and natural effect injection. A two-stage training strategy, consisting of general editing adaptation followed by effect-specific learning via Effect-LoRA, ensures strong instruction following and robust effect modeling. To further improve efficiency, we introduce spatiotemporal sparse tokenization, enabling high fidelity with substantially reduced computation. We also release a paired VFX editing dataset spanning $15$ high-quality visual styles. Extensive experiments show that IC-Effect delivers high-quality, controllable, and temporally consistent VFX editing, opening new possibilities for video creation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
