arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.02785cs.CV
收藏

DreamStyle: 视频风格化的一个统一框架

DreamStyle: A Unified Framework for Video Stylization

Intelligent Creation Lab, ByteDance(字节跳动智能创作实验室)

Mengtian Li, Jinshu Chen, Songtao Zhao, Wanquan Feng, Pengqi Tu, Qian He

AI总结:

DreamStyle通过统一框架支持文本、风格图像和第一帧引导的视频风格化,提升风格一致性和视频质量。

摘要:

视频风格化是视频生成模型的重要下游任务,尚未得到充分探索。其输入风格条件通常包括文本、风格图像和风格化第一帧。每种条件都有其特点:文本更具灵活性,风格图像提供更准确的视觉锚点,风格化第一帧使长视频风格化成为可能。然而,现有方法大多局限于单一类型的风格条件,限制了其应用范围。此外,缺乏高质量数据集导致风格不一致和时间闪烁。为了解决这些限制,我们引入DreamStyle,一个统一的视频风格化框架,支持(1)文本引导、(2)风格图像引导和(3)第一帧引导的视频风格化,同时配备精心设计的数据整理流程以获得高质量的配对视频数据。DreamStyle基于一个基础的图像到视频(I2V)模型,并使用低秩适应(LoRA)技术,通过特定于标记的上矩阵减少不同条件标记之间的混淆。定性和定量评估表明,DreamStyle在所有三种视频风格化任务中都表现出色,并在风格一致性和视频质量上优于竞争对手。

英文摘要:

Video stylization, an important downstream task of video generation models, has not yet been thoroughly explored. Its input style conditions typically include text, style image, and stylized first frame. Each condition has a characteristic advantage: text is more flexible, style image provides a more accurate visual anchor, and stylized first frame makes long-video stylization feasible. However, existing methods are largely confined to a single type of style condition, which limits their scope of application. Additionally, their lack of high-quality datasets leads to style inconsistency and temporal flicker. To address these limitations, we introduce DreamStyle, a unified framework for video stylization, supporting (1) text-guided, (2) style-image-guided, and (3) first-frame-guided video stylization, accompanied by a well-designed data curation pipeline to acquire high-quality paired video data. DreamStyle is built on a vanilla Image-to-Video (I2V) model and trained using a Low-Rank Adaptation (LoRA) with token-specific up matrices that reduces the confusion among different condition tokens. Both qualitative and quantitative evaluations demonstrate that DreamStyle is competent in all three video stylization tasks, and outperforms the competitors in style consistency and video quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑