arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.15819cs.CV

VideoSketcher:利用视频模型先验的序列草图生成

VideoSketcher: Sequential Sketch Generation Using Video Model Priors

MIT(麻省理工学院)

Hui Ren, Yuval Alaluf, Omer Bar Tal, Alexander Schwing, Antonio Torralba, Yael Vinker

AI总结:

提出VideoSketcher方法,结合LLM的语义规划与视频扩散模型的时序渲染,通过两阶段微调从少量样本学习笔画顺序与风格,生成高质量序列草图。

摘要:

素描本质上是序列化的:笔画逐步绘制以探索和完善想法。然而,大多数生成方法将草图视为静态图像,忽略了创造性探索背后的时间过程。建模这种序列结构仍然具有挑战性:先前的方法要么依赖大规模但多样性有限的人类绘制数据集,要么使用大型语言模型(LLM)生成绘制指令,但往往以视觉保真度为代价。我们提出VideoSketcher,一种通过将预训练的文本到视频扩散模型适应于草图形成的稀疏连续性质来生成高质量绘制过程的方法。我们的关键洞察是LLM和视频扩散模型提供互补优势:LLM作为语义规划器,将概念分解为逐步指令,而视频扩散模型作为强大的“渲染器”,将它们转化为时间连贯的草图序列。我们引入一种两阶段微调策略,将时间结构与视觉外观解耦:笔画顺序从合成形状组合中学习,而风格则从少至七幅手绘示例中提炼。尽管监督极少,我们的方法能够生成多样、高质量的序列草图,并忠实遵循指定的绘制顺序。我们的框架自然扩展到笔刷风格控制和自回归生成,支持艺术应用。

英文摘要:

Sketching is inherently sequential: strokes are drawn progressively to explore and refine ideas. Yet most generative approaches treat sketches as static images, ignoring the temporal process underlying creative exploration. Modeling this sequential structure remains challenging: prior methods either rely on large-scale human-drawn datasets with limited diversity, or use large language models (LLMs) to produce drawing instructions, often at the cost of visual fidelity. We present VideoSketcher, a method for generating high-quality sketching processes by adapting pretrained text-to-video diffusion models to the sparse, continuous nature of sketch formation. Our key insight is that LLMs and video diffusion models offer complementary strengths: LLMs act as semantic planners that decompose concepts into step-by-step instructions, while video diffusion models serve as powerful "renderers" that translate them into temporally coherent sketch sequences. We introduce a two-stage fine-tuning strategy that decouples temporal structure from visual appearance: stroke ordering is learned from synthetic shape compositions, while style is distilled from as few as seven hand-drawn examples. Despite minimal supervision, our method can generate diverse, high-quality sequential sketches that faithfully follow specified drawing orders. Our framework naturally extends to brush style control and autoregressive generation, supporting artistic applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑