VectorFusion:基于像素扩散模型的文本到SVG的抽象
VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models
摘要:
扩散模型在文本到图像的合成中已经显示出令人印象深刻的结果。使用大量的字幕图像数据集,扩散模型学习生成高度多样化的对象和场景的光栅图像。然而,设计师经常使用矢量表示的图像,如可缩放矢量图形(SVG)的数字图标或艺术。矢量图形可以缩放到任何大小,是紧凑的。我们表明,一个文本条件下的扩散模型训练图像的像素表示,可以用来生成SVG可导出的矢量图形。我们这样做没有访问标题SVG的大型数据集。通过优化可微分矢量图形光栅化器,我们的方法,VectorFusion,提取抽象的语义知识的预训练的扩散模型。受最近文本到3D工作的启发,我们使用Score Distillation Sampling学习与标题一致的SVG。为了加快生成速度并提高保真度,VectorFusion还从图像样本中进行了重新排序。实验表明比以前的工作更高的质量,并展示了一系列的风格,包括像素艺术和草图。请参阅我们的项目网页https://ajayj.com/vectorfusion。
英文摘要:
Diffusion models have shown impressive results in text-to-image synthesis. Using massive datasets of captioned images, diffusion models learn to generate raster images of highly diverse objects and scenes. However, designers frequently use vector representations of images like Scalable Vector Graphics (SVGs) for digital icons or art. Vector graphics can be scaled to any size, and are compact. We show that a text-conditioned diffusion model trained on pixel representations of images can be used to generate SVG-exportable vector graphics. We do so without access to large datasets of captioned SVGs. By optimizing a differentiable vector graphics rasterizer, our method, VectorFusion, distills abstract semantic knowledge out of a pretrained diffusion model. Inspired by recent text-to-3D work, we learn an SVG consistent with a caption using Score Distillation Sampling. To accelerate generation and improve fidelity, VectorFusion also initializes from an image sample. Experiments show greater quality than prior work, and demonstrate a range of styles including pixel art and sketches. See our project webpage at https://ajayj.com/vectorfusion .
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。