arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2010.14702cs.GR
收藏

最佳纹理:通过最佳传输的快速和鲁棒的纹理合成和风格转移

Optimal Textures: Fast and Robust Texture Synthesis and Style Transfer through Optimal Transport

Eric Risser

摘要:

本文提出了一种轻量级的,高质量的纹理合成算法,易于推广到其他应用,如风格转移和纹理混合。我们通过自动编码器的瓶颈层内的深度神经激活向量来表示纹理特征,并将纹理合成问题框定为正在合成的图像的激活值与示例纹理的激活值之间的最佳传输。为了找到这种最佳的传输映射,我们利用N维概率密度函数(PDF)转移过程,该过程迭代PDF基的多个随机旋转,并匹配每个维度上的1D边缘分布。这实现了与昂贵的基于反向传播的神经纹理合成方法相当的质量和灵活性,但具有实现交互速率的潜力。我们证明,一阶统计提供了一个更强大的表示纹理比二阶统计,今天使用的。我们提出了一个扩展的算法,降低了神经特征空间的维数。我们利用一个多尺度的粗到细的合成金字塔来捕获和保留较大的图像特征;在一个框架下统一颜色和风格转移;并进一步增强该系统与一个新的掩蔽方案,重新采样和重新加权的特征分布用户引导的纹理绘制和有针对性的风格转移。

英文摘要:

This paper presents a light-weight, high-quality texture synthesis algorithm that easily generalizes to other applications such as style transfer and texture mixing. We represent texture features through the deep neural activation vectors within the bottleneck layer of an auto-encoder and frame the texture synthesis problem as optimal transport between the activation values of the image being synthesized and those of an exemplar texture. To find this optimal transport mapping, we utilize an N-dimensional probability density function (PDF) transfer process that iterates over multiple random rotations of the PDF basis and matches the 1D marginal distributions across each dimension. This achieves quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but with the potential of achieving interactive rates. We demonstrate that first order statistics offer a more robust representation for texture than the second order statistics that are used today. We propose an extension of this algorithm that reduces the dimensionality of the neural feature space. We utilize a multi-scale coarse-to-fine synthesis pyramid to capture and preserve larger image features; unify color and style transfer under one framework; and further augment this system with a novel masking scheme that re-samples and re-weights the feature distribution for user-guided texture painting and targeted style transfer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑