arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.03798cs.CV
收藏

Paint Transformer:具有中风预测的前馈神经绘画

Paint Transformer: Feed Forward Neural Painting with Stroke Prediction

Baidu Inc.(百度公司);Nanjing University(南京大学);Rutgers University(罗格斯大学)

Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Ruifeng Deng, Xin Li, Errui Ding, Hao Wang

摘要:

神经绘画是指为给定图像产生一系列笔划并使用神经网络非照片真实地重新创建它的过程。虽然基于强化学习(RL)的代理可以生成一个笔划序列一步一步的任务,它是不容易的训练一个稳定的RL代理。另一方面,笔划优化方法在大的搜索空间中迭代地搜索笔划参数集合;这种低效率显著地限制了它们的流行和实用性。不同于以往的方法,在本文中,我们制定的任务作为一个集预测问题,并提出了一种新的基于变压器的框架,称为油漆Transformer,预测的参数与前馈网络的中风集。通过这种方式,我们的模型可以并行生成一组笔划,并接近实时的获得尺寸为512 * 512的最终绘画。更重要的是,由于没有数据集可用于训练Paint Transformer,我们设计了一个自训练管道,这样它就可以在没有任何现成数据集的情况下进行训练,同时仍然具有出色的泛化能力。实验表明,我们的方法取得了更好的绘画性能比以前的便宜的训练和推理成本。代码和模型可用。

英文摘要:

Neural painting refers to the procedure of producing a series of strokes for a given image and non-photo-realistically recreating it using neural networks. While reinforcement learning (RL) based agents can generate a stroke sequence step by step for this task, it is not easy to train a stable RL agent. On the other hand, stroke optimization methods search for a set of stroke parameters iteratively in a large search space; such low efficiency significantly limits their prevalence and practicality. Different from previous methods, in this paper, we formulate the task as a set prediction problem and propose a novel Transformer-based framework, dubbed Paint Transformer, to predict the parameters of a stroke set with a feed forward network. This way, our model can generate a set of strokes in parallel and obtain the final painting of size 512 * 512 in near real time. More importantly, since there is no dataset available for training the Paint Transformer, we devise a self-training pipeline such that it can be trained without any off-the-shelf dataset while still achieving excellent generalization capability. Experiments demonstrate that our method achieves better painting performance than previous ones with cheaper training and inference costs. Codes and models are available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑