arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1901.10024cs.LGcs.GRstat.ML
收藏

跨域图像操作演示

Cross-Domain Image Manipulation by Demonstration

Google AI(谷歌人工智能研究院);Boston University(波士顿大学)

Ben Usman, Nick Dufour, Kate Saenko, Chris Bregler

摘要:

在这项工作中,我们提出了一个模型,可以操纵各个视觉属性的对象在一个真实的场景中使用的例子,各自的属性操作如何影响模拟的输出。作为一个例子,我们训练我们的模型来操纵一个人的脸的表情,使用非真实感的3D渲染的脸与不同的表情。我们的模型设法保留真实面部的所有其他视觉属性,例如头部方向,即使这个和其他属性在真实或合成域中没有标记。由于我们的模型只使用这种操作的“合成演示”来孤立地学习操作特定属性,而没有明确提供标签,因此它可以应用于形状,纹理,照明和其他难以测量或表示为实值向量的属性。我们衡量的程度,我们的模型保留一个真实图像的其他属性时,一个单一的特定属性被操纵。我们使用数字数据集来分析属性分布的差异如何影响我们模型的性能,并在一个更困难的环境中展示结果:学习使用非真实感3D渲染来操纵真实的人脸。

英文摘要:

In this work we propose a model that can manipulate individual visual attributes of objects in a real scene using examples of how respective attribute manipulations affect the output of a simulation. As an example, we train our model to manipulate the expression of a human face using nonphotorealistic 3D renders of a face with varied expression. Our model manages to preserve all other visual attributes of a real face, such as head orientation, even though this and other attributes are not labeled in either real or synthetic domain. Since our model learns to manipulate a specific property in isolation using only "synthetic demonstrations" of such manipulations without explicitly provided labels, it can be applied to shape, texture, lighting, and other properties that are difficult to measure or represent as real-valued vectors. We measure the degree to which our model preserves other attributes of a real image when a single specific attribute is manipulated. We use digit datasets to analyze how discrepancy in attribute distributions affects the performance of our model, and demonstrate results in a far more difficult setting: learning to manipulate real human faces using nonphotorealistic 3D renders.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑