使用风格码进行多样化语义图像编辑
Diverse Semantic Image Editing with Style Codes
AI总结：
针对语义图像编辑中新增物体风格缺失、边界不连续及可见性差异建模不足的问题，提出基于风格码区分编码可见与部分可见物体的框架，提升了定量表现并生成多样化无缝结果。
摘要：
语义图像编辑需要根据语义图对像素进行补全。这是一项具有挑战性的任务，因为这种补全既需要与上下文协调一致，又需要严格遵循语义图。以往为该任务提出的大多数方法都试图从被擦除的图像中编码全部信息。然而，当一个物体（例如汽车）被添加到场景中时，仅凭上下文无法编码其风格。另一方面，能够输出多样化生成结果的模型却难以输出在生成部分与未擦除部分之间具有无缝边界的图像。此外，以往方法没有一种机制能够以不同方式编码可见物体和部分可见物体的风格，从而获得更好的性能。在本文中，我们提出了一个框架，该框架能够通过一种新颖机制编码可见物体和部分可见物体，以实现风格编码和最终生成的一致性。我们与以往的条件图像生成和语义图像编辑算法进行了广泛比较。大量实验表明，我们的方法显著优于当前最先进的方法。我们的方法不仅取得了更好的定量结果，还能提供多样化的结果。已发布的代码和演示请参见项目网页：https://github.com/hakansivuk/DivSem。
英文摘要：
Semantic image editing requires inpainting pixels following a semantic map. It is a challenging task since this inpainting requires both harmony with the context and strict compliance with the semantic maps. The majority of the previous methods proposed for this task try to encode the whole information from erased images. However, when an object is added to a scene such as a car, its style cannot be encoded from the context alone. On the other hand, the models that can output diverse generations struggle to output images that have seamless boundaries between the generated and unerased parts. Additionally, previous methods do not have a mechanism to encode the styles of visible and partially visible objects differently for better performance. In this work, we propose a framework that can encode visible and partially visible objects with a novel mechanism to achieve consistency in the style encoding and final generations. We extensively compare with previous conditional image generation and semantic image editing algorithms. Our extensive experiments show that our method significantly improves over the state-of-the-art. Our method not only achieves better quantitative results but also provides diverse results. Please refer to the project web page for the released code and demo: https://github.com/hakansivuk/DivSem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
