什么降低了编辑能力？面向改进GAN反演的领域特定混合细化
What Decreases Editing Capability? Domain-Specific Hybrid Refinement for Improved GAN Inversion
AI总结：
本文提出领域特定混合细化方法DHR，通过分割域内外区域并结合权重与特征调制，在提升GAN反演保真度的同时保持编辑能力。
摘要：
近年来，反演方法开始关注生成器中额外的高速率信息（例如权重或中间特征），以细化从嵌入潜码获得的反演和编辑结果。尽管这些技术在重建方面取得了合理的改进，但它们降低了编辑能力，尤其是在复杂图像上（例如包含遮挡、细节背景和伪影的图像）。一个关键的核心问题是在细化反演结果的同时避免编辑能力退化。为了解决这一问题，我们提出了领域特定混合细化（Domain-Specific Hybrid Refinement, DHR），该方法借鉴了两种主流细化技术的优缺点，在保持编辑能力的同时提升保真度。具体而言，我们首先提出领域特定分割（Domain-Specific Segmentation），将图像分割为两部分：域内部分和域外部分。细化过程旨在保持域内区域的可编辑性，并提升两个域的保真度。我们通过权重调制和特征调制来细化这两部分，称之为混合调制细化（Hybrid Modulation Refinement）。我们提出的方法与所有潜码嵌入方法兼容。扩展实验表明，我们的方法在真实图像反演和编辑方面达到了最先进的水平。代码可在 https://github.com/caopulan/Domain-Specific_Hybrid_Refinement_Inversion 获取。
英文摘要：
Recently, inversion methods have focused on additional high-rate information in the generator (e.g., weights or intermediate features) to refine inversion and editing results from embedded latent codes. Although these techniques gain reasonable improvement in reconstruction, they decrease editing capability, especially on complex images (e.g., containing occlusions, detailed backgrounds, and artifacts). A vital crux is refining inversion results, avoiding editing capability degradation. To tackle this problem, we introduce Domain-Specific Hybrid Refinement (DHR), which draws on the advantages and disadvantages of two mainstream refinement techniques to maintain editing ability with fidelity improvement. Specifically, we first propose Domain-Specific Segmentation to segment images into two parts: in-domain and out-of-domain parts. The refinement process aims to maintain the editability for in-domain areas and improve two domains' fidelity. We refine these two parts by weight modulation and feature modulation, which we call Hybrid Modulation Refinement. Our proposed method is compatible with all latent code embedding methods. Extension experiments demonstrate that our approach achieves state-of-the-art in real image inversion and editing. Code is available at https://github.com/caopulan/Domain-Specific_Hybrid_Refinement_Inversion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
