arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.20989cs.CV
收藏

循环一致性调谐用于分层图像分解

Cycle-Consistent Tuning for Layered Image Decomposition

Shenzhen University(深圳大学);Hebrew University of Jerusalem(耶路撒冷希伯来大学);Tel Aviv University(特拉维夫大学)

Zheng Gu, Min Lu, Zhida Sun, Dani Lischinski, Daniel Cohen-Or, Hui Huang

AI总结:

本文提出一种基于循环一致性调谐的图像分解框架,通过轻量级LoRA适应和双向监督提升分层分离的鲁棒性和泛化能力。

摘要:

在现实世界图像中解构视觉层一直是视觉和图形领域持续的挑战,因为这些层往往涉及非线性和全局耦合的相互作用,包括阴影、反射和透视畸变。在本文中,我们提出了一种上下文图像分解框架,利用大型扩散基础模型进行分层分离。我们专注于具有挑战性的标志-物体分解情况,目标是从其出现的表面上分离标志,同时忠实保留两个层次。我们的方法通过轻量级LoRA适应对预训练扩散模型进行微调,并引入了循环一致性调谐策略,联合训练分解和合成模型,强制分解和重新合成图像之间的重建一致性。这种双向监督在层表现出复杂相互作用的情况下显著增强了鲁棒性。此外,我们引入了一种渐进自我改进过程,该过程通过迭代地将高质量模型生成示例添加到训练集中来改进性能。广泛的实验表明,我们的方法实现了准确且一致的分解,并且在其他分解类型上也表现出良好的泛化能力,表明其作为分层图像分解统一框架的潜力。

英文摘要:

Disentangling visual layers in real-world images is a persistent challenge in vision and graphics, as such layers often involve non-linear and globally coupled interactions, including shading, reflection, and perspective distortion. In this work, we present an in-context image decomposition framework that leverages large diffusion foundation models for layered separation. We focus on the challenging case of logo-object decomposition, where the goal is to disentangle a logo from the surface on which it appears while faithfully preserving both layers. Our method fine-tunes a pretrained diffusion model via lightweight LoRA adaptation and introduces a cycle-consistent tuning strategy that jointly trains decomposition and composition models, enforcing reconstruction consistency between decomposed and recomposed images. This bidirectional supervision substantially enhances robustness in cases where the layers exhibit complex interactions. Furthermore, we introduce a progressive self-improving process, which iteratively augments the training set with high-quality model-generated examples to refine performance. Extensive experiments demonstrate that our approach achieves accurate and coherent decompositions and also generalizes effectively across other decomposition types, suggesting its potential as a unified framework for layered image decomposition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑