arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.05088cs.CV

RADIANCE:使用IP-适配器进行相对自适应去噪以增强新颖概念

RADIANCE: Relative Adaptive Denoising with IP-Adapter for Novel Concept Enhancement

National Yang Ming Chiao Tung University(国立阳明交通大学)

Zi-Xiang Ni, Bo-Lun Huang, Teng-Fang Hsiao, Bo-Kai Ruan, Hong-Han Shuai

AI总结:

针对文本到图像扩散模型合成罕见概念的问题,提出无训练框架RADIANCE,通过三个模块组件增强预训练主干,经实验验证其能提升合成效果。

摘要:

文本到图像(T2I)扩散模型取得显著进展,但合成涉及不寻常属性-对象配对的罕见概念仍有困难。我们提出RADIANCE,一个将推理视为闭环反馈过程的无训练框架。RADIANCE用三个模块化组件增强预训练主干,实验表明其能提升合成效果。

英文摘要:

Text-to-image (T2I) diffusion models have achieved striking progress but still struggle to synthesize rare concepts involving unusual attribute-object pairings, often resulting in concept omission or semantic drift where a dominant entity overwhelms the generation. Tracing these failures to a lack of compositional balance during the denoising trajectory, we propose RADIANCE, a training-free framework that treats inference as a closed-loop feedback process. RADIANCE augments pretrained backbones with three modular components: (1) a Compositional Similarity Monitor (CSM) that tracks the emergence of objects and attributes in intermediate latents via CLIP-based feedback; (2) a Bidirectional Scale Controller (BSC) that applies a reactive "restoring force" using positive and negative IP-Adapter scales to rebalance biased trajectories; and (3) a Feedback Guidance Scheduler (FGS) that coordinates these updates across timesteps without additional training. We further extend the framework to multi-object prompts via Delayed Adapter Activation (DAA) and Layer-wise Alternating Guidance (LAG) to prevent premature concept fusion. By overlapping monitoring and denoising through pipelined execution, RADIANCE maintains competitive latency while significantly enhancing the per-sample success rate and effective throughput. Experiments on RareBench and T2I-CompBench demonstrate that RADIANCE consistently enhances compositional alignment and perceptual quality over state-of-the-art baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑