RADIANCE:使用IP-适配器进行相对自适应去噪以增强新颖概念
RADIANCE: Relative Adaptive Denoising with IP-Adapter for Novel Concept Enhancement
AI总结:
针对文本到图像扩散模型合成罕见概念的问题,提出无训练框架RADIANCE,通过三个模块组件增强预训练主干,经实验验证其能提升合成效果。
摘要:
文本到图像(T2I)扩散模型取得显著进展,但合成涉及不寻常属性-对象配对的罕见概念仍有困难。我们提出RADIANCE,一个将推理视为闭环反馈过程的无训练框架。RADIANCE用三个模块化组件增强预训练主干,实验表明其能提升合成效果。
英文摘要:
Text-to-image (T2I) diffusion models have achieved striking progress but still struggle to synthesize rare concepts involving unusual attribute-object pairings, often resulting in concept omission or semantic drift where a dominant entity overwhelms the generation. Tracing these failures to a lack of compositional balance during the denoising trajectory, we propose RADIANCE, a training-free framework that treats inference as a closed-loop feedback process. RADIANCE augments pretrained backbones with three modular components: (1) a Compositional Similarity Monitor (CSM) that tracks the emergence of objects and attributes in intermediate latents via CLIP-based feedback; (2) a Bidirectional Scale Controller (BSC) that applies a reactive "restoring force" using positive and negative IP-Adapter scales to rebalance biased trajectories; and (3) a Feedback Guidance Scheduler (FGS) that coordinates these updates across timesteps without additional training. We further extend the framework to multi-object prompts via Delayed Adapter Activation (DAA) and Layer-wise Alternating Guidance (LAG) to prevent premature concept fusion. By overlapping monitoring and denoising through pipelined execution, RADIANCE maintains competitive latency while significantly enhancing the per-sample success rate and effective throughput. Experiments on RareBench and T2I-CompBench demonstrate that RADIANCE consistently enhances compositional alignment and perceptual quality over state-of-the-art baselines.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。