arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2505.20525cs.CV

MultLFG:使用频域引导的无训练多LoRA组合

MultLFG: Training-free Multi-LoRA composition using Frequency-domain Guidance

Johns Hopkins University(约翰霍普金斯大学);Samsung AI Center Toronto(三星AI中心多伦多)

Aniket Roy, Maitreya Suin, Ketul Shah, Rama Chellappa

AI总结:

MultLFG提出一种基于频域引导的无训练多LoRA组合框架,通过时间步和频带自适应融合策略,提升组合保真度和图像质量,在ComposLoRA基准上优于现有方法。

摘要:

低秩适配(LoRA)作为一种计算高效的方法,在生成模型的微调中日益受到重视,能够以最小的开销实现不同的视觉概念合成。然而,现有方法难以在无训练的情况下有效合并多个LoRA适配器,尤其是在涉及多种视觉元素的复杂组合中。我们提出了MultLFG,一种新颖的无训练多LoRA组合框架,利用频域引导实现多个LoRA的自适应融合。与现有方法统一聚合特定概念的LoRA不同,MultLFG采用时间步和频带自适应融合策略,根据特定时间步和频带的内容相关性选择性激活相关LoRA。这种频率敏感的引导不仅改善了空间一致性,还提供了对多LoRA组合的更精细控制,从而产生更准确和一致的结果。在ComposLoRA基准上的实验评估表明,MultLFG显著增强了跨各种风格和概念集的组合保真度和图像质量,在多概念生成任务中优于最先进的基线。代码将发布。

英文摘要:

Low-Rank Adaptation (LoRA) has gained prominence as a computationally efficient method for fine-tuning generative models, enabling distinct visual concept synthesis with minimal overhead. However, current methods struggle to effectively merge multiple LoRA adapters without training, particularly in complex compositions involving diverse visual elements. We introduce MultLFG, a novel framework for training-free multi-LoRA composition that utilizes frequency-domain guidance to achieve adaptive fusion of multiple LoRAs. Unlike existing methods that uniformly aggregate concept-specific LoRAs, MultLFG employs a timestep and frequency subband adaptive fusion strategy, selectively activating relevant LoRAs based on content relevance at specific timesteps and frequency bands. This frequency-sensitive guidance not only improves spatial coherence but also provides finer control over multi-LoRA composition, leading to more accurate and consistent results. Experimental evaluations on the ComposLoRA benchmark reveal that MultLFG substantially enhances compositional fidelity and image quality across various styles and concept sets, outperforming state-of-the-art baselines in multi-concept generation tasks. Code will be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑