arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.27199cs.CV
收藏

让触发器控制:面向频率的Dropout用于有效的标记控制

Let Triggers Control: Frequency-Aware Dropout for Effective Token Control

Yonsei University(延世大学);Onoma AI

Junyoung Koh, Hoyeon Moon, Dongha Kim, Seungmin Lee, Sanghyun Park, Min Song

AI总结:

本文提出频率感知Dropout方法,通过分析触发器与上下文的共现关系,提升文本到图像生成中触发器的可控性与个性化能力,无需增加参数或架构修改。

摘要:

文本到图像模型如Stable Diffusion已实现了前所未有的高质量视觉合成。随着这些模型的发展,生成模型的个性化——通常通过低秩适应(LoRA)和专用触发器标记实现——已成为研究重点。先前工作假设通过单个触发器标记微调来表示新概念,但往往导致可控性差,触发器标记单独无法可靠唤起预期概念。我们将其归因于在微调过程中触发器标记与周围上下文频繁共现,导致其表示纠缠并削弱了标记的语义独特性。为此,我们提出频率感知Dropout(FAD)——一种新的正则化技术,可在不增加新参数的情况下提升提示可控性。FAD包含两个关键组件:共现分析和课程启发式的调度。在基于标记的扩散模型(SD~1.5和SDXL)和自然语言驱动的骨干(FLUX和Qwen-Image)上进行的定性和定量分析显示,在提示保真度、风格精度和用户感知质量方面均取得一致提升。我们的方法提供了一种简单而有效的Dropout策略,增强了文本到图像生成的可控性和个性化能力。值得注意的是,它在不引入额外参数或架构修改的情况下实现了这些改进,使其能够轻松应用于现有模型,计算开销极低。

英文摘要:

Text-to-image models such as Stable Diffusion have achieved unprecedented levels of high-fidelity visual synthesis. As these models advance, personalization of generative models -- commonly facilitated through Low-Rank Adaptation (LoRA) with a dedicated trigger token -- has become a significant area of research. Previous works have naively assumed that fine-tuning with a single trigger token to represent new concepts. However, this often results in poor controllability, where the trigger token alone fails to reliably evoke the intended concept. We attribute this issue to the frequent co-occurrence of the trigger token with the surrounding context during fine-tuning, which entangles their representations and compromises the token's semantic distinctiveness. To disentangle this, we propose Frequency-Aware Dropout (FAD) -- a novel regularization technique that improves prompt controllability without adding new parameters. FAD consists of two key components: co-occurrence analysis and curriculum-inspired scheduling. Qualitative and quantitative analyses across token-based diffusion models (SD~1.5 and SDXL) and natural language--driven backbones (FLUX and Qwen-Image) demonstrate consistent gains in prompt fidelity, stylistic precision, and user-perceived quality. Our method provides a simple yet effective dropout strategy that enhances controllability and personalization in text-to-image generation. Notably, it achieves these improvements without introducing additional parameters or architectural modifications, making it readily applicable to existing models with minimal computational overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑