arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.04841cs.CVcs.LG
收藏

这种损失具有信息量吗?通过追踪目标动态实现更快的文本到图像定制

Is This Loss Informative? Faster Text-to-Image Customization by Tracking Objective Dynamics

MIPT(莫斯科物理技术学院);Yandex;HSE University(高等经济大学)

Anton Voronov, Mikhail Khoroshikh, Artem Babenko, Max Ryabinin

AI总结:

该研究通过分析文本到图像个性化方法的训练动态,提出一种基于固定输入集训练目标计算的即插即用早停准则,在Stable Diffusion上实现最多8倍加速且质量无明显下降。

摘要:

文本到图像生成模型代表了图像合成演进的下一阶段,提供了一种自然的方式来实现对结果既灵活又细粒度的控制。一个新兴的研究领域是将大型文本到图像模型快速适配到较小的数据集或新的视觉概念上。然而,许多高效的适配方法训练时间较长,这限制了它们的实际应用,拖慢了实验进度,并消耗了过多的GPU资源。在这项工作中,我们研究了流行的文本到图像个性化方法(如Textual Inversion或DreamBooth)的训练动态,旨在加速它们。我们观察到大多数概念在早期阶段就已学会,之后质量不再提升,但标准的训练收敛指标无法表明这一点。相反,我们提出了一种简单的即插即用早停准则,该准则仅需在所有训练迭代中对一组固定的输入计算常规训练目标。我们在Stable Diffusion上针对48个不同概念和三种个性化方法的实验证明了我们方法的竞争性性能,它使适配速度提高了最多8倍,而质量没有显著下降。

英文摘要:

Text-to-image generation models represent the next step of evolution in image synthesis, offering a natural way to achieve flexible yet fine-grained control over the result. One emerging area of research is the fast adaptation of large text-to-image models to smaller datasets or new visual concepts. However, many efficient methods of adaptation have a long training time, which limits their practical applications, slows down experiments, and spends excessive GPU resources. In this work, we study the training dynamics of popular text-to-image personalization methods (such as Textual Inversion or DreamBooth), aiming to speed them up. We observe that most concepts are learned at early stages and do not improve in quality later, but standard training convergence metrics fail to indicate that. Instead, we propose a simple drop-in early stopping criterion that only requires computing the regular training objective on a fixed set of inputs for all training iterations. Our experiments on Stable Diffusion for 48 different concepts and three personalization methods demonstrate the competitive performance of our approach, which makes adaptation up to 8 times faster with no significant drops in quality.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑