arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.11253cs.CV
收藏

CountSteer:在扩散模型中通过引导注意力实现物体计数

CountSteer: Steering Attention for Object Counting in Diffusion Models

Hyemin Boo, Hyoryung Kim, Myungjin Lee, Seunghyeon Lee, Jiyoung Lee, Jang-Hwan Choi, Hyunsoo Cho

AI总结:

CountSteer通过引导扩散模型的注意力机制,提升指定物体数量的生成精度,实现更可控的图像生成。

摘要:

文本到图像的扩散模型能够生成逼真且连贯的图像,但往往无法遵循文本中的数字指令,揭示了语言与视觉表示之间的差距。有趣的是,我们发现这些模型并非完全对数字无感——它们在输出是否符合指定数量时,内部信号会以一致的方式变化。这一观察表明,模型已经编码了一种潜在的数字正确性概念,可以用来更精确地引导生成。基于这一直觉,我们引入了CountSteer,一种无需训练的方法,通过在推理过程中引导模型的交叉注意力隐藏状态来提高指定物体数量的生成效果。在我们的实验中,CountSteer在不牺牲视觉质量的情况下,将物体计数的准确性提高了约4%,展示了迈向更可控且语义可靠的文本到图像生成的一个简单而有效的方法。

英文摘要:

Text-to-image diffusion models generate realistic and coherent images but often fail to follow numerical instructions in text, revealing a gap between language and visual representation. Interestingly, we found that these models are not entirely blind to numbers-they are implicitly aware of their own counting accuracy, as their internal signals shift in consistent ways depending on whether the output meets the specified count. This observation suggests that the model already encodes a latent notion of numerical correctness, which can be harnessed to guide generation more precisely. Building on this intuition, we introduce CountSteer, a training-free method that improves generation of specified object counts by steering the model's cross-attention hidden states during inference. In our experiments, CountSteer improved object-count accuracy by about 4% without compromising visual quality, demonstrating a simple yet effective step toward more controllable and semantically reliable text-to-image generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑