arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2602.03753cs.CV
收藏

测试时条件化与表征对齐的视觉特征

REPA-G: Test-Time Conditioning with Representation-Aligned Visual Features

Department of XXX, University of YYY, Location, Country（XXX系，YYY大学，地点，国家）；School of ZZZ, Institute of WWW, Location, Country（ZZZ学院，WWW研究所，地点，国家）

Nicolas Sereyjol-Garros, Ellington Kirby, Victor Letzelter, Victor Besnier, Nermin Samet

备注：
NeurIPS 2026
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

REPA-G通过利用对齐的视觉表征，在推理时实现灵活的条件化生成，提供多尺度控制和多概念组合能力。

摘要：

尽管使用自监督模型进行表征对齐已被证明可以提高扩散模型的训练效果，但其在增强推理时条件化方面的潜力仍 largely 未被探索。我们引入了表征对齐引导（REPA-G），一种利用这些对齐表征、具有丰富语义属性的框架，以从生成中实现测试时条件化。通过在推理时优化相似性目标（潜在函数），我们引导去噪过程朝向由预训练特征提取器提取的条件表征。我们的方法在多个尺度上提供了灵活的控制，从通过单个补丁实现的细粒度纹理匹配到使用全局图像特征标记实现的广泛语义引导。我们进一步将此扩展到多概念组合，允许不同概念的忠实组合。REPA-G完全在推理时运行，提供了一种灵活且精确的替代方案，替代通常模糊的文本提示或粗略的类别标签。我们从理论上证明了这种引导如何使采样从潜在函数诱导的倾斜分布中进行。在ImageNet和COCO上的定量结果表明，我们的方法实现了高质量、多样化的生成。代码可在https://github.com/valeoai/REPA-G上获得。

英文摘要：

While representation alignment with self-supervised models has been shown to improve diffusion model training, its potential for enhancing inference-time conditioning remains largely unexplored. We introduce Representation-Aligned Guidance (REPA-G), a framework that leverages these aligned representations, with rich semantic properties, to enable test-time conditioning from features, in generation. By optimizing a similarity objective (the potential) at inference, we steer the denoising process toward a conditioned representation extracted from a pre-trained feature extractor. Our method provides versatile control at multiple levels of granularity, ranging from patch level matching via single patches to broad semantic guidance using global image feature tokens. We further extend this to multi-concept composition, allowing for the faithful combination of distinct concepts. REPA-G operates entirely at inference time with no additional training required, offering a flexible and precise alternative to often ambiguous text prompts or coarse class labels. Our approach achieves high-quality, diverse generations on ImageNet and COCO. Code is available at https://github.com/valeoai/REPA-G

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑