arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.16636cs.CV
收藏

利用全局和局部语义重新绑定潜在信息以进行纠缠扩散

REGLUE Your Latents with Global and Local Semantics for Entangled Diffusion

University of West Attica(西阿提卡大学);VRG, FEE, Czech Technical University in Prague(布拉格捷克技术大学)

Giorgos Petsangourakis, Christos Sgouropoulos, Bill Psomas, Theodoros Giannakopoulos, Giorgos Sfikas, Ioannis Kakogeorgiou

AI总结:

研究针对潜在扩散模型语义监督不足问题,提出REGLUE框架,联合建模VAE图像潜在信息、局部VFM语义和全局[CLS]令牌,用轻量级卷积语义压缩器聚合特征,经实验验证该框架能提升FID并加速收敛。

摘要:

潜在扩散模型(LDMs)实现了先进的图像合成,但其重建式去噪目标仅提供间接语义监督,高级语义出现缓慢,需要更长训练时间并限制样本质量。近期工作从视觉基础模型(VFMs)注入语义,要么通过表示对齐外部注入,要么在扩散过程中仅联合建模VFMs特征的狭窄切片内部注入,未充分利用丰富、非线性、多层空间语义。我们引入REGLUE(具有全局-局部统一编码的表示纠缠),一个统一的潜在扩散框架,在单个SiT主干内联合建模(i)VAE图像潜在信息,(ii)紧凑局部(补丁级)VFM语义,以及(iii)全局(图像级)[CLS]令牌。一个轻量级卷积语义压缩器将多层VFM特征非线性聚合为低维、空间结构化表示,在扩散过程中与VAE潜在信息纠缠。外部对齐损失进一步使内部表示向冻结的VFM目标正则化。在ImageNet 256x256上,REGLUE持续改进FID并加速收敛,优于SiT-B/2和SiT-XL/2基线,以及REPA、ReDi和REG。广泛实验表明(a)空间VFM语义至关重要,(b)非线性压缩是释放其全部益处的关键,(c)全局令牌和外部对齐在我们的全局-局部-潜在联合建模框架中作为互补的轻量级增强。代码可在该https URL获取。

英文摘要:

Latent diffusion models (LDMs) achieve state-of-the-art image synthesis, yet their reconstruction-style denoising objective provides only indirect semantic supervision: high-level semantics emerge slowly, requiring longer training and limiting sample quality. Recent works inject semantics from Vision Foundation Models (VFMs) either externally via representation alignment or internally by jointly modeling only a narrow slice of VFM features inside the diffusion process, under-utilizing the rich, nonlinear, multi-layer spatial semantics available. We introduce REGLUE (Representation Entanglement with Global-Local Unified Encoding), a unified latent diffusion framework that jointly models (i) VAE image latents, (ii) compact local (patch-level) VFM semantics, and (iii) a global (image-level) [CLS] token within a single SiT backbone. A lightweight convolutional semantic compressor nonlinearly aggregates multi-layer VFM features into a low-dimensional, spatially structured representation, which is entangled with the VAE latents in the diffusion process. An external alignment loss further regularizes internal representations toward frozen VFM targets. On ImageNet 256x256, REGLUE consistently improves FID and accelerates convergence over SiT-B/2 and SiT-XL/2 baselines, as well as over REPA, ReDi, and REG. Extensive experiments show that (a) spatial VFM semantics are crucial, (b) non-linear compression is key to unlocking their full benefit, and (c) global tokens and external alignment act as complementary, lightweight enhancements within our global-local-latent joint modeling framework. The code is available at https://github.com/giorgospets/reglue .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑