arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.22492cs.CVcs.AIcs.MM
收藏

微小推理时间缩放与潜在验证器

Tiny Inference-Time Scaling with Latent Verifiers

University of Modena and Reggio Emilia(摩德纳和雷吉奥艾米利亚大学);University of Pisa(比萨大学)

Davide Bucciarelli, Evelyn Turri, Lorenzo Baraldi, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

AI总结:

本文提出VHS验证器,直接在扩散变换器单步生成器的中间隐藏表示上操作,减少验证成本并提升性能,实现更高效的推理时间缩放。

摘要:

在推理时间缩放中,使用验证器评分和选择候选输出是一种有效方法。常见的选择是使用多模态大语言模型(MLLM)作为验证器,虽然能提升性能但引入了显著的推理时间成本。扩散管道在自动编码器的潜在空间中操作以减少计算,但MLLM验证器仍需将候选解码到像素空间并重新编码到视觉嵌入空间,导致冗余且昂贵的操作。本文提出验证器在隐藏状态(VHS),一种直接在扩散变换器(DiT)单步生成器的中间隐藏表示上操作的验证器。VHS无需解码到像素空间即可分析生成器特征,从而减少每个候选的验证成本,同时提高或匹配MLLM基的竞争对手性能。我们显示,在仅有少量候选每个提示的微小推理预算下,VHS实现了更高效的推理时间缩放,将联合生成和验证时间减少63.3%,计算FLOPs减少51%,VRAM使用减少14.5%,在相同推理时间预算下在GenEval上实现+2.7%的改进。

英文摘要:

Inference-time scaling has emerged as an effective way to improve generative models at test time by using a verifier to score and select candidate outputs. A common choice is to employ Multimodal Large Language Models (MLLMs) as verifiers, which can improve performance but introduce substantial inference-time cost. Indeed, diffusion pipelines operate in an autoencoder latent space to reduce computation, yet MLLM verifiers still require decoding candidates to pixel space and re-encoding them into the visual embedding space, leading to redundant and costly operations. In this work, we propose Verifier on Hidden States (VHS), a verifier that operates directly on intermediate hidden representations of Diffusion Transformer (DiT) single-step generators. VHS analyzes generator features without decoding to pixel space, thereby reducing the per-candidate verification cost while improving or matching the performance of MLLM-based competitors. We show that, under tiny inference budgets with only a small number of candidates per prompt, VHS enables more efficient inference-time scaling reducing joint generation-and-verification time by 63.3%, compute FLOPs by 51% and VRAM usage by 14.5% with respect to a standard MLLM verifier, achieving a +2.7% improvement on GenEval at the same inference-time budget.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑