arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.03574cs.LGcs.CV
收藏

小视觉语言模型的高效测试时缩放

Efficient Test-Time Scaling for Small Vision-Language Models

Technical University of Denmark(丹麦技术大学);Pioneer Center for AI(先锋人工智能中心);University of Copenhagen(哥本哈根大学)

Mehmet Onurcan Kaya, Desmond Elliott, Dim P. Papadopoulos

AI总结:

本文提出两种高效测试时缩放策略,通过模型内部特征提升小视觉语言模型的性能,同时保持计算效率。

摘要:

小视觉语言模型(VLMs)提供了一种计算效率较高的替代方案,但以泛化能力较弱和下游任务性能较差为代价。这些不足可以通过测试时缩放技术来解决,但现有方法通常计算成本较高,与小模型的资源高效设计目标相悖。为了解决这些限制,我们提出两种新颖且高效的测试时缩放策略,利用模型内部特征而非外部监督:(i)测试时增强(TTAug),生成多个增强输入并在令牌层面聚合输出,无需参数更新;(ii)测试时适应(TTAdapt),在推理过程中利用TTAug生成的基于共识的伪标签对模型参数进行适应。通过在九个基准测试中进行广泛实验,我们展示了在保持计算效率以适应资源受限环境的同时,一致的性能提升。我们的方法的通用性在不同规模的模型和不同VLMs之间均得到验证,无需额外调优。

英文摘要:

Small Vision-Language Models (VLMs) provide a computationally efficient alternative to larger models, at the cost of weaker generalization abilities and downstream task performance. These shortcomings could be addressed by test-time scaling techniques, but existing methods are typically computationally demanding, contradicting the resource-efficient design goals of small models. To address these limitations, we propose two novel and efficient test-time scaling strategies that leverage the model-internal features rather than external supervision: (i) Test-Time Augmentation (TTAug), which generates multiple augmented inputs and aggregates outputs at the token level without parameter updates, and (ii) Test-Time Adaptation (TTAdapt), which adapts model parameters during inference using consensus-based pseudolabels from TTAug. Through extensive experiments across nine benchmarks, we demonstrate consistent performance improvements while maintaining computational efficiency suitable for resource-constrained environments. The generality of our approach is demonstrated both within models at different scales and across different VLMs without additional tuning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑