测试时匹配：在多模态模型中解锁组合推理
Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models
AI总结：
本文提出测试时匹配算法，通过改进评估指标提升多模态模型的组合推理能力，使SigLIP-B16和GPT-4.1在Winoground等基准上取得新突破。
摘要：
前沿AI模型虽取得显著进展，但近期研究表明其在组合推理任务上表现不佳，常在基准测试中表现低于随机猜测。本文重新审视该问题，发现常用评估指标系统性低估了模型能力。为纠正这一偏差，我们引入了群体匹配分数以更准确评估模型能力。此外，新指标下的正确性可通过简单过拟合步骤转换为旧指标下的正确性。这一调整使SigLIP-B16超越所有先前结果，GPT-4.1在Winoground上首次超越估计的人类表现。基于此洞察，我们提出测试时匹配（TTM），一种迭代、自我改进的算法，无需外部监督即可进一步提升模型性能。TTM带来额外非平凡改进：例如，TTM使SigLIP-B16在MMVP-VLM上超越GPT-4.1，建立新的最先进水平。TTM还扩展至对比视觉语言模型之外，使生成式多模态模型在多个基准上取得明显提升。重要的是，TTM在无指标诱导效应或群体结构的基准上仍表现良好，在挑战性数据集如WhatsUp上实现高达85.7%的相对提升。在16个跨多样设置的数据集变体上，实验表明TTM一致提升模型性能，推动组合推理前沿。
英文摘要：
Frontier AI models have achieved remarkable progress, yet recent studies suggest they struggle with compositional reasoning, often performing at or below random chance on established benchmarks. We revisit this problem and show that widely used evaluation metrics systematically underestimate model capability. To correct this artifact, we introduce a group matching score that more faithfully evaluates model capability. Moreover, correctness under the new metric can be translated into correctness under existing metrics via a simple overfitting step. This adjustment enables SigLIP-B16 to surpass all previous results and GPT-4.1 to yield the first result surpassing estimated human performance on Winoground. Building on this insight, we propose Test-Time Matching (TTM), an iterative, self-improving algorithm that further bootstraps model performance without any external supervision. TTM delivers additional, non-trivial improvements: for example, TTM enables SigLIP-B16 to surpass GPT-4.1 on MMVP-VLM, establishing a new state of the art. TTM also extends beyond contrastive vision-language models, yielding clear gains on a generative multimodal model across benchmarks. Importantly, TTM remains broadly effective even on benchmarks without metric-induced effects or group structures, achieving relative gains up to 85.7% on challenging datasets such as WhatsUp. Across 16 dataset variants spanning diverse setups, our experiments demonstrate that TTM consistently improves model performance and advances the frontier of compositional reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
