简洁是推理效率的灵魂：通过数据策展诱导VLM的简洁性
Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation
AI总结：
本文提出通过预训练数据策展诱导视觉语言模型输出简洁答案，从而降低推理成本，在保持精度的同时实现35倍成本优势。
摘要：
推理效率通常通过缩小模型来实现：蒸馏、剪枝、量化和稀疏路由各自降低了每token成本，同时将token数量视为固定。但输出长度一直在膨胀，而这正是标准工具包未触及的部分。在这里，我们认为简洁是缺失的推理效率杠杆，而预训练数据策展是实现它的实用方法：在简洁、正确的数据上训练的模型学会用更少的token回答；即它具有更低的“成本通过率”。我们将VLM策展流程应用于MAmmoTH-VL单图像子集，并比较了在我们的策展数据、标准MAmmoTH-VL数据和外部开源前沿VLM上训练的模型。在受控的20个评估集和14个1B-4B激活参数的VLM上，我们通过每个模型的回归固定输出长度，将简洁性与质量分离，并以每个正确答案的FLOPs对模型进行定价。策展相对于最冗长的4B比较模型（Qwen3.5-4B）获得了35倍的成本通过率优势，精度相差约1个百分点（每个正确答案0.41 vs 14.58 TFLOPs；平均精度0.691 vs 0.704）。策展还相对于未策展基线获得了+17.55个百分点的匹配长度精度增益，且该增益随模型规模增长（从1B时的+16.7个百分点到4B时的+21.2个百分点）。这种简洁性改进不牺牲质量：通用冗长在任何能力或规模下都不带来精度提升，而推理结构化的冗长仍能赚取其token的窗口从2B时的8个能力组中的4个缩小到4B时的8个中的1个。在每个示例中，简洁模型甚至达到了冗长推理模型遗漏的正确答案，这表明推理是一个独立的策展目标，而不是简洁性所放弃的东西。在此设置中，推理效率是一个每个正确答案的token问题，而简洁性是直接针对它的杠杆。
英文摘要：
Inference efficiency is typically pursued by shrinking the model: distillation, pruning, quantization, and sparse routing each lower per-token cost while treating token count as fixed. But output length has been inflating, and it is precisely the component the standard toolkit leaves untouched. Here, we argue that brevity is the missing inference-efficiency lever, and that pretraining data curation is a practical way to pull it: a model trained on concise, correct data learns to answer in fewer tokens; i.e. it has a lower Cost-of-Pass. We apply our VLM curation pipeline to the MAmmoTH-VL single-image subset, and compare models trained on our curated data, the standard MAmmoTH-VL data, and external open-weight frontier VLMs. On a controlled 20-evaluation set and 14 VLMs at 1B-4B activated parameters, we hold output length fixed with a per-model regression, separating brevity from quality, and price models in FLOPs per correct answer. Curation buys a 35x Cost-of-Pass advantage over the most verbose 4B comparator (Qwen3.5-4B) within $\sim$1 pp of accuracy (0.41 vs 14.58 TFLOPs per correct answer; 0.691 vs 0.704 mean accuracy). Curation also buys a +17.55-percentage-point matched-length accuracy gain over the uncurated baseline that grows with model scale (from +16.7 pp at 1B to +21.2 pp at 4B). This brevity improvement concedes no quality: generic verbosity buys no accuracy at any capability or scale, and the window where reasoning-structured verbosity still earns its tokens shrinks from 4 of 8 capability groups at 2B to 1 of 8 at 4B. Per example, the concise model even reaches correct answers the verbose reasoning model misses, marking reasoning as a distinct curation target rather than something brevity gives up. Inference efficiency in this regime is a tokens-per-correct problem, and brevity is the lever that targets it directly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
