OmniThoughtVis: 一种可扩展的蒸馏流水线，用于可部署的多模态推理模型
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
AI总结：
本文提出OmniThoughtVis流水线，通过大规模多模态Co T监督将高容量教师模型的推理能力转移到小型部署模型，实现了在多个基准上的性能提升。
摘要：
近年来，多模态大语言模型（MLLMs）在视觉-语言任务中展示了强大的链式推理能力，但其在现实系统中的直接部署常受限于延迟和资源约束。在实践中，较小的MLLMs更受青睐于在线服务，但其推理性能受限于缺乏大规模、高质量的多模态CoT监督。本文提出了OmniThoughtVis，一种可扩展的数据整理和蒸馏流水线，用于将多模态推理能力从高容量教师模型转移到较小的部署导向MLLMs。从多样化的开源种子池开始，我们的流水线生成结构化的CoT轨迹，并进行推理难度、答案质量和语义任务标签的联合注释。为了在大规模下保持数据质量，我们结合基于规则的过滤、难度感知选择和基于标签的多样性采样，最终得到一个包含180万个样本的整理语料库，支持可控的子集构建以供下游训练。我们使用OmniThoughtVis将Qwen3-VL模型从2B参数蒸馏到8B参数，并在九个多模态推理基准上进行评估。所得到的蒸馏模型在不同模型规模上都表现出一致的提升，包括在MathVerse上提升高达+16.8分，在MMMU-Pro上提升+5.6分。值得注意的是，蒸馏后的4B模型在某些任务上与未蒸馏的8B基线相当或优于它，突显了可扩展推理蒸馏在部署导向MLLMs中的实际价值。
英文摘要：
Recent multimodal large language models (MLLMs) have shown strong chain-of-thought (CoT) reasoning ability on vision-language tasks, but their direct deployment in real-world systems is often limited by latency and resource constraints. In practice, smaller MLLMs are preferred for online serving, yet their reasoning performance is bottlenecked by the lack of large-scale, high-quality multimodal CoT supervision. In this paper, we present OmniThoughtVis, a scalable data curation and distillation pipeline for transferring multimodal reasoning capabilities from high-capacity teacher models to smaller, deployment-oriented MLLMs. Starting from a diverse open-source seed pool, our pipeline generates structured CoT traces and performs joint annotation of reasoning difficulty, answer quality, and semantic task tags. To maintain data quality at scale, we combine rule-based filtering, difficulty-aware selection, and tag-based diversity sampling, resulting in a curated corpus of 1.8M samples that supports controllable subset construction for downstream training. We use OmniThoughtVis to distill Qwen3-VL models from 2B to 8B parameters and evaluate them on nine multimodal reasoning benchmarks. The resulting distilled models show consistent gains across model scales, including improvements of up to +16.8 points on MathVerse and +5.6 points on MMMU-Pro for the 4B model. Notably, the distilled 4B model matches or surpasses the undistilled 8B baseline on several tasks, highlighting the practical value of scalable reasoning distillation for deployment-oriented MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
