OmniMoE-VL：一种具有耦合视觉-深度路由的稀疏视觉-语言模型
OmniMoE-VL: A Sparse Vision-Language Model with Coupled Visual-Depth Routing
浏览论文内容
中文总结 AI 辅助
OmniMoE-VL通过耦合视觉-深度路由投影器，为每个图像-提示对选择稀疏中间视觉深度，实现问题依赖的视觉访问，在八个基准上以28B总参数和9B激活参数达到85.9平均分，验证了耦合与条件化视觉访问的有效性。
中文摘要 AI 辅助
视觉-语言模型（VLMs）越来越多地使用稀疏混合专家（MoE）来扩展语言侧的计算，然而视觉信息通常仅在通过固定的跨模态接口之后才被路由。这留下了一个重要的未决决策：对于给定的问题，哪些中间视觉表示应该暴露给语言计算？我们引入了OmniMoE-VL，一种具有耦合视觉-深度路由投影器的稀疏VLM。对于每个图像-提示对，投影器选择一组稀疏的中间视觉深度，并重用由此产生的全局偏好来指导局部补丁融合和向语言模型的动态视觉注入。这种设计实现了依赖于问题的视觉访问，同时保留了原生视觉标记序列，并补充了视觉和语言堆栈中的标记级专家路由。在八个基于图像的基准测试中，OmniMoE-VL以280亿总参数和90亿激活参数实现了85.9的平均得分。对照比较表明，路由视觉接口提供了主要的架构增益，而匹配的路由和组件控制、相同图像路由分析以及路由干预进一步支持了耦合和问题条件化视觉访问的价值。
英文摘要
Vision-language models (VLMs) increasingly use sparse mixture-of-experts (MoE) to scale language-side computation, yet visual information is typically routed only after passing through a fixed cross-modal interface. This leaves an important decision unresolved: which intermediate visual representations should be exposed to language computation for a given question? We introduce OmniMoE-VL, a sparse VLM with a coupled visual-depth routed projector. For each image-prompt pair, the projector selects a sparse set of intermediate visual depths and reuses the resulting global preference to guide both local patch fusion and dynamic visual injection into the language model. This design enables question-dependent visual access while preserving the native visual-token sequence, and complements token-level expert routing in the vision and language stacks. Across eight image-based benchmarks, OmniMoE-VL achieves an average score of 85.9 with 28B total and 9B activated parameters. Controlled comparisons show that the routed visual interface provides the dominant architectural gain, while matched route and component controls, same-image route analysis, and route interventions further support the value of coupling and question-conditioned visual access.
