LatentRouter: 在看到答案之前，我们能否选择合适的多模态模型？
LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?
AI总结：
LatentRouter通过多模态效用预测实现多模态大语言模型的路由，通过隐式通信和胶囊修正提升模型选择准确性，在MMR-Bench和VL-RouterBench实验中优于基线方法。
摘要：
多模态大语言模型（MLLMs）在OCR、图表理解、空间推理、视觉问答、成本和延迟等方面具有异质性优势。有效的MLLM路由需要超越估计查询难度：路由器必须将当前图像-问题输入的多模态需求与每个候选模型的能力匹配。我们提出了LatentRouter，将MLLM路由形式化为反事实多模态效用预测。给定图像-问题查询，LatentRouter提取学习到的多模态路由胶囊，用模型能力令牌表示每个候选MLLM，并在这些状态之间进行隐式通信以估计选择每个模型的性能。分布性结果头预测模型特定的反事实质量，而有界的胶囊修正通过共享每模型评分和可用性遮蔽处理变化的候选池。实验表明，LatentRouter在MMR-Bench和VL-RouterBench上优于固定模型、特征级和学习路由基线。额外分析显示，增益在多模态任务组中最强，当模型选择依赖于视觉、布局敏感或推理导向需求时，隐式通信是主要贡献者。代码可在https://github.com/LabRAI/LatentRouter获取。
英文摘要：
Multimodal large language models (MLLMs) have heterogeneous strengths across OCR, chart understanding, spatial reasoning, visual question answering, cost, and latency. Effective MLLM routing therefore requires more than estimating query difficulty: a router must match the multimodal requirements of the current image-question input with the capabilities of each candidate model. We propose LatentRouter, a router that formulates MLLM routing as counterfactual multimodal utility prediction. Given an image-question query, LatentRouter extracts learned multimodal routing capsules, represents each candidate MLLM with a model capability token, and performs latent communication between these states to estimate how each model would perform if selected. A distributional outcome head predicts model-specific counterfactual quality, while a bounded capsule correction refines close decisions without allowing residual signals to dominate the prediction. The resulting utility-based policy supports performance-oriented and performance-cost routing, and handles changing candidate pools through shared per-model scoring with availability masking. Experiments on MMR-Bench and VL-RouterBench show that LatentRouter outperforms fixed-model, feature-level, and learned-router baselines. Additional analyses show that the gains are strongest on multimodal task groups where model choice depends on visual, layout-sensitive, or reasoning-oriented requirements, and that latent communication is the main contributor to the improvement. The code is available at: https://github.com/LabRAI/LatentRouter.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
