AI 中文总结
针对视觉-语言混合专家模型的负载不平衡问题，提出ReBA方法，通过引入图像和文本的独立路由项，在保持任务准确率的同时降低了不同场景下的负载。
AI 中文摘要
视觉-语言混合专家（MoE）批次包含不同数量的图像和文本token，图像分辨率、图像数量、分块（tiling）以及提示长度都会改变这种token组合。我们将标准的token级Switch辅助损失称为Std-Aux，Std-Aux仅平衡混合负载，因此在某一种token组合下，图像和文本的大负载误差可能相互抵消。在我们的主模型上，同一训练好的路由器在不同图像分辨率下的负载不平衡程度变化超过五倍。我们固定图像和文本的负载分布，推导了token组合变化时的精确负载曲线，图像-文本负载差距控制着对token组合的敏感性，物理预处理也会改变条件分布，而固定分布定律不包含此类变化。为设计解决方案，我们研究路由器的输入结构，图像和文本占据不同区域，视觉token按源图像强分组，模态边界催生了独立的图像和文本项，图像边界则催生了每图像一个等权重路由实例。ReBA（即Relax Within, Balance Across）实现了这两种选择。在四个拆分主干网络上，ReBA在所有报告的基准输入上均降低了负载，同时保持平均任务准确率与Std-Aux相当，ReBA还降低了测试范围内的平均负载，以及分辨率和分块变化下的最坏物理负载。代码可在https URL获取。
英文摘要
Vision-language MoE batches contain different numbers of image and text tokens. Image resolution, image count, tiling, and prompt length all change this token mix. We call the standard token-level Switch auxiliary loss Std-Aux. Std-Aux balances only the mixed load, so large image and text load errors can cancel at one mix. On our main model, the same trained router shows more than a fivefold change in load imbalance across image resolutions. We hold the image and text load profiles fixed and derive the exact load curve as the token mix varies. The image-text load gap controls sensitivity to the token mix. Physical preprocessing can also change the conditional profiles. The fixed-profile law excludes such changes. To design a remedy, we examine the router input structure. Image and text occupy distinct regions, while visual tokens group strongly by source image. The modality boundary motivates separate image and text terms. The image boundary motivates one equal-weight routing instance per image. ReBA, or Relax Within, Balance Across, implements both choices. Across four split backbones, ReBA lowers load on every reported benchmark input while keeping mean task accuracy comparable to Std-Aux. ReBA also lowers average load over the tested range and worst physical load under resolution and tiling shifts. Code is available at https://github.com/ZiangWu-77/ReBA.
Comments22 pages, including appendices. Code available at https://github.com/ZiangWu-77/ReBA
