Astrea：一种基于MOE的渐进式对齐视觉理解模型
Astrea: A MOE-based Visual Understanding Model with Progressive Alignment
AI总结：
针对视觉语言模型中异构专家负载不均的问题，提出基于渐进式预对齐的多专家协作架构Astrea，通过异构专家协调、动态知识融合与增强优化框架，在12项基准任务上平均性能提升4.7%。
摘要：
基于混合专家架构的视觉语言模型已成为多模态理解的关键范式，为整合视觉与语言信息提供了强大框架。然而，任务日益复杂多样，在协调异构视觉专家间的负载均衡方面带来了重大挑战，优化某一专家的性能往往会损害其他专家的能力。为解决任务异构性与专家负载不平衡问题，我们提出Astrea，一种基于渐进式预对齐的新型多专家协作VLM架构。Astrea引入三项关键创新：1）异构专家协调机制，将四个专门化模型（检测、分割、分类、描述）整合为涵盖基本视觉理解要素的综合专家矩阵；2）动态知识融合策略，采用渐进式预对齐通过对比学习在VLM潜在空间内协调专家，辅以概率激活的随机残差连接以保持知识连续性；3）增强的优化框架，利用动量对比学习进行长程依赖建模，并采用自适应权重分配器进行实时专家贡献校准。在跨越VQA、图像描述和跨模态检索的12项基准任务上的广泛评估表明，Astrea优于现有最先进模型，平均性能提升+4.7%。本研究首次实证证明，渐进式预对齐策略使VLM能够克服任务异构性限制，为开发通用多模态智能体奠定了新的方法论基础。
英文摘要：
Vision-Language Models (VLMs) based on Mixture-of-Experts (MoE) architectures have emerged as a pivotal paradigm in multimodal understanding, offering a powerful framework for integrating visual and linguistic information. However, the increasing complexity and diversity of tasks present significant challenges in coordinating load balancing across heterogeneous visual experts, where optimizing one specialist's performance often compromises others' capabilities. To address task heterogeneity and expert load imbalance, we propose Astrea, a novel multi-expert collaborative VLM architecture based on progressive pre-alignment. Astrea introduces three key innovations: 1) A heterogeneous expert coordination mechanism that integrates four specialized models (detection, segmentation, classification, captioning) into a comprehensive expert matrix covering essential visual comprehension elements; 2) A dynamic knowledge fusion strategy featuring progressive pre-alignment to harmonize experts within the VLM latent space through contrastive learning, complemented by probabilistically activated stochastic residual connections to preserve knowledge continuity; 3) An enhanced optimization framework utilizing momentum contrastive learning for long-range dependency modeling and adaptive weight allocators for real-time expert contribution calibration. Extensive evaluations across 12 benchmark tasks spanning VQA, image captioning, and cross-modal retrieval demonstrate Astrea's superiority over state-of-the-art models, achieving an average performance gain of +4.7\%. This study provides the first empirical demonstration that progressive pre-alignment strategies enable VLMs to overcome task heterogeneity limitations, establishing new methodological foundations for developing general-purpose multimodal agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
