arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2606.03444cs.CVcs.AI

PRISM: 通过自组织专家专业化协同视觉基础模型

PRISM: Synergizing Vision Foundation Models via Self-organized Expert Specialization

University of Science and Technology of China(中国科学技术大学)

Ying Tang, Dong Li, Youjia Zhang, Zikai Song, Junqing Yu, Wei Yang

AI总结:

提出PRISM框架,采用双流混合专家(MoE)架构,通过两阶段范式(先解构专家知识使其专业化,再动态重组为任务特定路径)解决视觉基础模型集成中的负迁移问题,在PASCAL-Context和NYUD-v2上达到新最优。

摘要:

将多种视觉基础模型(VFM)的互补优势统一到单个高效模型中是非常理想的,但受到整体蒸馏中固有的负迁移的挑战。为了解决这些特征冲突,我们引入了 extbf{PRISM},一种新颖的双流混合专家(MoE)框架,通过模块化专业化协同VFM。我们提出了一个两阶段范式:(1)专业知识解构,其中教师条件路由器引导专家在不同的表示子空间中专业化以减轻干扰,然后(2)动态重组,其中路由器学习将这些专家组装成针对下游任务定制的计算路径。在PASCAL-Context和NYUD-v2上的实验表明, extbf{PRISM}建立了新的最先进水平,验证了稀疏、涌现的专业化是集成多样化视觉知识的可扩展方法。

英文摘要:

Unifying the complementary strengths of diverse Vision Foundation Models (VFMs) into a single efficient model is highly desirable but challenged by the negative transfer inherent in monolithic distillation. To address these feature conflicts, we introduce \textbf{PRISM}, a novel dual-stream Mixture-of-Experts (MoE) framework that synergizes VFMs via modular specialization. We propose a two-stage paradigm: (1) expertise deconstruction, where a teacher-conditional router guides experts to specialize in distinct representational subspaces to mitigate interference, followed by (2) dynamic recomposition, where the router learns to assemble these experts into tailored computational pathways for downstream tasks. Experiments on PASCAL-Context and NYUD-v2 show that \textbf{PRISM} establishes a new state of the art, validating that sparse, emergent specialization is a scalable approach for integrating diverse visual knowledge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑