arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.04670cs.AI
收藏

改进多模态脑编码模型的动态主体感知路由

Improving Multimodal Brain Encoding Model with Dynamic Subject-awareness Routing

Xuanhua Yin, Runkai Zhao, Weidong Cai

AI总结:

AFIRE和MIND通过动态主体感知路由提升多模态脑编码模型的性能,增强跨受试者泛化能力并实现可解释的专家模式。

摘要:

自然 fMRI编码必须处理多模态输入、转换融合风格以及显著的跨受试者差异。我们引入了AFIRE(多模态fMRI响应编码的无偏框架),一个无偏接口,它标准化了来自各种编码器的对齐时间后融合标记,并引入了MIND,一个即插即用的专家混合解码器,具有主体感知的动态门控。通过端到端训练进行全脑预测,AFIRE将解码器与上游融合解耦,而MIND结合了基于标记的Top-K稀疏路由和主体先验,以个性化专家使用而不牺牲通用性。在多个多模态backbone和受试者上的实验显示,与强基线相比有持续的改进,增强了跨受试者泛化能力,并且可解释的专家模式与内容类型相关联。该框架为新编码器和数据集提供了简单的附加点,使自然神经影像研究能够实现稳健的、即插即用的性能。

英文摘要:

Naturalistic fMRI encoding must handle multimodal inputs, shifting fusion styles, and pronounced inter-subject variability. We introduce AFIRE (Agnostic Framework for Multimodal fMRI Response Encoding), an agnostic interface that standardizes time-aligned post-fusion tokens from varied encoders, and MIND, a plug-and-play Mixture-of-Experts decoder with a subject-aware dynamic gating. Trained end-to-end for whole-brain prediction, AFIRE decouples the decoder from upstream fusion, while MIND combines token-dependent Top-K sparse routing with a subject prior to personalize expert usage without sacrificing generality. Experiments across multiple multimodal backbones and subjects show consistent improvements over strong baselines, enhanced cross-subject generalization, and interpretable expert patterns that correlate with content type. The framework offers a simple attachment point for new encoders and datasets, enabling robust, plug-and-improve performance for naturalistic neuroimaging studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑