bioMoR：面向有效基因组学习的生物学引导递归混合模型

中文摘要 AI 辅助

用于高维组学分析的Transformer模型需处理数千个基因或通路，其中仅需对一部分进行深度计算。递归混合模型（Mixture-of-Recursions, MoR）通过自适应令牌选择或专家选择路由提升效率。我们提出的bioMoR，据我们所知，是首个将MoR应用于基因水平和通路水平学习的框架。我们的贡献包括确定了在MoR主干中整合结构化生物学知识的三个位置：基于图的信息共享优化令牌嵌入；结构偏差引导自注意力关注生物学相关令牌；图感知路由器利用邻域信息确定每个令牌的递归深度。这些技术围绕我们的核心见解展开：令牌交互的额外知识可有效帮助模型构建嵌入并选择需更深度学习的令牌。在涵盖不同组学数据类型的8个基准测试中，采用统一的五折交叉验证协议评估，bioMoR相比最强的生物学无关MoR基线，平均宏F1提升8.2个百分点，平衡准确率提升7.1个百分点，同时参数减少75%，且相比非递归Transformer，FLOPs最多减少58%。所选标记基因或通路具备生物学可解释性，而它们的令牌特定递归深度则揭示了计算资源的分配方式。

英文摘要

Transformer models for high-dimensional omics analysis process thousands of genes or pathways, although only a subset requires deep computation. Mixture-of-Recursions (MoR) improves efficiency through adaptive token-choice or expert-choice routing. We propose bioMoR, which, to the best of our knowledge, is the first framework to apply MoR to gene-level and pathway-level learning. Our contributions include identifying three locations for integrating structured biological knowledge within an MoR backbone: graph-based information sharing refines token embeddings, a structural bias guides self-attention toward biologically related tokens, and a graph-aware router uses neighborhood information to determine each token's recursion depth. These techniques are centered on our insight that additional knowledge of token interaction can effectively help models construct embeddings and select which tokens should be learned more deeply. Across eight benchmarks spanning diverse omics data types and evaluated under a unified five-fold cross-validation protocol, bioMoR improves average macro-F1 by 8.2 percentage points and balanced accuracy by 7.1 percentage points over the strongest biology-agnostic MoR baseline while using 75 percent fewer parameters and up to 58 percent fewer FLOPs than a non-recursive Transformer. The selected marker genes or pathways provide biological interpretability, while their token-specific recursion depths reveal how computation is allocated.

发表机构

• Iowa State University（爱荷华州立大学）
• Stanford University（斯坦福大学）

机构由 AI 辅助整理，请以论文原文为准。