arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.00233q-bio.GNcs.LG
收藏

基于Transformer的单细胞多模态预测

Single-Cell Multimodal Prediction via Transformers

Michigan State University(密歇根州立大学);Emory University(埃默里大学)

Wenzhuo Tang, Hongzhi Wen, Renming Liu, Jiayuan Ding, Wei Jin, Yuying Xie, Hui Liu, Jiliang Tang

AI总结:

针对多模态单细胞数据建模中静态图与GNN的局限,提出端到端Transformer框架scMoFormer,整合外部知识并建模模态内与跨模态交互,在基准数据集上性能优异,并在NeurIPS 2022竞赛中获Kaggle银牌。

摘要:

近年来,多模态单细胞技术的发展使得从单个细胞获取多种组学数据成为可能,从而能够更深入地理解细胞状态和动态变化。然而,多模态单细胞数据的激增也带来了在建模不同模态之间复杂相互作用方面的巨大挑战。近期先进的方法侧重于构建静态交互图,并应用图神经网络(GNNs)从多模态数据中学习。然而,这种静态图可能是次优的,因为它们没有利用下游任务的信息;同时,GNN在深层堆叠时也存在一些固有的局限性。为了解决这些问题,在本工作中,我们研究了如何以端到端的方式利用Transformer处理多模态单细胞数据,同时利用下游任务信息。具体而言,我们提出了scMoFormer框架,该框架能够轻松地整合外部领域知识,并建模每种模态内部以及跨模态的相互作用。大量实验表明,scMoFormer在各种基准数据集上取得了优异的性能。值得注意的是,在NeurIPS 2022竞赛中,scMoFormer在不使用集成方法的情况下获得了Kaggle银牌,排名为24/1221(前2%)。我们的实现已在Github上公开。

英文摘要:

The recent development of multimodal single-cell technology has made the possibility of acquiring multiple omics data from individual cells, thereby enabling a deeper understanding of cellular states and dynamics. Nevertheless, the proliferation of multimodal single-cell data also introduces tremendous challenges in modeling the complex interactions among different modalities. The recently advanced methods focus on constructing static interaction graphs and applying graph neural networks (GNNs) to learn from multimodal data. However, such static graphs can be suboptimal as they do not take advantage of the downstream task information; meanwhile GNNs also have some inherent limitations when deeply stacking GNN layers. To tackle these issues, in this work, we investigate how to leverage transformers for multimodal single-cell data in an end-to-end manner while exploiting downstream task information. In particular, we propose a scMoFormer framework which can readily incorporate external domain knowledge and model the interactions within each modality and cross modalities. Extensive experiments demonstrate that scMoFormer achieves superior performance on various benchmark datasets. Remarkably, scMoFormer won a Kaggle silver medal with the rank of 24/1221 (Top 2%) without ensemble in a NeurIPS 2022 competition. Our implementation is publicly available at Github.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑