arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.17680cs.LG
收藏

BioBridge: 蛋白质与语言的桥梁:利用大语言模型增强生物推理

BioBridge: Bridging Proteins and Language for Enhanced Biological Reasoning with LLMs

School of Computer Science and Technology(计算机科学与技术学院);College of Computer Science and Artificial Intelligence(计算机科学与人工智能学院);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室)

Yujia Wang, Jihong Guan, Wengen Li, Shuigeng Zhou, Xuhong Wang

AI总结:

BioBridge结合蛋白质领域知识与通用语言能力,通过领域自适应持续预训练框架提升生物推理性能。

摘要:

现有的蛋白质语言模型(PLMs)常常在多种任务上适应性有限,并且在不同的生物背景下泛化能力差。相比之下,通用的大语言模型(LLMs)缺乏对蛋白质序列的解释能力,在领域特定知识方面也有所不足,限制了其在生物语义推理中的有效性。为了结合两者的优点,我们提出了BioBridge,一种用于蛋白质理解的领域自适应持续预训练框架。该框架采用领域增量持续预训练(DICP)将蛋白质领域知识和通用推理语料库同时注入到LLM中,有效缓解了灾难性遗忘问题。通过PLM-Projector-LLM流水线实现跨模态对齐,将蛋白质序列嵌入映射到语言模型的语义空间中。最终采用端到端优化,统一支持多种任务,包括蛋白质属性预测和知识问答。我们提出的BioBridge在多个蛋白质基准测试中(如EC和BindingDB)的表现与主流PLMs相当。它在通用理解任务(如MMLU和RACE)上也取得了与LLMs相当的结果。这展示了其结合领域特定适应性和通用语言能力的创新优势。

英文摘要:

Existing Protein Language Models (PLMs) often suffer from limited adaptability to multiple tasks and exhibit poor generalization across diverse biological contexts. In contrast, general-purpose Large Language Models (LLMs) lack the capability to interpret protein sequences and fall short in domain-specific knowledge, limiting their capacity for effective biosemantic reasoning. To combine the advantages of both, we propose BioBridge, a domain-adaptive continual pretraining framework for protein understanding. This framework employs Domain-Incremental Continual Pre-training (DICP) to infuse protein domain knowledge and general reasoning corpus into a LLM simultaneously, effectively mitigating catastrophic forgetting. Cross-modal alignment is achieved via a PLM-Projector-LLM pipeline, which maps protein sequence embeddings into the semantic space of the language model. Ultimately, an end-to-end optimization is adopted to uniformly support various tasks, including protein property prediction and knowledge question-answering. Our proposed BioBridge demonstrates performance comparable to that of mainstream PLMs on multiple protein benchmarks, such as EC and BindingDB. It also achieves results on par with LLMs on general understanding tasks like MMLU and RACE. This showcases its innovative advantage of combining domain-specific adaptability with general-purpose language competency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑