探索蛋白质语言模型架构引起的抗体理解偏差
Exploring Protein Language Model Architecture-Induced Biases for Antibody Comprehension
AI总结:
本研究探讨了蛋白质语言模型架构对抗体理解偏差的影响,通过对比不同模型在抗体靶向预测中的表现,揭示了架构选择对生物特征提取的关键作用。
摘要:
最近在蛋白质语言模型(PLMs)方面的进步展示了在理解蛋白质序列方面的显著能力。然而,不同模型架构在捕捉抗体特定的生物特性方面能有多大的能力仍不清楚。在本工作中,我们系统地研究了PLMs中的架构选择如何影响其理解抗体序列特征和功能的能力。我们评估了三种最先进的PLMs——AntiBERTa、BioBERT和ESM2——在抗体靶向特异性预测任务上,与通用语言模型(GPT-2)基线进行比较。我们的结果表明,尽管所有PLMs都实现了高分类准确率,但它们在捕捉生物特征如V基因使用、体细胞超突变模式和同种型信息时表现出不同的偏差。通过注意力归因分析,我们显示抗体特异性模型如AntiBERTa自然学会专注于互补决定区域(CDRs),而通用蛋白质模型显著受益于显式CDR聚焦训练策略。这些发现提供了模型架构与生物特征提取之间关系的见解,为未来在计算抗体设计中的PLM发展提供了有价值的指导。
英文摘要:
Recent advances in protein language models (PLMs) have demonstrated remarkable capabilities in understanding protein sequences. However, the extent to which different model architectures capture antibody-specific biological properties remains unexplored. In this work, we systematically investigate how architectural choices in PLMs influence their ability to comprehend antibody sequence characteristics and functions. We evaluate three state-of-the-art PLMs-AntiBERTa, BioBERT, and ESM2--against a general-purpose language model (GPT-2) baseline on antibody target specificity prediction tasks. Our results demonstrate that while all PLMs achieve high classification accuracy, they exhibit distinct biases in capturing biological features such as V gene usage, somatic hypermutation patterns, and isotype information. Through attention attribution analysis, we show that antibody-specific models like AntiBERTa naturally learn to focus on complementarity-determining regions (CDRs), while general protein models benefit significantly from explicit CDR-focused training strategies. These findings provide insights into the relationship between model architecture and biological feature extraction, offering valuable guidance for future PLM development in computational antibody design.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。