构建生物学可靠蛋白质语言模型的语言学启发路线图
Linguistically inspired roadmap for building biologically reliable protein language models
AI总结：
本文提出一条语言学启发的蛋白质语言模型构建路线图，从训练数据、分词、嵌入和模型解释等环节引入语言学规则提取思想，以提升模型可解释性并揭示序列-功能关系的生物学机制。
摘要：
基于深度神经网络的语言模型（LMs）正越来越多地应用于大规模蛋白质序列数据，以预测蛋白质功能。然而，由于目前的蛋白质语言模型方法在很大程度上是黑箱模型，因而难以解释，它们无助于从根本上理解序列与功能之间的映射关系，从而阻碍了基于规则的生物治疗药物开发。我们认为，从语言学中汲取的指导——语言学是一个专门从自然语言数据中进行分析性规则提取的领域——有助于构建更具可解释性的蛋白质语言模型，使其更有可能学习到相关的领域特定规则。蛋白质序列数据与语言序列数据之间的差异，要求蛋白质语言模型相比自然语言语言模型集成更多的领域特定知识。在此，我们针对训练数据、分词、词元嵌入、序列嵌入和模型解释等方面，提供了一条基于语言学的蛋白质语言模型流程选择路线图。将语言学思想融入蛋白质语言模型，能够推动下一代可解释机器学习模型的发展，并有望揭示序列与功能关系背后的生物学机制。
英文摘要：
Deep neural-network-based language models (LMs) are increasingly applied to large-scale protein sequence data to predict protein function. However, being largely black-box models and thus challenging to interpret, current protein LM approaches do not contribute to a fundamental understanding of sequence-function mappings, hindering rule-based biotherapeutic drug development. We argue that guidance drawn from linguistics, a field specialized in analytical rule extraction from natural language data, can aid with building more interpretable protein LMs that are more likely to learn relevant domain-specific rules. Differences between protein sequence data and linguistic sequence data require the integration of more domain-specific knowledge in protein LMs compared to natural language LMs. Here, we provide a linguistics-based roadmap for protein LM pipeline choices with regard to training data, tokenization, token embedding, sequence embedding, and model interpretation. Incorporating linguistic ideas into protein LMs enables the development of next-generation interpretable machine-learning models with the potential of uncovering the biological mechanisms underlying sequence-function relationships.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
