重新思考基于文本的蛋白质理解：检索还是大语言模型？
Rethinking Text-based Protein Understanding: Retrieval or LLM?
AI总结：
本文揭示现有蛋白质-文本基准存在数据泄漏和评估指标不适配问题，提出基于生物实体的新评估框架及检索增强方法，在蛋白质到文本生成上优于微调大语言模型且支持免训练场景。
摘要：
近年来，蛋白质-文本模型因其在蛋白质生成和理解方面的潜力而受到广泛关注。当前方法侧重于通过持续预训练和多模态对齐将蛋白质相关知识整合到大语言模型中，从而同时理解文本描述和蛋白质序列。通过对现有模型架构和基于文本的蛋白质理解基准的深入分析，我们发现当前基准中存在显著的数据泄漏问题。此外，源自自然语言处理的传统评估指标无法准确评估模型在该领域的性能。为了解决这些局限性，我们重新整理了现有数据集，并引入了一个基于生物实体的新型评估框架。基于我们的观察，我们提出了一种检索增强方法，该方法在蛋白质到文本生成方面显著优于微调后的大语言模型，并在免训练场景中展现出准确性和效率。我们的代码和数据可在 https://github.com/IDEA-XL/RAPM 查看。
英文摘要：
In recent years, protein-text models have gained significant attention for their potential in protein generation and understanding. Current approaches focus on integrating protein-related knowledge into large language models through continued pretraining and multi-modal alignment, enabling simultaneous comprehension of textual descriptions and protein sequences. Through a thorough analysis of existing model architectures and text-based protein understanding benchmarks, we identify significant data leakage issues present in current benchmarks. Moreover, conventional metrics derived from natural language processing fail to accurately assess the model's performance in this domain. To address these limitations, we reorganize existing datasets and introduce a novel evaluation framework based on biological entities. Motivated by our observation, we propose a retrieval-enhanced method, which significantly outperforms fine-tuned LLMs for protein-to-text generation and shows accuracy and efficiency in training-free scenarios. Our code and data can be seen at https://github.com/IDEA-XL/RAPM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
