用几何注释解释潜在蛋白质语言模型的特征
Interpreting Latent Protein Language Model Features with Geometric Annotations
浏览论文内容
中文总结 AI 辅助
本研究提出基于蛋白质Cα主链几何特征的自动可扩展方法，解释ESM-2的SAE特征，发现局部几何与多数SAE特征显著相关，可区分同数据库注释的SAE特征，为残基级SAE神经元激活蛋白质注释提供了稳健方法，搭建了机制可解释性与结构生物学的桥梁。
中文摘要 AI 辅助
蛋白质语言模型（pLMs）编码蛋白质序列信息，可用于结构预测等下游任务，但其内部表征尚未被充分理解。稀疏自编码器（SAEs）是将潜在pLM表征解纠缠为可解释特征的有前景工具，但现有注释流程主要依赖数据库标签衍生的蛋白质级注释和对最高激活序列的大语言模型（LLM）注释，这类注释可能忽略稀疏特征编码的局部残基级和几何模式。我们引入一种自动且可扩展的方法，利用蛋白质Cα主链的几何启发特征解释ESM-2中的SAE特征。在ESM-2的8M层中，FDR控制的发现分析显示，局部几何与许多SAE特征显著相关，且具有不同的预测强度，其覆盖范围超出了基于数据库和序列的方法。特别地，几何可区分具有相同数据库注释的SAE特征，揭示了已知生物标签内的子结构。相当一部分SAE特征在未注释的宏基因组蛋白质序列上激活，使我们能够利用SAE注释更好地理解这些序列。此外，接触预测层面的消融实验表明，移除已发现的几何特征会使ESM-2的预测接触图向描述符方向偏移。这提供了一种稳健的残基级SAE神经元激活蛋白质注释方法，为机制可解释性与结构生物学搭建了桥梁。
英文摘要
Protein language models (pLMs) encode information about protein sequences which enable downstream tasks such as structure prediction, but their internal representations are not well understood. Sparse autoencoders (SAEs) provide a promising tool to disentangle latent pLM representations into interpretable features, but existing annotation pipelines largely rely on protein-level annotations derived from database labels and LLM annotations of top activating sequences. Such annotations can overlook the localized residue-level and geometric patterns encoded by sparse features. We introduce an automated and scalable method for interpreting SAE features in ESM-2 by using geometrically inspired features of the protein $\text{C}_α$ backbone. Across ESM-2 8M layers, an FDR-controlled discovery analysis shows that local geometry is significantly associated with many SAE features, with varying levels of predictive strength, expanding coverage beyond database and sequence-based methods. In particular, geometry can distinguish SAE features sharing the same database annotation, revealing substructure within known biological labels. A significant portion of SAE features activate on unannotated metagenomic protein sequences enabling us to use our SAE annotations to better understand these sequences. In addition, ablation experiments at the level of contact prediction show that removing found geometric features shifts ESM-2's predicted contact maps in the direction of the descriptor. This provides a robust method of annotating proteins activated within SAE neurons at a residue level, providing a bridge between mechanistic interpretability and structural biology.
发表机构
- School of Mathematics（数学学院）
- University of Edinburgh（爱丁堡大学）
机构由 AI 辅助整理，请以论文原文为准。
