控制蛋白质语言模型中的重复
Controlling Repetition in Protein Language Models
AI总结:
本文提出UCCS方法,通过受约束数据集引导蛋白质生成,有效减少重复并保持折叠可靠性。
摘要:
蛋白质语言模型(PLMs)已推动了结构预测和从头蛋白质设计的发展,但它们在生成过程中经常陷入病理性的重复。与文本不同,在蛋白质中,重复会削弱结构信心和功能可行性。为统一这一问题,我们首次系统研究了PLMs中的重复。我们首先提出了定量指标来表征基序级和同多聚体重复,然后展示了其对折叠可靠性的负面影响。为解决这一挑战,我们提出了UCCS(效用受控对比引导),它通过受约束的数据集引导蛋白质生成。不同于简单地对比高重复与低重复序列,我们构建了对比集,以最大化重复差异的同时紧密控制结构效用。这种解耦产生了专门针对重复的引导向量,而不会降低折叠性。在推理过程中注入这些向量,可以持续减少重复,而无需重新训练或启发式解码。在CATH、UniRef50和SCOP上使用ESM-3和ProtGPT2的实验表明,我们的方法优于解码惩罚和其他基线,显著降低重复的同时保持AlphaFold置信度分数。我们的结果确立了重复控制作为PLMs的核心挑战,并突显了数据集引导的引导作为可靠蛋白质生成的原理性方法。
英文摘要:
Protein language models (PLMs) have enabled advances in structure prediction and de novo protein design, yet they frequently collapse into pathological repetition during generation. Unlike in text, where repetition merely reduces readability, in proteins it undermines structural confidence and functional viability. To unify this problem, we present the first systematic study of repetition in PLMs. We first propose quantitative metrics to characterize motif-level and homopolymer repetition and then demonstrate their negative impact on folding reliability. To address this challenge, we propose UCCS (Utility-Controlled Contrastive Steering), which steers protein generation with a constrained dataset. Instead of naively contrasting high- vs. low-repetition sequences, we construct contrastive sets that maximize differences in repetition while tightly controlling for structural utility. This disentanglement yields steering vectors that specifically target repetition without degrading foldability. Injected at inference, these vectors consistently reduce repetition without retraining or heuristic decoding. Experiments with ESM-3 and ProtGPT2 in CATH, UniRef50, and SCOP show that our method outperforms decoding penalties and other baselines, substantially lowering repetition while preserving AlphaFold confidence scores. Our results establish repetition control as a central challenge for PLMs and highlight dataset-guided steering as a principled approach for reliable protein generation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。