冻结但并非始终可及：基因组语言模型的表征分析
Frozen but Not Always Accessible: A Representation Analysis of Genomic Language Models
浏览论文内容
中文总结 AI 辅助
本研究分析了DNABERT-2等基因组语言模型作为冻结特征提取器时，在不同基因组任务中的表征可及性，发现其性能随任务变化，局部生物信号存在但并非总能通过池化嵌入获取。
中文摘要 AI 辅助
基因组基础模型越来越多地被用作冻结的特征提取器，用于下游序列预测，为全微调提供了一种计算高效的替代方案。然而，目前尚不清楚在无需特定任务适配的情况下，这些模型编码的生物信息何时可被获取。我们对冻结的基因组语言模型在调控、表观遗传、启动子、剪接位点和变异效应预测任务中的表征可及性展开分析。我们在统一的冻结探测协议下评估了DNABERT-2、Nucleotide Transformer、HyenaDNA、GENERATOR-v2和Omni-DNA，同时将诊断读出分析与验证选择检查相分离。我们的结果揭示了一种一致的任务依赖模式：冻结探测在启动子任务上恢复了95%-100%的微调性能，但剪接位点的平均恢复率降至60%-88%。冻结嵌入在编码区和物种分类等广泛的基因组基准任务上也具有竞争力，但在某些调控和OCR任务上存在更大差距。分层探测、计算机诱变、变异效应预测和嵌入几何表明，局部生物信号部分存在于冻结表征中，但并不总能通过最终的池化嵌入被获取。
英文摘要
Genomic foundation models are increasingly reused as frozen feature extractors for downstream sequence prediction, offering a compute-efficient alternative to full fine-tuning. However, it remains unclear when biological information encoded by these models is accessible without task-specific adaptation. We present a representation-accessibility analysis of frozen genomic language models across regulatory, epigenetic, promoter, splice-site, and variant-effect prediction tasks. We evaluate DNABERT-2, Nucleotide Transformer, HyenaDNA, GENERATOR-v2, and Omni-DNA under unified frozen-probing protocols, while separating diagnostic readout analyses from validation-selected checks. Our results reveal a consistent task-dependent pattern: frozen probes recover 95-100 % of fine-tuned performance on promoter tasks, but average splice-site recovery drops to 60-88 %. Frozen embeddings are also competitive on broad Genomic Benchmark tasks such as coding-region and species-discrimination classification, but show larger gaps on some regulatory and OCR tasks. Layer-wise probing, in-silico mutagenesis, variant-effect prediction, and embedding geometry show that local biological signal is partially present in frozen representations, but is not always accessible through final pooled embeddings.
