连续扩散变换器用于设计合成调控元件
Continuous Diffusion Transformers for Designing Synthetic Regulatory Elements
AI总结:
本文提出连续扩散变换器生成细胞类型特异性调控DNA序列,通过改进模型结构和微调方法,显著提升调控活性预测性能。
摘要:
我们提出了一种参数高效的扩散变换器(DiT),用于生成200碱基对的细胞类型特异性调控DNA序列。通过将DNA-扩散的U-Net主干替换为配备2D卷积神经网络输入编码器的变换器去噪器,我们的模型在13个epoch(60倍更少)内达到了U-Net的最佳验证损失,并且收敛更低39%,同时通过BLAT将记忆从生成序列中与训练数据对齐的比例从5.3%降低到1.7%。消融实验显示CNN编码器是必不可少的:没有它,无论选择哪种位置嵌入方式,验证损失都会增加70%。我们进一步应用DDPO微调,使用Enformer作为奖励模型,实现了预测调控活性的38倍提升。在独立预测任务上对DRAKES进行交叉验证,确认了改进反映了真实的调控信号,而非奖励模型过拟合。
英文摘要:
We present a parameter-efficient Diffusion Transformer (DiT) for generating 200bp cell-type-specific regulatory DNA sequences. By replacing the U-Net backbone of DNA-Diffusion with a transformer denoiser equipped with a 2D CNN input encoder, our model matches the U-Net's best validation loss in 13 epochs (60$\times$ fewer) and converges 39% lower, while reducing memorization from 5.3% to 1.7% of generated sequences aligning to training data via BLAT. Ablations show the CNN encoder is essential: without it, validation loss increases 70% regardless of positional embedding choice. We further apply DDPO finetuning using Enformer as a reward model, achieving a 38$\times$ improvement in predicted regulatory activity. Cross-validation against DRAKES on an independent prediction task confirms that improvements reflect genuine regulatory signal rather than reward model overfitting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。