LDARNet: 用于基因组建模的DNA自适应表示网络与可学习分词
LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling
AI总结：
提出LDARNet，一种结合动态分块和双向路由的120M参数层次基因组基础模型，在27个任务中优于更大模型，并发现学习到的边界与生物学基序对齐。
摘要：
基因组基础模型越来越多地采用大型语言模型架构，但几乎普遍依赖于固定的分词方案，如$k$-mers、BPE或单核苷酸，这些方案强加了可能掩盖生物学相关结构的任意序列边界。我们提出了LDARNet，一个120M参数的层次基因组基础模型，它将H-Net风格的动态分块从自回归生成适应到掩码语言建模，结合了BiMamba-2状态空间层与局部注意力、双向路由以及基于比值的正则化器，以在无监督的情况下诱导自适应标记边界。在来自Nucleotide Transformer和Genomic Benchmarks套件的27个任务上进行微调后，LDARNet在紧凑模型（<300M参数）中取得了11/18的胜率，并在5个组蛋白修饰任务上取得了最先进的结果，优于高达20倍大的模型。一个FLOPs匹配的对照实验将学习到的路由确定为这些增益的来源：在相同计算量下，学习到的边界在组蛋白任务上比固定网格边界高出多达14个百分点。进一步的核苷酸分辨率分析表明，学习到的边界在无监督的情况下与典型的启动子基序和剪接连接点对齐，为基因组基础模型中的自适应分词提供了生物学解释。
英文摘要：
Genomic foundation models increasingly adopt large language model architectures, yet almost universally rely on fixed tokenization schemes such as $k$-mers, BPE, or single nucleotides, which impose arbitrary sequence boundaries that may obscure biologically relevant structure. We present LDARNet, a 110M-parameter hierarchical genomic foundation model that adapts H-Net-style dynamic chunking from autoregressive generation to masked language modeling, combining BiMamba-2 state-space layers with local attention, bidirectional routing, and a ratio-based regularizer to induce adaptive token boundaries without supervision. Fine-tuned on 27 tasks from the Nucleotide Transformer and Genomic Benchmarks suites, LDARNet achieves 15/18 wins among compact models ($<$300M parameters) and the best overall result on 9 of the 10 histone modification tasks, outperforming models up to 20$\times$ larger. A FLOPs-matched controlled experiment isolates learned routing as the source of these gains: learned boundaries beat fixed-grid boundaries by up to 14 percentage points on histone tasks at identical compute. Nucleotide-resolution analysis further shows that the learned boundaries align with canonical promoter motifs and splice junctions without supervision, providing a biological interpretation for adaptive tokenization in genomic foundation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
