arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.08044cs.CLcs.AIcs.LG
收藏

快速字节潜在变换器

Fast Byte Latent Transformer

FAIR at Meta(Meta的FAIR);Stanford University(斯坦福大学);University of Washington(华盛顿大学)

Julie Kallini, Artidoro Pagnoni, Tomasz Limisiewicz, Gargi Ghosh, Luke Zettlemoyer, Christopher Potts, Xiaochuang Han, Srinivasan Iyer

AI总结:

本文提出BLT Diffusion和BLT Self-speculation等方法,通过并行生成和验证步骤提升字节级语言模型的生成速度和质量,降低内存带宽消耗。

摘要:

近期的字节级语言模型(LMs)在无需子词词汇的情况下可与标记级模型匹配性能,但其实用性受限于缓慢的逐字生成。我们通过新的训练和生成技术解决了这一瓶颈,首先引入BLT Diffusion(BLT-D),一种新的模型和最快的BLT变体,通过辅助的块级扩散目标与标准的下一字预测损失共同训练。这使推理过程能够在每个解码步骤中并行生成多个字节,显著减少生成序列所需的前向传递次数。其次,我们提出两种受推测解码启发的扩展,以牺牲部分速度换取更高的生成质量:BLT Self-speculation(BLT-S),其中BLT的局部解码器继续生成超过其正常补丁边界以草稿字节,然后通过单次完整模型前向传递验证;以及BLT Diffusion+Verification(BLT-DV),在扩散生成后增加一个自回归验证步骤。所有方法在生成任务中可能实现约50%的内存带宽成本降低。每种方法都有其独特优势,共同克服了字节级LMs的实际应用关键障碍。

英文摘要:

Recent byte-level language models (LMs) match the performance of token-level models without relying on subword vocabularies, yet their utility is limited by slow, byte-by-byte autoregressive generation. We address this bottleneck in the Byte Latent Transformer (BLT) through new training and generation techniques. First, we introduce BLT Diffusion (BLT-D), a new model and our fastest BLT variant, trained with an auxiliary block-wise diffusion objective alongside the standard next-byte prediction loss. This enables an inference procedure that generates multiple bytes in parallel per decoding step, substantially reducing the number of forward passes required to generate a sequence. Second, we propose two extensions inspired by speculative decoding that trade some of this speed for higher generation quality: BLT Self-speculation (BLT-S), in which BLT's local decoder continues generating past its normal patch boundaries to draft bytes, which are then verified with a single full-model forward pass; and BLT Diffusion+Verification (BLT-DV), which augments BLT-D with an autoregressive verification step after diffusion-based generation. All methods may achieve an estimated memory-bandwidth cost over 50% lower than BLT on generation tasks. Each approach offers its own unique advantages, together removing key barriers to the practical use of byte-level LMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑