arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.01030cs.CLcs.AIcs.LG
收藏

Birdie:通过奖励驱动目标与课程推进状态空间模型发展

Birdie: Advancing State Space Models with Reward-Driven Objectives and Curricula

George Mason University(乔治梅森大学);Stanford University(斯坦福大学);Liquid AI;Archimedes AI

Sam Blouir, Jimmy T. H. Smith, Antonios Anastasopoulos, Amarda Shehu

AI总结:

针对状态空间模型长程上下文内检索能力弱的问题,提出不改动架构的训练流程Birdie,结合双向处理、动态预训练目标与强化学习,显著提升检索密集型任务性能,缩小与Transformers的差距并保持计算效率。

摘要:

高效状态空间模型(SSMs),例如线性循环神经网络和线性注意力变体,相比Transformers具备计算优势,但在需要长程上下文内检索的任务(如文本复制、联想回忆和长上下文问答)中表现不佳。此前应对这些挑战的研究多聚焦于架构修改,往往会重新引入计算低效性。本文提出一种全新的训练流程Birdie,无需改动SSMs的架构即可显著增强其上下文内检索能力。我们的方法将双向输入处理与专用预训练目标的动态混合相结合,并通过强化学习进行优化。我们还提出了一种新的双向SSM架构,可实现从双向上下文处理到因果生成的无缝切换。实验评估表明,Birdie在多号码电话簿查询、长段落问答和填充等检索密集型任务上的性能显著提升,缩小了与Transformers的性能差距,同时保留了计算效率。研究结果凸显了训练流程在发挥SSMs固定状态容量方面的重要性,为提升其能力提供了新方向。所有代码和预训练模型可在https://www.github.com/samblouir/birdie获取,支持JAX和PyTorch。

英文摘要:

Efficient state space models (SSMs), such as linear recurrent neural networks and linear attention variants, offer computational advantages over Transformers but struggle with tasks requiring long-range in-context retrieval-like text copying, associative recall, and question answering over long contexts. Previous efforts to address these challenges have focused on architectural modifications, often reintroducing computational inefficiencies. In this paper, we propose a novel training procedure, Birdie, that significantly enhances the in-context retrieval capabilities of SSMs without altering their architecture. Our approach combines bidirectional input processing with dynamic mixtures of specialized pre-training objectives, optimized via reinforcement learning. We introduce a new bidirectional SSM architecture that seamlessly transitions from bidirectional context processing to causal generation. Experimental evaluations demonstrate that Birdie markedly improves performance on retrieval-intensive tasks such as multi-number phone book lookup, long paragraph question-answering, and infilling. This narrows the performance gap with Transformers, while retaining computational efficiency. Our findings highlight the importance of training procedures in leveraging the fixed-state capacity of SSMs, offering a new direction to advance their capabilities. All code and pre-trained models are available at https://www.github.com/samblouir/birdie, with support for JAX and PyTorch.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑