arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06221cs.CL
收藏

UniPrefill: 通过块级动态稀疏化实现通用长上下文预填加速

UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification

MAIS&NLPR, CASIA(MAIS与NLPR,中国科学院自动化研究所);UCAS(中国科学技术大学);WeChat, Tencent(微信,腾讯)

Qihang Fan, Huaibo Huang, Zhiying Wu, Bingning Wang, Ran He

AI总结:

本文提出UniPrefill框架,通过块级动态稀疏化提升长上下文预填效率,兼容各种模型架构,实现TTFT加速,支持连续批处理和vLLM调度策略。

摘要:

随着大型语言模型(LLMs)持续快速进步,它们在增强能力的同时也要求更长的上下文长度。为提高长上下文处理的推理效率,近年来提出了一些新型低复杂度混合架构,有效缓解了长上下文推理的计算负担。然而,现有长上下文预填加速研究主要集中在稀疏注意力机制上,这些方法仅在全注意力模型上达到最大加速效果。当应用于新兴架构——如线性/全注意力混合或滑动窗口/全注意力混合架构时,这些预填加速方法会显著退化性能。此外,此类方法通常与连续批处理不兼容,难以集成到现代推理引擎如vLLM中。为此,我们提出UniPrefill,一种适用于几乎所有模型架构的预填加速框架,直接在token级别加速模型计算。我们进一步将UniPrefill实现为连续批处理操作符,并扩展vLLM的调度策略以原生支持预填-解码共处理和UniPrefill的张量并行,使其无缝集成到vLLM中。UniPrefill在Time-To-First-Token(TTFT)上实现最高2.1倍的加速,随着并发请求数量的增加,加速效果愈发显著。

英文摘要:

As large language models (LLMs) continue to advance rapidly, they are becoming increasingly capable while simultaneously demanding ever-longer context lengths. To improve the inference efficiency of long-context processing, several novel low-complexity hybrid architectures have recently been proposed, effectively alleviating the computational burden of long-context inference. However, existing research on long-context prefill acceleration remains predominantly focused on sparse attention mechanisms, which achieve their maximum speedup only on full-attention models. When transferred to emerging architectures--such as linear/full attention hybrids or sliding window/full attention hybrids--these prefill acceleration approaches suffer significant performance degradation. Furthermore, such methods are generally incompatible with continuous batching, making them difficult to integrate into modern inference engines such as vLLM. To this end, we propose UniPrefill, a prefill acceleration framework applicable to virtually any model architecture, which directly accelerates the model's computation at the token level. We further implement UniPrefill as a continuous batching operator and extend vLLM's scheduling strategy to natively support prefill-decode co-processing and tensor parallel for UniPrefill, enabling its seamless integration into vLLM. UniPrefill achieves up to 2.1x speedup in Time-To-First-Token (TTFT), with the acceleration becoming increasingly pronounced as the number of concurrent requests grows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑