RouteRelay：事件触发跨层路由复用用于高效动态稀疏注意力

中文摘要 AI 辅助

动态稀疏注意力通过在每个Transformer层将每个查询块路由到少量键块，降低了长上下文预填充成本。稀疏注意力核避免了大多数token交互，但路由器仍然逐层重建块-块得分矩阵，即使所选路由变化很小。我们提出RouteRelay，一种与路由器无关的方法，仅跨深度复用路由元数据，同时继续使用当前层的查询、键和值计算注意力。锚定层执行完整路由。中间层对先前的前k个路由以及一个包含近似遗漏和随机探测块的紧凑哨兵集进行重新评分。仅当哨兵挑战查询行最弱的选定块时，该查询行才会被重新路由。我们给出了前k个稳定性条件、关于遗漏挑战者的概率界以及行选择性GPU执行设计。在可复现的实证评估中，RouteRelay在低、中、高跨层漂移下分别重新路由25.0%、55.4%和78.2%的行，同时保留至少99.99%的路由召回率。在各种路由规模下，随着键块数量从128增长到1024，RouteRelay在评估38.4%-51.6%的完整路由得分对的同时保留了100.0%的召回率。其未融合的CPU执行仍慢于密集矩阵乘法，暴露了行压缩和账本更新作为主要内核工程目标。

英文摘要

Dynamic sparse attention reduces long-context prefill cost by routing each query chunk to a small set of key chunks at every Transformer layer. The sparse attention kernel avoids most token interactions, but the router still rebuilds a chunk--chunk score matrix layer after layer, even when the selected routes change little. We introduce RouteRelay, a router-agnostic method that reuses only route metadata across depth while continuing to compute attention with the current layer's queries, keys, and values. Anchor layers perform full routing. Intermediate layers rescore the previous top-$k$ route and a compact sentinel set of near-miss and randomly probed chunks. A query row is rerouted only when a sentinel challenges its weakest selected chunk. We give a top-$k$ stability condition, a probabilistic bound on missed challengers, and a row-selective GPU execution design. In a reproducible empirical evaluation, RouteRelay retains at least 99.99% route recall while rerouting 25.0%, 55.4%, and 78.2% of rows under low, moderate, and high cross-layer drift, respectively. Across routing scales, RouteRelay retains 100.0% recall while evaluating 38.4--51.6% of full-routing score pairs as the key-chunk count grows from 128 to 1024. Its unfused CPU execution remains slower than dense matrix multiplication, exposing row compaction and ledger updates as the main kernel-engineering targets.

发表机构

• Xiamen University of Technology（厦门理工大学）
• Fuzhou University（福州大学）

机构由 AI 辅助整理，请以论文原文为准。