Q-Delta：超越键值关联状态演化
Q-Delta: Beyond Key-Value Associative State Evolution
AI总结：
提出Q-Delta，一种查询感知的delta规则，将混合键-查询预测误差融入状态演化，实现联合校正动态，在语言建模和长上下文检索任务上优于强基线。
摘要：
线性注意力将序列建模重新表述为循环状态演化，实现高效的线性时间推理。在键值关联范式下，现有方法将查询的作用限制在读出操作，使其与状态演化解耦。我们表明，查询条件状态读出在累积记忆上诱导出结构化的值预测，补充了基于键的检索。基于这一洞察，我们提出Q-Delta，一种查询感知的delta规则，将混合键-查询预测误差融入状态演化，在保持delta规则效率的同时实现联合校正动态。我们为所得动态建立了稳定性保证，并推导出硬件高效的块状并行公式，以及自定义Triton实现。实验结果表明，在语言建模和长上下文检索任务上，优化稳定、吞吐量具有竞争力，且一致优于强基线。
英文摘要：
Linear attention reformulates sequence modeling as recurrent state evolution, enabling efficient linear-time inference. Under the key-value associative paradigm, existing approaches restrict the role of the query to the readout operation, decoupling it from state evolution. We show that query-conditioned state readout induces a structured value prediction over accumulated memory that complements key-based retrieval. Based on this insight, we propose Q-Delta, a query-aware delta rule that integrates mixed key-query prediction errors into state evolution, enabling jointly corrective dynamics while preserving delta-rule efficiency. We establish stability guarantees for the resulting dynamics and derive a hardware-efficient chunkwise-parallel formulation with a custom Triton implementation. Empirical results demonstrate stable optimization, competitive throughput, and consistent improvements over strong baselines on language modeling and long-context retrieval tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
