AI 中文总结
Fathom通过逐查询反向注水分配位预算读取KV缓存位平面，在百万token场景下比现有稀疏解码方法更快且更省字节，同时保持与精确top-k相当的准确率。
AI 中文摘要
当智能体会话运行至百万token且同时驻留多个会话时，KV缓存及其排序索引位于主机内存中，而针对top-k步骤对所有n个键进行排序的扫描成为限制解码的流量瓶颈。我们提出Fathom，一种键扫描方法，其中每个查询决定读取每个键通道的位数。4位K缓存按通道主序存储为位平面，因此t个平面的前缀恰好是该通道的t位量化器，查询通过对其通道的方差加权重要性进行反向注水来分配其位预算。在Qwen3-8B上处理一百万个token时，解码步骤的GPU时间比使用Double Sparsity、Loki和SparQ r=32的136位扫描快1.67倍，并且在与SparQ的68位读取（r=16）相同的GPU时间内，Fathom在七个模型和上下文设置中的六个上读取的字节数减少18%，且注意力误差更低。在RULER风格的任务上，每个逐token扫描均与精确top-k解码匹配，而在真实编码智能体会话中，Fathom在92位时达到最准确的136位扫描的步骤一致性。存储是量化服务栈已有的4位K副本，当索引驻留在GPU内存中时，该方法并不更快。
英文摘要
When agentic sessions run to a million tokens with many sessions resident at once, the KV cache and the index that ranks it live in host memory, and the scan that ranks all n keys for a top-k step becomes the traffic that bounds decoding. We present Fathom, a key scan in which each query decides how many bits of each key channel to read. The 4-bit K cache is stored channel-major as bit planes, so a prefix of t planes is exactly the channel's t-bit quantizer, and the query spends its bit budget by reverse water-filling over the variance-weighted importance of its channels. At one million tokens on Qwen3-8B a decode step is 1.67x faster in GPU time than with the 136-bit scans of Double Sparsity, Loki and SparQ r=32, and in the same GPU time as SparQ's 68-bit read (r=16) Fathom reads 18% fewer bytes with lower attention error on six of seven model and context settings. On RULER-style tasks every per-token scan matches exact top-k decoding, and on real coding-agent sessions Fathom reaches the step agreement of the most accurate 136-bit scan at 92 bits. The store is the 4-bit K copy a quantized serving stack already holds, and the method is not faster when the index is resident in GPU memory.
Comments19 pages, 11 figures, 21 tables. Code and results: https://github.com/vivekkalyanarangan30/fathom
