UT-ACA:基于不确定性触发的自适应上下文分配用于长上下文推理
UT-ACA: Uncertainty-Triggered Adaptive Context Allocation for Long-Context Inference
AI总结:
UT-ACA通过动态调整上下文窗口,减少长上下文推理中的上下文使用量,同时保持生成质量。
摘要:
长上下文推理仍面临大规模语言模型中的注意力稀释和分布外退化问题。上下文选择通过关注关键值缓存条目子集来缓解这一限制,但大多数方法在解码过程中固定分配上下文预算,尽管token级上下文需求高度非均匀。为了解决这个问题,我们提出了不确定性触发的自适应上下文分配(UT-ACA),一种推理时框架,根据token级不确定性动态调整上下文窗口。UT-ACA学习一个不确定性检测器,结合语义嵌入和基于logit的置信度,同时考虑解码步骤中的不确定性累积。当指示不足证据时,UT-ACA会选择回滚、扩展上下文窗口并重新生成具有额外支持的token。实验表明,UT-ACA在长上下文设置中显著减少了平均上下文使用量,同时保持生成质量。
英文摘要:
Long-context inference remains challenging for large language models due to attention dilution and out-of-distribution degradation. Context selection mitigates this limitation by attending to a subset of key-value cache entries, yet most methods allocate a fixed context budget throughout decoding despite highly non-uniform token-level contextual demands. To address this issue, we propose Uncertainty-Triggered Adaptive Context Allocation (UT-ACA), an inference-time framework that dynamically adjusts the context window based on token-wise uncertainty. UT-ACA learns an uncertainty detector that combines semantic embeddings with logit-based confidence while accounting for uncertainty accumulation across decoding steps. When insufficient evidence is indicated, UT-ACA selectively rolls back, expands the context window, and regenerates the token with additional support. Experiments show that UT-ACA substantially reduces average context usage while preserving generation quality in long-context settings. Code and dataset are available at https://github.com/Tommy307/UT-ACA.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。