CSR：具有大规模缓存状态表示的无限时间 horizon 实时策略
CSR: Infinite-Horizon Real-Time Policies with Massive Cached State Representations
AI总结：
本文提出CSR框架，通过优化KV缓存重用和异步状态协调算法，实现大规模LLM在机器人中的实时应用，显著降低延迟并提升性能。
摘要：
在将大规模大语言模型（LLMs）作为连续认知引擎用于机器人时，处理大量状态历史所需的时至第一标记（TTFT）延迟成为瓶颈。现有解决方案如RAG或滑动窗口会牺牲全局上下文或导致高昂的重新计算成本。我们正式化了最小化延迟的最佳任务结构，并理论证明前缀稳定性、增量扩展性和异步状态协调是实时性能的必要条件。基于这些证明，我们引入缓存状态表示（CSR）框架作为这些属性的实用实现，确保最优的KV缓存重用。为了在无限时间范围内维持这些属性，我们进一步提出异步状态协调（ASR）算法，将状态内存驱逐任务卸载到并行计算资源以消除延迟尖峰。在连接到本地GPU服务器的物理机器人上，CSR在120K token上下文中，与标准基线相比，将延迟降低了26倍（14.67秒至0.56秒），在具有235B参数模型的情况下。在具身AI基准测试中，我们实现了SOTA召回（0.836 vs. 0.459）同时保持RAG级别的延迟。ASR被验证在连续现实世界操作中可以维持有界、无尖峰的TTFT超过10次驱逐周期。共同，CSR和ASR使大规模LLM能够作为持续运行、高频（> 2 Hz）的具身策略发挥作用。
英文摘要：
Deploying massive large language models (LLMs) as continuous cognitive engines for robotics is bottlenecked by the time-to-first-token (TTFT) latency required to process extensive state histories. Existing solutions like RAG or sliding windows compromise global context or incur prohibitive re-computation costs. We formalize the optimal task structure for minimizing latency and theoretically prove that prefix stability, incremental extensibility, and asynchronous state reconciliation are necessary conditions for real-time performance. Building on these proofs, we introduce the Cached State Representation (CSR) framework as the practical instantiation of these properties, ensuring optimal KV-cache reuse. To sustain these properties over infinite horizons, we further propose an Asynchronous State Reconciliation (ASR) algorithm that offloads state memory eviction to a parallel computational resource to eliminate latency spikes. On a physical robot wirelessly connected to an on-premise GPU server, CSR achieves a 26-fold latency reduction (14.67s to 0.56s) for 120K token contexts with a 235B parameter model compared to a standard baseline. On an embodied AI benchmark, we achieve SOTA recall (0.836 vs. 0.459) while maintaining RAG-level latency. ASR is validated to sustain bounded, spike-free TTFT over 10 eviction cycles in continuous real-world operation. Together, CSR and ASR enable massive LLMs to function as continuously operating, high-frequency (> 2 Hz) embodied policies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
