RW-TTT:面向请求自有测试时训练状态的批量服务
RW-TTT: Batched Serving for Request-Owned Test-Time Training State
AI总结:
提出RW-TTT框架,通过标记解码步骤的所有者、版本和读写效果,实现请求自有测试时训练状态下的高效批量LLM服务,在单GPU上达到274.61 tok/s聚合吞吐,较顺序服务提升9.31倍。
摘要:
测试时训练(TTT)在生成过程中通过读取和更新请求自有状态(如快速权重、低秩增量或流学习器状态)来调整LLM。这打破了假设共享静态权重的批量LLM服务:串行执行正确但缓慢,而朴素批处理可能破坏请求状态。我们将此问题形式化为读写TTT服务,并提出RW-TTT,它将每个解码步骤标记为其所有者、版本和READ/WRITE效果,仅批处理兼容阶段,并仅将更新提交给所有者。在单个GPU上,使用八个快速权重InPlace-TTT流,RW-TTT达到274.61聚合tok/s,比顺序服务快9.31倍,在相同内存预算下比每流副本快3.44倍。它在长上下文基准RULER上保持行为,并通过所有者和版本检查。
英文摘要:
Test-time training (TTT) adapts an LLM during generation by reading and updating request-owned state, such as fast weights, low-rank deltas, or streaming learner state. This breaks batched LLM serving, which assumes shared static weights: serial execution is correct but slow, while naive batching can corrupt request state. We formulate this problem as read-write TTT serving and present RW-TTT , which tags each decode step with its owner, version, and READ/WRITE effect, batches only compatible phases, and commits updates only to the owner. On one GPU with eight fast-weight InPlace-TTT streams, RW-TTT reaches 274.61 aggregate tok/s, 9.31x over sequential serving and 3.44x over per-stream replicas under the same memory budget. It preserves behavior on RULER, a long-context benchmark, and passes owner/version checks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。