arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.06337cs.AI
收藏

TokenMizer: 面向长程LLM上下文管理的图结构会话记忆

TokenMizer: Graph-Structured Session Memory for Long-Horizon LLM Context Management

Independent Researcher(独立研究者)

Shweta Mishra

AI总结:

提出TokenMizer,一种将LLM会话历史建模为类型化知识图的开源代理系统,通过混合提取、三级检查点和8层压缩流水线,在显著减少token开销的同时保留结构化决策信息。

摘要:

大型语言模型(LLM)在长程任务部署中面临一个基本约束:上下文窗口是有限的,而生产性工作会话却不是。当历史超过最大有效上下文窗口(MECW)时,关键的结构化信息——架构决策、任务转换、文件历史——会被静默丢弃。现有缓解方法将历史视为纯文本,破坏了使会话可恢复的关系结构。我们提出TokenMizer,一个将LLM会话历史建模为类型化知识图的开源代理系统。该模式定义了14种节点类型和7种边类型。混合提取流水线逐步填充图,而三级检查点系统将其序列化为紧凑的恢复块。8层压缩流水线减少上下文开销,语义缓存降低重复查询延迟。在涵盖5个领域的21个会话的受控基准上评估,TokenMizer展示了显著的token经济性。它生成的恢复块平均78个token(范围:42-124)——比评估基线(159-170个token)小2倍——同时实现了更高的决策召回率(+9-17个百分点)。关键的是,基线仅保留提到某项技术的事实;TokenMizer保留了其原理。在所有会话中,TokenMizer实现了平均任务召回率51.0%、决策召回率46.6%和文件召回率58.7%。方差反映了领域异质性:显式命令式表述(软件工程)得分高于隐式推理(研究)。消融研究表明模糊标签匹配是主要的改进因素(任务召回率+33个百分点)。启发式压缩实现了47.3%的token减少且零外部依赖。TokenMizer以一半的token成本提供了可查询的替代方案,优于文本保留基线。

英文摘要:

Long-horizon LLM sessions outlive their context windows, and the standard mitigations - truncation, summarization, retrieval - share a structural flaw: they treat history as flat text, discarding precisely the content that makes a session resumable: decisions and their rationales, task status, and file modification history. We present TokenMizer, an open-source transparent proxy that maintains session history as a typed knowledge graph and, at context boundaries, replaces the raw transcript with a token-budgeted serialization of session state. The schema comprises 14 node types and 7 edge types under an 8-state lifecycle in which decisions can be superseded or explicitly invalidated; bitemporal validity intervals support time-travel queries; and first-class decision-transition records preserve why each decision replaced its predecessor (trigger, reason, evidence). Version 0.3.1 embeds this memory core in a production-shaped serving layer - SSE streaming, security middleware, nine provider adapters, a monitoring dashboard, graph exports (D3 JSON, self-contained interactive HTML, Obsidian Canvas) - and exposes checkpoint/resume to agents as Model Context Protocol tools. The evaluation is deliberately minimal and fully provenanced: three synthetic sessions, heuristic-only extraction, one plain-summary baseline, every value traceable to a single versioned results file. Graph extraction ties the baseline on task recall (75.6%) and exceeds it on decision recall (85.0% vs. 70.0%) and file recall (100% vs. 91.7%), with 201-302-token resume blocks extracted in 8.1-529.9 ms per session. At n=3 these results are directional; ceiling effects and baseline weaknesses are analyzed explicitly. Code, benchmark runner, and the exact results file are released under the MIT licence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑