arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.08030cs.AIcs.CL
收藏

Free(): 学会遗忘的仅malloc推理模型

Free(): Learning to Forget in Malloc-Only Reasoning Models

Tencent AI Lab(腾讯AI实验室);Nanyang Technological University(南洋理工大学)

Yilun Zheng, Dongyang Ma, Tian Liang, Jiahao Xu, Xinting Huang, Lihui Chen, Haitao Mi, Yan Wang

AI总结:

Free()LM通过引入内在自我遗忘机制,解决推理模型中过多思考标记导致性能下降的问题,实现模型在不同规模上的性能提升,并在长周期任务中恢复性能。

摘要:

推理模型通过增加推理时的计算量来提升问题解决能力,但它们面临一个关键悖论:过多的思考标记往往会降低性能而不是提高它。我们归因于一个根本性的架构缺陷:标准LLM是

英文摘要:

Reasoning models enhance problem-solving by scaling test-time compute, yet they face a critical paradox: excessive thinking tokens often degrade performance rather than improve it. We attribute this to a fundamental architectural flaw: standard LLMs operate as "malloc-only" engines, continuously accumulating valid and redundant steps alike without a mechanism to prune obsolete information. To break this cycle, we propose Free()LM, a model that introduces an intrinsic self-forgetting capability via the Free-Module, a plug-and-play LoRA adapter. By iteratively switching between reasoning and cleaning modes, Free()LM dynamically identifies and prunes useless context chunks, maintaining a compact and noise-free state. Extensive experiments show that Free()LM provides consistent improvements across all model scales (8B to 685B). It achieves a 3.3% average improvement over top-tier reasoning baselines, even establishing a new SOTA on IMOanswerBench using DeepSeek V3.2-Speciale. Most notably, in long-horizon tasks where the standard Qwen3-235B-A22B model suffers a total collapse (0% accuracy), Free()LM restores performance to 50%. Our findings suggest that sustainable intelligence requires the freedom to forget as much as the power to think.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑