arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.15843cs.CLcs.AI
收藏

困惑性悖论:为什么代码在LLM提示中比数学压缩得更好

The Perplexity Paradox: Why Code Compresses Better Than Math in LLM Prompts

Bona Opera Studios(博纳歌剧工作室)

Warren Johnson

AI总结:

本文揭示代码在LLM提示中比数学压缩更有效的原因,并提出TAAC算法实现更高效的压缩。

摘要:

在"压缩或路由?"(Johnson, 2026)中,我们发现代码生成能够容忍剧烈的提示压缩(r >= 0.6),而链式推理则逐渐退化。该研究仅限于HumanEval(164个问题),未验证"困惑性悖论"机制,也未提供自适应算法。本文解决这三个缺口。首先,我们在六个代码基准(HumanEval, MBPP, HumanEval+, MultiPL-E)和四个推理基准(GSM8K, MATH, ARC-Challenge, MMLU-STEM)上验证,确认压缩阈值在语言和难度上具有普遍性。其次,我们进行了首次每token困惑度分析(n=723 tokens),揭示了"困惑性悖论":代码语法token被保留(高困惑度),而数学问题中的数值在任务关键性方面被修剪(低困惑度)。签名注入在通过率上恢复+34个百分点(5.3%到39.3%;Cohen's h=0.890)。第三,我们提出了TAAC(任务感知自适应压缩),实现了22%的成本降低和96%的质量保持,优于固定比率压缩7%。MBPP验证(n=1,800次试验)确认了系统性变化:在r=0.3时为3.6%,在r=1.0时为54.6%。

英文摘要:

In "Compress or Route?" (Johnson, 2026), we found that code generation tolerates aggressive prompt compression (r >= 0.6) while chain-of-thought reasoning degrades gradually. That study was limited to HumanEval (164 problems), left the "perplexity paradox" mechanism unvalidated, and provided no adaptive algorithm. This paper addresses all three gaps. First, we validate across six code benchmarks (HumanEval, MBPP, HumanEval+, MultiPL-E) and four reasoning benchmarks (GSM8K, MATH, ARC-Challenge, MMLU-STEM), confirming the compression threshold generalizes across languages and difficulties. Second, we conduct the first per-token perplexity analysis (n=723 tokens), revealing a "perplexity paradox": code syntax tokens are preserved (high perplexity) while numerical values in math problems are pruned despite being task-critical (low perplexity). Signature injection recovers +34 percentage points in pass rate (5.3% to 39.3%; Cohen's h=0.890). Third, we propose TAAC (Task-Aware Adaptive Compression), achieving 22% cost reduction with 96% quality preservation, outperforming fixed-ratio compression by 7%. MBPP validation (n=1,800 trials) confirms systematic variation: 3.6% at r=0.3 to 54.6% at r=1.0.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑