温度前后：创造性LLM生成的分布视角
Before and After Temperature: A Distributional View of Creative LLM Generation
AI总结：
通过分析采样温度重塑token分布的特征，提出一种无需参考的创造力评估方法，在Llama-3.1-8B-Instruct上达到Spearman ρ=0.918（LLM评判）和ρ=0.870（人类多数评判），显著优于现有基线。
摘要：
大型语言模型（LLM）创造力的无参考评估依赖于困惑度、熵和top-1边际。我们表明，一个更强的信号存在于流程中更早的一步：在抽取下一个token之前，采样温度如何重塑模型的token分布。在Llama-3.1-8B-Instruct生成的500个开放式创造性提示（温度T∈{0.3,0.8,1.5}）上，一个基于这种重塑的单个逐token特征预测提示内创造力排名的Spearman ρ=0.918（与平均gpt-4o/gemini-2.5-pro评判对比，n=500）和ρ=0.870（与三人人类多数排名对比，n=150）。四种标准无参考基线（自困惑度、平均预测熵、top-1边际、gzip压缩比）在两个真实标签上均达到|ρ|≈0.76：与平均LLM评判差距为+0.165，与人类多数评判差距为+0.110，两者均远大于基线之间的差异。两个真实标签面板之间的相关系数为ρ=0.83，高于人类间上限ρ=0.77，因此比较不受评判噪声的瓶颈限制。机制上，优势来自于不连贯区域的尖锐分布特征：在T=1.5时，累积质量宽度n_{95}(q)从约1个token膨胀到约131个token，且温度后的质量从温度前top-90%可能集合中泄漏约13个百分点。逐token聚合无法区分T=0.8和T=0.3；区分这两个连贯区域的任务留给序列级特征。
英文摘要：
Reference-free evaluation of large language model (LLM) creativity relies on perplexity, entropy, and top-1 margin. We show that a much stronger signal lives one step earlier in the pipeline: in how sampling temperature \emph{reshapes} the model's token distribution before the next token is drawn. On Llama-3.1-8B-Instruct generations of 500 open-ended creative prompts at $T \in \{0.3, 0.8, 1.5\}$, a single per-token feature derived from this reshaping predicts the within-prompt creativity rank at Spearman $ρ{=}0.918$ against an averaged gpt-4o\,/\,gemini-2.5-pro judge ($n{=}500$) and $ρ{=}0.870$ against a three-rater human-majority ranking ($n{=}150$). Each of four standard reference-free baselines (self-perplexity, mean predictive entropy, top-1 margin, gzip compression ratio) tops out at $|ρ|\!\approx\!0.76$ on both ground truths: a gap of $+0.165$ on averaged-LLM and $+0.110$ on human-majority, both far larger than the spread among the baselines themselves. The two ground-truth panels agree with each other at $ρ{=}0.83$, above the inter-human ceiling of $ρ{=}0.77$, so the comparison is not bottlenecked by judge noise. Mechanistically, the win comes from a sharp distributional signature of the incoherence regime: at $T{=}1.5$ the cumulative-mass width $n_{95}(q)$ inflates from $\sim\!1$ to ${\sim}\!131$ tokens and post-temperature mass leaks off the pre-temperature top-$90\%$ plausible set by about $13$ percentage points. The per-token aggregates do not separate $T{=}0.8$ from $T{=}0.3$; discriminating the two coherent regimes is left to sequence-level features.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
