推理的热力学特征：用于大型语言模型幻觉检测的自由能和谱形因子诊断
Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Models
AI总结：
提出自由能签名（Fes）作为谱描述符，将注意力拉普拉斯视为哈密顿量并提取热力学势和随机矩阵理论谱形因子，用于检测LLM幻觉，无需训练即可实现高AUROC。
摘要：
大型语言模型（LLM）中的幻觉检测对部署至关重要，近期研究表明注意力导出的图拉普拉斯谱携带关于推理质量的强信号。然而，先前的谱诊断仅通过少数特征值或手工选取的标量来总结拉普拉斯谱，忽略了其大部分结构。我们提出自由能签名（Fes），一种谱描述符，将每层的注意力拉普拉斯视为哈密顿量，并提取其热力学势（配分函数、自由能、谱熵、热容）以及随机矩阵理论（RMT）谱形因子。我们证明了三个结果：（i）Fes在注意力扰动下的Lipschitz稳定性；（ii）一个表达性结果，表明Fes丰富了有限谱摘要，并在明确的规则性和网格分辨率假设下逼近矩导出的谱泛函；（iii）基于Fes构建的无训练检测器AUROC的有限样本PAC界。实验上，在六个开源LLM和六个基准测试中，基于Fes描述符的轻量级探测在注意力谱基线中实现了最强的平均AUROC，相比LapEig平均提高+6.5 AUROC点，相比GoR-4平均提高+2.4点，且无需更新底层LLM。在完全无监督设置下，RMT偏差得分达到平均AUROC 0.71，提供了一个无标签但较弱的检测器。互补的RMT分析表明，正确生成表现出更接近Wigner-Dyson的谱统计，而幻觉表现出更接近Poisson的统计。匿名代码和配置在补充材料中提供。
英文摘要：
Hallucination detection in large language models (LLMs) is deployment-critical, and recent work shows that the spectrum of attention-derived graph Laplacians carries strong signal about reasoning quality. Prior spectral diagnostics, however, summarize the Laplacian spectrum by a handful of eigenvalues or hand-picked scalars, leaving most of its structure unused. We propose Free-Energy Signatures (Fes), a spectral descriptor that treats each layer's attention Laplacian as a Hamiltonian and extracts its thermodynamic potentials partition function, free energy, spectral entropy, heat capacity together with the random-matrix-theory (RMT) spectral form factor. We prove three results: (i)~Lipschitz stability of Fes under attention perturbation; (ii)~an expressiveness result showing that Fes enriches finite spectral summaries and approximates moment-derived spectral functionals under explicit regularity and grid-resolution assumptions; and (iii)~a finite-sample PAC bound on the AUROC of a training-free detector built from Fes. Empirically, across six open-weight LLMs and six benchmarks, a lightweight probe on Fes descriptors achieves the strongest aggregate AUROC among attention-spectral baselines, improving over LapEig by $+6.5$ AUROC points and over GoR-4 by $+2.4$ points on average, while requiring no update to the underlying LLM. In the fully unsupervised setting, an RMT-deviation score achieves mean AUROC $0.71$, providing a label-free but weaker detector. A complementary RMT analysis shows that correct generations exhibit more Wigner-Dyson like spectral statistics, whereas hallucinations exhibit more Poisson-like statistics. The anonymized code and config are provided in the supplementary material.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
