语义忠实与熵产度量：驯服LLM恶魔并管理幻觉
Semantic Faithfulness and Entropy Production Measures to Tame Your LLM Demons and Manage Hallucinations
AI总结：
本文提出语义忠实与熵产度量，通过信息论和热力学方法评估LLM的忠实度并控制幻觉，应用于公司SEC文件摘要生成。
摘要：
评估大型语言模型（LLMs）对给定任务的忠实度是一个复杂的挑战。我们提出两种新的无监督度量方法，用于忠实度评估，这些方法借鉴了信息论和热力学的见解。我们的方法将LLM视为一个双部分信息引擎，其中隐藏层充当Maxwell妖，通过提示Q控制将上下文C转换为答案A。我们将问题-上下文-答案（QCA）三元组建模为共享主题的概率分布。从C到Q和A的主题转换建模为转移矩阵Q和A，分别编码查询目标和实际结果。我们的语义忠实（SF）度量通过Kullback-Leibler（KL）散度来量化任意给定QCA三元组的忠实度。两个矩阵通过凸优化KL散度同时推断，最终的SF度量通过将最小散度映射到单位区间[0,1]获得，得分越高表示忠实度越高。此外，我们还提出了一种基于热力学的语义熵产（SEP）度量，用于答案生成，并表明高忠实度通常意味着低熵产。SF和SEP度量可以单独或联合用于LLM评估和幻觉控制。我们在LLM对公司SEC 10-K文件的摘要生成上展示了我们的框架。
英文摘要：
Evaluating faithfulness of Large Language Models (LLMs) to a given task is a complex challenge. We propose two new unsupervised metrics for faithfulness evaluation using insights from information theory and thermodynamics. Our approach treats an LLM as a bipartite information engine where hidden layers act as a Maxwell demon controlling transformations of context $C $ into answer $A$ via prompt $Q$. We model Question-Context-Answer (QCA) triplets as probability distributions over shared topics. Topic transformations from $C$ to $Q$ and $A$ are modeled as transition matrices ${\bf Q}$ and ${\bf A}$ encoding the query goal and actual result, respectively. Our semantic faithfulness (SF) metric quantifies faithfulness for any given QCA triplet by the Kullback-Leibler (KL) divergence between these matrices. Both matrices are inferred simultaneously via convex optimization of this KL divergence, and the final SF metric is obtained by mapping the minimal divergence onto the unit interval [0,1], where higher scores indicate greater faithfulness. Furthermore, we propose a thermodynamics-based semantic entropy production (SEP) metric in answer generation, and show that high faithfulness generally implies low entropy production. The SF and SEP metrics can be used jointly or separately for LLM evaluation and hallucination control. We demonstrate our framework on LLM summarization of corporate SEC 10-K filings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
