语言模型的交叉熵博弈：从隐式知识到通用能力度量
Cross-Entropy Games for Language Models: From Implicit Knowledge to General Capability Measures
AI总结：
本文提出交叉熵（Xent）博弈框架，将LLM的隐式知识转化为一系列可计算任务，用于度量语言模型的通用能力，并借助进化动力学思想探索该空间。
摘要：
大型语言模型（LLMs）在文本上定义了概率测度。通过考虑隐式知识问题，即LLM“知道”这样一个测度意味着什么，以及这在算法上意味着什么，我们自然地被引导去制定一系列超越生成式采样的任务，涉及摘要、反事实思维、异常检测、原创性搜索、反向提示、辩论、创造性求解等形式。这些任务可以表述为基于LLM测度的博弈，我们称之为交叉熵（Xent）博弈。Xent博弈可以是单人的或多人的。它们涉及交叉熵得分和交叉熵约束，并且可以表示为简单的计算图和程序。我们展示了Xent博弈空间足够大，足以包含大量有趣的例子，同时可以从基本的博弈论一致性公理构建。然后我们讨论了如何使用Xent博弈空间来衡量LLM的能力。这导致了Xent博弈度量的构建：有限族的Xent博弈，可用作能力基准，从给定的范围中通过提取覆盖测度来构建。为了解决与衡量通用能力挑战相关的无界范围问题，我们提出使用受进化动力学启发的思想，以连贯的方式探索Xent博弈空间。
英文摘要：
Large Language Models (LLMs) define probability measures on text. By considering the implicit knowledge question of what it means for an LLM to know such a measure and what it entails algorithmically, we are naturally led to formulate a series of tasks that go beyond generative sampling, involving forms of summarization, counterfactual thinking, anomaly detection, originality search, reverse prompting, debating, creative solving, etc. These tasks can be formulated as games based on LLM measures, which we call Cross-Entropy (Xent) Games. Xent Games can be single-player or multi-player. They involve cross-entropy scores and cross-entropy constraints, and can be expressed as simple computational graphs and programs. We show the Xent Game space is large enough to contain a wealth of interesting examples, while being constructible from basic game-theoretic consistency axioms. We then discuss how the Xent Game space can be used to measure the abilities of LLMs. This leads to the construction of Xent Game measures: finite families of Xent Games that can be used as capability benchmarks, built from a given scope, by extracting a covering measure. To address the unbounded scope problem associated with the challenge of measuring general abilities, we propose to explore the space of Xent Games in a coherent fashion, using ideas inspired by evolutionary dynamics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
