arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.03689cs.CLcs.AI
收藏

评估生成模型中隐含的世界模型

Evaluating the World Model Implicit in a Generative Model

Harvard University(哈佛大学);MIT(麻省理工学院);Cornell University(康奈尔大学)

Keyon Vafa, Justin Y. Chen, Ashesh Rambachan, Jon Kleinberg, Sendhil Mullainathan

AI总结:

针对大语言模型隐式学习世界模型的可能性,本文提出基于确定性有限自动机形式化该问题,并受 Myhill-Nerode 定理启发设计新评估指标,揭示现有生成模型的世界模型缺乏连贯性且存在脆弱性。

摘要:

最近的研究表明,大型语言模型可能会隐式地学习世界模型。我们应如何评估这种可能性?我们将底层现实由确定性有限自动机 governing 的情况下的这一问题进行了形式化。这包括诸如简单逻辑推理、地理导航、游戏博弈和化学等多种多样的问题。受语言理论中经典的 Myhill-Nerode 定理启发,我们提出了用于评估世界模型恢复的新评估指标。我们在三个领域展示了它们的效用:游戏博弈、逻辑谜题和导航。在所有领域中,我们所考察的生成模型在评估世界模型的现有诊断方法上表现良好,但我们的评估指标揭示了它们的世界模型远不如表面上看起来那样连贯。这种不连贯性导致了脆弱性:使用生成模型来解决相关但存在细微差异的任务可能会导致失败。构建能够有意义地捕捉其所建模领域底层逻辑的生成模型将具有巨大价值;我们的结果提出了评估给定模型距离该目标有多近的新方法。

英文摘要:

Recent work suggests that large language models may implicitly learn world models. How should we assess this possibility? We formalize this question for the case where the underlying reality is governed by a deterministic finite automaton. This includes problems as diverse as simple logical reasoning, geographic navigation, game-playing, and chemistry. We propose new evaluation metrics for world model recovery inspired by the classic Myhill-Nerode theorem from language theory. We illustrate their utility in three domains: game playing, logic puzzles, and navigation. In all domains, the generative models we consider do well on existing diagnostics for assessing world models, but our evaluation metrics reveal their world models to be far less coherent than they appear. Such incoherence creates fragility: using a generative model to solve related but subtly different tasks can lead to failures. Building generative models that meaningfully capture the underlying logic of the domains they model would be immensely valuable; our results suggest new ways to assess how close a given model is to that goal.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑