SIMMER: 基于世界模型评估LLM可执行规划中的潜在故障
SIMMER: Benchmarking Latent Failures in LLM Executable Planning with a World Model
AI总结：
提出SIMMER基准，通过人工策划的厨房领域符号世界模型，评估LLM规划中的潜在故障；实验发现前沿模型最多17%无错误计划，56%含潜在故障，多数不可逆；反事实预演可减少72%潜在故障和75%不可逆案例。
摘要：
大型语言模型（LLM）越来越多地被部署为家庭环境中自主代理的规划器。虽然现有基准评估LLM生成的计划是否成功执行，但它们忽略了一种关键类型的故障：潜在故障。与立即故障（在执行时触发即时反馈并允许及时纠正）不同，潜在故障不会立即停止计划执行，而是悄无声息地损害目标实现。在严重情况下，它们会导致不可逆的损害。为弥补这一空白，我们引入了SIMMER，这是一个通过人工策划的、基于厨房领域的符号世界模型来评估LLM规划中潜在故障的基准。SIMMER定义了一个世界模型，包含77个动作、262个独特对象和约46,800种语义真实的可能交互，这些交互源自真实世界的烹饪脚本。然后，它利用一个状态机执行器，根据世界模型验证计划，并检测即时前提违规、潜在危险和不可逆故障。在六个LLM上的实验表明，即使是最前沿的模型，其无错误计划最多也只有17%。此外，高达56%的计划包含潜在故障，其中大多数导致不可逆后果。我们进一步证明，通过反事实预演进行显式状态推理可以将潜在故障减少高达72%，不可逆案例减少高达75%，这为更鲁棒的LLM规划器指明了一个有前景的方向。
英文摘要：
Large language models (LLMs) are increasingly deployed as planners for autonomous agents in household environments. While existing benchmarks evaluate whether LLM-generated plans execute successfully, they overlook a critical type of failure: latent failures. Unlike immediate failures that trigger instant feedback at execution time and enable timely correction, latent failures do not immediately halt plan execution but silently compromise goal achievement. In severe cases, they cause irreversible harm. To address this gap, we introduce SIMMER, a benchmark for evaluating latent failures in LLM planning through a human-curated symbolic world model grounded in the kitchen domain. SIMMER defines a world model comprising 77 actions, 262 unique objects, and approximately 46,800 possible interactions that are semantically realistic, derived from real-world cooking scripts. It then leverages a state machine executor that validates plans against the world model and detects immediate precondition violations, latent hazards, and irreversible failures. Experiments across six LLMs show that even frontier models achieve at most 17% error-free plans. Moreover, up to 56% of plans contain latent failures, the majority of which lead to irreversible consequences. We further demonstrate that explicit state reasoning via counterfactual foresight simulation can reduce latent failures by up to 72% and irreversible cases by up to 75%, suggesting a promising direction for more robust LLM planners.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
