PRISM：在模拟的具身环境中进行规划与意图推理
PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments
AI总结：
PRISM通过诊断性基准测试识别具身代理失败的根源，通过三个能力层级评估基本能力、推理能力和长周期能力，揭示不同模型在空间定位、意图解析和长周期协调上的性能差异。
摘要：
当基于LLM的具身代理在家庭任务中失败时，问题可能源于误识别的对象、遗忘的子目标或行动序列不佳，但现有基准仅报告单一成功率，无法确定哪个认知模块负责。我们提出了PRISM，一个诊断基准，重新定义问题：不仅问代理是否成功，还问最可能导致失败的能力是什么。PRISM基于五个逼真多房间公寓（每套4-8间），将300个人工验证的任务分为三个能力层级——基本能力、推理能力和长周期能力，分别隔离感知到行动的 grounding、隐含意图解析和持续多步骤协调。PRISM提供一个代理无关的可执行动作API，允许任意代理（LLM代理、VLM代理、符号规划器、RL策略和混合系统）在相同基准协议下进行端到端评估。为了支持更深入的诊断，可选的感知、记忆和规划探针可以被采用、替换或完全绕过，从而在需要时实现受控的组件级分析。在七个现代LLM上的实验建立了一个清晰的层次：在Oracle感知下，显式空间定位不是主导的失败来源，隐含意图解析是所有模型家族的显著瓶颈，而长周期协调暴露了明显的能力建设悬崖——轻量模型崩溃到20.0%的成功率，同时消耗比前沿模型更多的token，这是补偿性过度推理而非真实规划能力的标志。项目页面：https://sj-li.com/PROJ/PRISM。
英文摘要：
When an LLM-based embodied agent fails at a household task, the culprit could be misidentified objects, forgotten sub-goals, or poor action sequencing -- yet existing benchmarks report only a single success rate, making it impossible to tell which cognitive module is responsible. We present PRISM, a diagnostic benchmark that reframes this problem: rather than asking only \textit{did the agent succeed?}, PRISM asks \textit{which capability is most likely responsible for failure?} Built on five photorealistic multi-room apartments (4--8 rooms each), PRISM structures 300 human-verified tasks into three capability tiers -- \textit{Basic Ability}, \textit{Reasoning Ability}, and \textit{Long-horizon Ability} -- that isolate perception-to-action grounding, implicit intent resolution, and sustained multi-step coordination respectively. PRISM exposes an agent-agnostic executable action API that allows arbitrary agents: LLM agents, VLM agents, symbolic planners, RL policies, and hybrid systems, to be evaluated end-to-end under the same benchmark protocol. To support deeper diagnosis, optional probes for perception, memory, and planning can be adopted, replaced, or bypassed entirely, enabling controlled component-level analysis when desired. Experiments on seven contemporary LLMs establish a clear hierarchy: explicit spatial grounding is not the dominant failure source under oracle perception, implicit intent resolution is a significant bottleneck for all model families, and long-horizon coordination exposes a stark capability cliff -- lightweight models collapse to as low as 20.0\% success while simultaneously consuming more tokens than their frontier counterparts, a signature of compensatory over-reasoning rather than genuine planning capability. Project page: \href{https://sj-li.com/PROJ/PRISM}{link}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
