Ego2World：将自我中心烹饪视频编译为可执行世界以进行信念状态规划

中文摘要 AI 辅助

自我中心视频捕捉了人们如何进行日常活动，然而测试一个智能体需要评估其自身选择行动的后果。我们引入了Ego2World，一个将标注的烹饪活动转化为部分观测下可执行规划环境的基准。其编译器将源步骤和对象链接到符号动作规则、持久世界状态和明确的任务条件，使研究人员能够执行智能体提出的动作并检查其结果。世界状态和智能体信念被分开维护，从而能够对持续任务中的规划与信息重用进行受控研究。在105个任务上评估六个规划器表明，被接受的操作往往未能满足任务目标。执行轨迹和条件检查区分了中断运行、部分达成以及未达成目标而完成执行的情况。在另一项配对的Qwen-Plus研究中，持久信念将动作有效性提高了4.15个百分点，并将视觉查询尝试减少了90.27%，同时使用了更高的令牌数且未检测到完成度增益。Ego2World提供了一个可复用的测试平台，用于追踪规划和记忆选择如何影响执行、观测需求及任务达成，将记录的人类活动与交互式智能体的开发和评估联系起来。

英文摘要

Egocentric videos capture how people carry out everyday activities, yet testing an agent requires evaluating the consequences of actions it chooses itself. We introduce Ego2World, a benchmark that turns annotated cooking activities into executable planning environments under partial observation. Its compiler links source steps and objects to symbolic action rules, persistent world states, and explicit task conditions, so researchers can execute an agent's proposed actions and check their outcomes. World state and agent belief are maintained separately, enabling controlled studies of planning and information reuse across continuing tasks. Evaluating six planners on 105 tasks shows that accepted operations often leave task goals unmet. Execution traces and condition checks distinguish interrupted runs, partial attainment, and completed execution without goal attainment. In a separate paired Qwen-Plus study, persistent belief improves action validity by 4.15 percentage points and reduces visual-query attempts by 90.27%, with higher token use and no detected completion gain. Ego2World provides a reusable testbed for tracing how planning and memory choices affect execution, observation demand, and task attainment, connecting recorded human activity to the development and evaluation of interactive agents.

发表机构

• Nankai University（南开大学）
• National University of Singapore（新加坡国立大学）
• Institute for Infocomm Research, A*STAR（资讯通信研究院，新加坡科技研究局）

机构由 AI 辅助整理，请以论文原文为准。