arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.14977cs.RO
收藏

EnactOpt:面向具身智能体的诊断引导式执行框架进化

EnactOpt: Diagnosis-Guided Harness Evolution for Embodied Agents

Tsinghua University(清华大学);Peking University(北京大学);Kling AI(克林人工智能公司);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

ZhiYuan Feng, Zhuoshi Pan, Zhaolu Kang, Zhongxia Chen, Shuo Yang, Sen Ye, Heng Wang, Han Li

AI总结:

EnactOpt在不更新底层模型的前提下,利用有限执行反馈联合进化具身智能体的文本技能与上下文构建代码,在多个基准任务上提升了任务成功率且优于基线方法。

摘要:

基于大模型构建的具身智能体依赖执行框架(execution harness)来组织观测结果、引导动作,并将执行反馈融入后续决策。失败不仅可能源于模型能力有限,还可能源于框架呈现证据和指导使用的方式。这些模型外部机制提供了无需更新模型参数的适配目标,但收集特定任务的交互经验成本高昂。仅任务级结果无法揭示模型输入中是否缺少相关证据,或证据存在但使用不当。此外,这两方面相互耦合:有效的动作指导可能需要当前上下文缺失的证据,而仅暴露该证据无法确保做出恰当响应。我们提出EnactOpt,它在保持底层模型和运行时固定的前提下,利用有限的执行反馈共同进化可复用的文本技能和上下文构建代码。EnactOpt通过将动作反馈与实际模型输入及后续动作关联来诊断相邻决策;结构化的回合摘要通过提示级编辑操作引导联合修订;候选框架需经过兼容性检查和基于验证的保留。在VLABench、ALFWorld和具身智能体接口(Embodied Agent Interface, EAI)上,EnactOpt相比初始执行框架将任务成功率提升了约10-27个百分点,且优于GEPA、TextGrad和Trace2Skill等基线方法。这些提升可延伸至未用于框架进化或候选选择的任务条件。研究结果支持了具身适配的互补路径:通过共同优化证据呈现和动作执行方式,将有限的执行经验转化为可复用的行为改进。

英文摘要:

Embodied agents built on foundation models rely on an execution harness to organize observations, guide actions, and incorporate execution feedback into subsequent decisions. Failures can arise not only from limited model capabilities, but also from how the harness presents evidence and guides its use. These model-external mechanisms provide an adaptation target without updating model parameters, but task-specific interaction experience can be costly to collect. Task-level outcomes alone do not reveal whether relevant evidence was missing from the model input or was available but inadequately used. Moreover, these two aspects are coupled: effective action guidance may require evidence that the current context omits, while exposing that evidence alone does not ensure an appropriate response. We introduce EnactOpt, which uses limited execution feedback to jointly evolve reusable textual skills and context-construction code while keeping the underlying models and runtime fixed. It diagnoses adjacent decisions by relating action feedback to the actual model input and subsequent action. Structured episode summaries guide joint revisions through prompt-level edit operators, and candidate harnesses undergo compatibility checks and validation-based retention. Across VLABench, ALFWorld, and Embodied Agent Interface (EAI), EnactOpt improves task success by approximately 10-27 percentage points over the initial harnesses and outperforms the evaluated GEPA, TextGrad, and Trace2Skill baselines. These gains extend to task conditions not used for harness evolution or candidate selection. The results support a complementary route to embodied adaptation: turning limited execution experience into reusable behavioral improvements by jointly refining how evidence is presented and acted upon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑