arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.00929cs.AI
收藏

在程序合成代理中学习层次规划的抽象

Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents

Harvard University(哈佛大学);MIT(麻省理工学院);Institute for Human and Machine Cognition(人机认知研究院);Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University, Cambridge, MA, USA(哈佛大学马萨诸塞州剑桥市自然与人工智能研究学院)

Zergham Ahmed, Kazuki Irie, Joshua B. Tenenbaum, Christopher J. Bates, Samuel J. Gershman

AI总结:

TheoryCoder-2通过主动学习可重用的抽象,提升了程序合成代理在复杂任务中的样本效率和泛化能力。

摘要:

人类学习抽象并利用它们高效地进行规划,以快速在不同任务间泛化——这种能力仍然是最先进的大语言模型(LLM)代理和深度强化学习(RL)系统所面临的挑战。受认知科学中人们如何形成抽象和对世界知识的直观理论的启发,Theory-Based RL(TBRL)系统,如TheoryCoder,通过有效使用抽象表现出强大的泛化能力。然而,它们严重依赖于人类提供的抽象,并回避了抽象学习问题。我们介绍了TheoryCoder-2,一种新的TBRL代理,利用LLM的上下文学习能力,通过从经验中合成抽象并将其整合到层次规划过程中,主动学习可重用的抽象,而不是依赖于人工指定的抽象。我们在多样化的环境中进行了实验,包括BabyAI、Minihack和如Sokoban的VGDL游戏。我们发现,TheoryCoder-2比基线LLM代理增强经典规划领域构造、基于推理的规划和先前程序合成代理如WorldCoder的样本效率显著更高。TheoryCoder-2能够解决基线失败的复杂任务,而仅需最小的人类提示,不同于先前的TBRL系统。

英文摘要:

Humans learn abstractions and use them to plan efficiently to quickly generalize across tasks -- an ability that remains challenging for state-of-the-art large language model (LLM) agents and deep reinforcement learning (RL) systems. Inspired by the cognitive science of how people form abstractions and intuitive theories of their world knowledge, Theory-Based RL (TBRL) systems, such as TheoryCoder, exhibit strong generalization through effective use of abstractions. However, they heavily rely on human-provided abstractions and sidestep the abstraction-learning problem. We introduce TheoryCoder-2, a new TBRL agent that leverages LLMs' in-context learning ability to actively learn reusable abstractions rather than relying on hand-specified ones, by synthesizing abstractions from experience and integrating them into a hierarchical planning process. We conduct experiments on diverse environments, including BabyAI, Minihack and VGDL games like Sokoban. We find that TheoryCoder-2 is significantly more sample-efficient than baseline LLM agents augmented with classical planning domain construction, reasoning-based planning, and prior program-synthesis agents such as WorldCoder. TheoryCoder-2 is able to solve complex tasks that the baselines fail, while only requiring minimal human prompts, unlike prior TBRL systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑