arXiv 2504.041912025-09-12cs.CVcs.RO

GROVE：一种用于学习开放词汇物理技能的通用的奖励机制

GROVE: A Generalized Reward for Learning Open-Vocabulary Physical Skill

Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；State Key Laboratory of General Artificial Intelligence, BIGAI（北京通用人工智能研究院通用人工智能国家重点实验室）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）

Jieming Cui, Tengyu Liu, Ziyu Meng, Jiale Yu, Ran Song, Wei Zhang, Yixin Zhu, Siyuan Huang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对模拟智能体开放词汇物理技能学习，提出GROVE通用奖励框架，结合LLM生成约束与VLM反馈迭代优化，经Pose2CLIP桥接域差距，显著提升自然度与任务完成度并加速训练。

摘要：

为模拟智能体学习开放词汇的物理技能是人工智能领域的一项重大挑战。当前的强化学习方法面临关键限制：手动设计的奖励在不同任务间缺乏可扩展性，而基于演示的方法则难以泛化到其训练分布之外。我们提出了GROVE，一个通用的奖励框架，无需手动工程或特定任务的演示即可实现开放词汇的物理技能学习。我们的关键洞察在于，大语言模型（LLMs）和视觉语言模型（VLMs）提供了互补的指导——LLMs生成捕捉任务需求的精确物理约束，而VLMs评估运动语义和自然度。通过迭代设计过程，基于VLM的反馈持续优化LLM生成的约束，形成一个自我改进的奖励系统。为了弥合模拟与自然图像之间的领域差距，我们开发了Pose2CLIP，一个轻量级的映射器，能够高效地将智能体姿态直接投影到语义特征空间，而无需计算昂贵的渲染。跨多种具身形态和学习范式的广泛实验证明了GROVE的有效性，在运动自然度上提高了22.2%，任务完成得分上提高了25.7%，同时训练速度比先前方法快8.4倍。这些结果为在模拟环境中可扩展的物理技能获取奠定了新的基础。

英文摘要：

Learning open-vocabulary physical skills for simulated agents presents a significant challenge in artificial intelligence. Current reinforcement learning approaches face critical limitations: manually designed rewards lack scalability across diverse tasks, while demonstration-based methods struggle to generalize beyond their training distribution. We introduce GROVE, a generalized reward framework that enables open-vocabulary physical skill learning without manual engineering or task-specific demonstrations. Our key insight is that Large Language Models(LLMs) and Vision Language Models(VLMs) provide complementary guidance -- LLMs generate precise physical constraints capturing task requirements, while VLMs evaluate motion semantics and naturalness. Through an iterative design process, VLM-based feedback continuously refines LLM-generated constraints, creating a self-improving reward system. To bridge the domain gap between simulation and natural images, we develop Pose2CLIP, a lightweight mapper that efficiently projects agent poses directly into semantic feature space without computationally expensive rendering. Extensive experiments across diverse embodiments and learning paradigms demonstrate GROVE's effectiveness, achieving 22.2% higher motion naturalness and 25.7% better task completion scores while training 8.4x faster than previous methods. These results establish a new foundation for scalable physical skill acquisition in simulated environments.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。