ELEMENTAL：从演示与视觉-语言模型中交互式学习，用于机器人奖励设计
ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Robotics
AI总结：
针对LLM设计机器人奖励时特征权衡、分布外泛化和纯文本表示不足的问题，本文提出融合语言、视觉演示、IRL与自我反思的ELEMENTAL框架，显著提升任务成功率与泛化能力。
摘要：
强化学习（RL）已在机器人任务中展现出令人信服的性能，但其成功往往取决于复杂且临时设定的奖励函数设计。研究人员已探索大型语言模型（LLM）如何帮助非专家用户更轻松地指定奖励函数。然而，LLM难以平衡不同特征的重要性，对分布外机器人任务的泛化能力较差，并且仅凭基于文本的描述无法恰当地表示问题。为应对这些挑战，我们提出 ELEMENTAL（intEractive LEarning froM dEmoNstraTion And Language），这是一种将自然语言指导与视觉用户演示相结合的新颖框架，旨在使机器人行为更好地与用户意图对齐。通过引入视觉输入，ELEMENTAL克服了纯文本任务规范的局限，同时利用逆强化学习（IRL）平衡特征权重并最优地匹配所演示的行为。ELEMENTAL还通过自我反思引入迭代反馈循环，以改进特征、奖励和策略学习。我们的实验结果表明，ELEMENTAL在任务成功率上比先前工作高出42.3%，并在分布外任务中实现41.3%更好的泛化，突显了其在LfD中的鲁棒性。
英文摘要：
Reinforcement learning (RL) has demonstrated compelling performance in robotic tasks, but its success often hinges on the design of complex, ad hoc reward functions. Researchers have explored how Large Language Models (LLMs) could enable non-expert users to specify reward functions more easily. However, LLMs struggle to balance the importance of different features, generalize poorly to out-of-distribution robotic tasks, and cannot represent the problem properly with only text-based descriptions. To address these challenges, we propose ELEMENTAL (intEractive LEarning froM dEmoNstraTion And Language), a novel framework that combines natural language guidance with visual user demonstrations to align robot behavior with user intentions better. By incorporating visual inputs, ELEMENTAL overcomes the limitations of text-only task specifications, while leveraging inverse reinforcement learning (IRL) to balance feature weights and match the demonstrated behaviors optimally. ELEMENTAL also introduces an iterative feedback-loop through self-reflection to improve feature, reward, and policy learning. Our experiment results demonstrate that ELEMENTAL outperforms prior work by 42.3% on task success, and achieves 41.3% better generalization in out-of-distribution tasks, highlighting its robustness in LfD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
