面向复杂长周期机器人操作任务的内在语言引导探索
Intrinsic Language-Guided Exploration for Complex Long-Horizon Robotic Manipulation Tasks
AI总结：
针对稀疏奖励下复杂长周期机器人操作任务的强化学习探索难题，提出IGE-LLMs框架，用大语言模型作内在奖励引导探索，实验证明其性能更优、模块化强、参数鲁棒性好。
摘要：
当前强化学习算法在稀疏且复杂的环境中表现不佳，尤其是在包含大量不同序列的长周期操作任务中问题尤为突出。本研究提出了IGE-LLMs（大语言模型内在引导探索，Intrinsically Guided Exploration from Large Language Models）框架。该框架将大语言模型（LLMs）用作辅助内在奖励，引导强化学习的探索过程，以解决具有稀疏奖励的复杂长周期机器人操作任务。我们在两类场景中评估了该框架及相关内在学习方法：一类是存在探索挑战的环境，另一类是同时面临探索和长周期双重挑战的复杂机器人操作任务。结果表明，IGE-LLMs具备以下特性：（i）与相关内在方法及直接使用LLM进行决策的方式相比，性能显著更高；（ii）可与现有学习方法结合并形成互补，凸显了其模块化特性；（iii）对不同的内在缩放参数相当不敏感；（iv）在不确定性水平和任务周期提升的情况下仍能保持鲁棒性。
英文摘要：
Current reinforcement learning algorithms struggle in sparse and complex environments, most notably in long-horizon manipulation tasks entailing a plethora of different sequences. In this work, we propose the Intrinsically Guided Exploration from Large Language Models (IGE-LLMs) framework. By leveraging LLMs as an assistive intrinsic reward, IGE-LLMs guides the exploratory process in reinforcement learning to address intricate long-horizon with sparse rewards robotic manipulation tasks. We evaluate our framework and related intrinsic learning methods in an environment challenged with exploration, and a complex robotic manipulation task challenged by both exploration and long-horizons. Results show IGE-LLMs (i) exhibit notably higher performance over related intrinsic methods and the direct use of LLMs in decision-making, (ii) can be combined and complement existing learning methods highlighting its modularity, (iii) are fairly insensitive to different intrinsic scaling parameters, and (iv) maintain robustness against increased levels of uncertainty and horizons.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
