ExploRLLM：用大语言模型引导强化学习中的探索
ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models
AI总结：
针对机器人操作中RL样本效率低、FMs物理理解有限的问题，提出ExploRLLM方法，结合FMs生成策略代码和表示以提升RL收敛性、残余RL智能体弥补FMs缺陷，在桌面操作任务中优于纯FMs和RL基线且实现零样本仿真到现实迁移。
摘要：
在机器人操作中，强化学习（RL）常面临样本效率低和收敛不确定的问题，尤其是在大观测和动作空间中。基础模型（FMs）提供了一种替代方案，在零样本和少样本设置中展现出潜力，但由于物理和空间理解有限，它们可能不可靠。我们提出ExploRLLM，一种结合两种范式优势的方法：FMs通过生成策略代码和高效表示来提升RL收敛性，而残余RL智能体则弥补FMs有限的物理理解。实验表明，ExploRLLM在桌面操作任务中优于基于FMs的策略和RL基线，且真实世界实验显示其策略具有良好的零样本仿真到现实迁移能力。补充材料见https://explorllm.github.io。
英文摘要：
In robot manipulation, Reinforcement Learning (RL) often suffers from low sample efficiency and uncertain convergence, especially in large observation and action spaces. Foundation Models (FMs) offer an alternative, demonstrating promise in zero-shot and few-shot settings. However, they can be unreliable due to limited physical and spatial understanding. We introduce ExploRLLM, a method that combines the strengths of both paradigms. In our approach, FMs improve RL convergence by generating policy code and efficient representations, while a residual RL agent compensates for the FMs' limited physical understanding. We show that ExploRLLM outperforms both policies derived from FMs and RL baselines in table-top manipulation tasks. Additionally, real-world experiments show that the policies exhibit promising zero-shot sim-to-real transfer. Supplementary material is available at https://explorllm.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
