RDGen: 通过强化学习生成高质量机器人学习的演示
RDGen: Demonstration Generation for High-Quality Robot Learning via Reinforcement Learning
AI总结：
提出RDGen框架，利用从仿真到真实的强化学习策略生成高质量机器人演示轨迹，用于训练视觉-语言-动作模型，相比人工遥操作产生更平滑轨迹并提升下游性能。
摘要：
视觉-语言-动作（VLA）模型已成为通用机器人控制的一种有前景的范式。然而，其性能仍然从根本上受限于高质量机器人轨迹数据的可用性。在当前的机器人学习实践中，这些数据主要通过人类遥操作收集，这需要大量人力、成本高昂且难以扩展。在本文中，我们提出了RDGen，一种用于生成高质量机器人演示的仿真到真实强化学习框架。RDGen并非仅将强化学习用作最终控制策略，而是利用训练好的RL策略作为结构化的轨迹生成器。该系统由一个基于VLM的任务解析器（用于识别任务相关物体）、一个基于Grounding DINO的物体定位器以及一个从仿真迁移到真实机器人的RL策略组成。然后，成功的 rollout 被收集为干净、高质量的演示，用于下游VLA训练，而仿真阶段进一步以极低的边际成本提供可扩展的额外轨迹来源。在拾取和放置任务上的实验表明，迁移后的RL策略实现了高任务成功率。与人类遥操作相比，RDGen生成的轨迹显著更平滑，并产生更优的下游VLA性能。这些结果表明，RL生成的演示可以作为机器人策略学习更可靠和一致的监督信号。
英文摘要：
Vision-Language-Action (VLA) models have emerged as a promising paradigm for general-purpose robot control. However, their performance remains fundamentally constrained by the availability of high-quality robot trajectory data. In current robot learning practice, such data are primarily collected through human teleoperation, which is labor-intensive, costly, and difficult to scale. In this paper, we propose RDGen, a sim-to-real reinforcement learning framework for generating high-quality robot demonstrations. Rather than employing reinforcement learning solely as the final control policy, RDGen leverages trained RL policies as a structured trajectory generator. The system consists of a VLM-based task parser that identifies task-relevant objects, a Grounding DINO-based object localizer, and an RL policy transferred from simulation to the real robot. Successful rollouts are then harvested as clean, high-quality demonstrations for downstream VLA training, while the simulation stage further provides a scalable source of additional trajectories at little marginal cost. Experiments on a pick-and-place task demonstrate that the transferred RL policy achieves a high task success rate. Compared with human teleoperation, RDGen produces significantly smoother trajectories and yields superior downstream VLA performance. These results indicate that RL-generated demonstrations can serve as more reliable and consistent supervisory signals for robot policy learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
