发表机构
University of Louisville（路易斯维尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出基于虚拟现实的数据收集流程，在Isaac Sim中收集200个立方体堆叠演示，训练行为克隆策略，实现无需重复遥操作的演示复用。
AI 中文摘要
模仿学习对机器人操作具有吸引力，但对于需要重复重置场景的多阶段任务，收集演示数据仍然是瓶颈。本工作提出了一种用于立方体堆叠的虚拟现实数据收集流程，使用自定义的5自由度机械臂，在NVIDIA Isaac Sim和Isaac Lab环境中进行。操作员通过HTC Vive Pro 2、Manus Quantum手套和OpenXR提供SE(3)末端执行器命令，以生成任务演示。所提出的框架通过重放记录的轨迹、将任务空间命令转换为关节空间动作，并使用更新的传感器或状态配置重新渲染演示，将演示收集与数据集构建分离。这使得先前收集的演示可以重复用于新的观测和动作空间，而无需重复人工遥操作。该任务要求将红色立方体堆叠在蓝色立方体上，并将绿色立方体堆叠在红色立方体上，立方体位置随机化。在30分钟内收集了200个虚拟演示，与45个真实世界演示相比，Isaac Mimic生成了100个额外样本。使用LeRobot风格的双摄像头观测，从虚拟演示中训练了一个行为克隆策略，并在仿真中进行了评估。
英文摘要
Imitation learning is attractive for robot manipulation, but collecting demonstrations remains a bottleneck for multi-stage tasks requiring repeated scene resets. This work presents a virtual-reality data-collection pipeline for cube-stacking with a custom 5-DoF arm in NVIDIA Isaac Sim and Isaac Lab. Using an HTC Vive Pro 2, Manus Quantum gloves, and OpenXR, an operator provides SE(3) end-effector commands to generate task demonstrations. The proposed framework separates demonstration collection from dataset construction by replaying recorded trajectories, converting task-space commands into joint-space actions, and re-rendering demonstrations with updated sensor or state configurations. This allows previously collected demonstrations to be reused for new observation and action spaces without repeating human teleoperation. The task requires stacking the red cube on the blue cube and the green cube on the red cube, with randomized cube placement. In 30 minutes, 200 virtual demonstrations were collected, compared with 45 real-world demonstrations, and Isaac Mimic generated 100 additional samples. A behavior-cloning policy was trained from the virtual demonstrations using LeRobot-style dual-camera observations and evaluated in simulation.
Comments5 pages, 3 figures. Accepted manuscript. Accepted for publication in the 2026 IEEE National Aerospace and Electronics Conference (NAECON)
