机器人桌面操作中生成式视觉预见与任务无关姿态估计的结合
Generative Visual Foresight Meets Task-Agnostic Pose Estimation in Robotic Table-Top Manipulation
AI总结：
该研究提出结合生成式视觉预见与任务无关姿态估计的闭环框架GVF-TAPE，通过生成视频预测与姿态解耦提取实现机器人桌面操作的实时自适应与跨任务泛化，降低对任务特定数据的依赖。
摘要：
非结构化环境下的机器人操作要求系统能够在多样任务间泛化，同时保持稳健可靠的性能。我们提出GVF-TAPE——一种将生成式视觉预见与任务无关姿态估计相结合的闭环框架，可实现可扩展的机器人操作。GVF-TAPE采用生成式视频模型，根据单张侧视RGB图像和任务描述预测未来的RGB-D帧，提供指导机器人动作的视觉规划。随后，一个解耦的姿态估计模型从预测帧中提取末端执行器姿态，通过底层控制器将其转化为可执行命令。通过在闭环中迭代整合视频预见与姿态估计，GVF-TAPE可在广泛任务中实现实时自适应操作。在仿真和真实环境中开展的大量实验表明，我们的方法降低了对任务特定动作数据的依赖，且泛化效果优异，为智能机器人系统提供了一种实用且可扩展的解决方案。
英文摘要：
Robotic manipulation in unstructured environments requires systems that can generalize across diverse tasks while maintaining robust and reliable performance. We introduce {GVF-TAPE}, a closed-loop framework that combines generative visual foresight with task-agnostic pose estimation to enable scalable robotic manipulation. GVF-TAPE employs a generative video model to predict future RGB-D frames from a single side-view RGB image and a task description, offering visual plans that guide robot actions. A decoupled pose estimation model then extracts end-effector poses from the predicted frames, translating them into executable commands via low-level controllers. By iteratively integrating video foresight and pose estimation in a closed loop, GVF-TAPE achieves real-time, adaptive manipulation across a broad range of tasks. Extensive experiments in both simulation and real-world settings demonstrate that our approach reduces reliance on task-specific action data and generalizes effectively, providing a practical and scalable solution for intelligent robotic systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
