Model-free policy learning has been shown to be capable of learning manipulation policies which can solve long-time horizon tasks using single-step manipulation primitives. However, training these policies is a time-consuming process requiring large amounts of data. We propose the Local Dynamics Model (LDM) which efficiently learns the state-transition function for these manipulation primitives. By combining the LDM with model-free policy learning, we can learn policies which can solve complex manipulation tasks using one-step lookahead planning. We show that the LDM is both more sample-efficient and outperforms other model architectures. When combined with planning, we can outperform other model-based and model-free policies on several challenging manipulation tasks in simulation.
Transporters with Visual Foresight for Solving Unseen Rearrangement Tasks
The Johns Hopkins University(约翰斯·霍普金斯大学);National University of Singapore(新加坡国立大学);NVIDIA(英伟达)
重排任务已被确定为智能机器人操作的一个关键挑战,但很少有方法允许精确构建看不见的结构。我们提出了一个视觉预见模型的拾取和放置重排操作,这是能够有效地学习。此外,我们开发了一个多模态的行动建议模块,它建立在目标条件运输网络,一个国家的最先进的模仿学习方法。我们的基于图像的任务规划方法,Transporters with Visual Focal,能够从少量数据中学习,并以zero-shot方式推广到多个看不见的任务。TVF能够在仿真和真实机器人实验中提高最先进的模仿学习方法在看不见的任务上的性能。特别是,看不见的任务的平均成功率从55.4%提高到78.5%,在模拟实验和30%到63.3%,在真正的机器人实验时,只有几十个专家演示。视频和代码可在我们的项目网站上获得:https://chirikjianlab.github.io/tvf/
PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
Department of Automation, Tsinghua University(自动化系,清华大学);Shenzhen International Graduation School, Tsinghua University(深圳国际研究生院,清华大学);Department of Computer Science and Technology, Tsinghua University(计算机科学与技术系,清华大学);School of Mechanical and Aerospace Engineering, Nanyang Technological University(机械与航空航天工程学院,南洋理工大学)
AeroPlace-Flow: Language-Grounded Object Placement for Aerial Manipulators via Visual Foresight and Object Flow
Robotics Research Center, IIIT Hyderabad, India(印度IIIT海得拉巴机器人研究中心);Department of Computer Science, University of Manchester, UK(英国曼彻斯特大学计算机科学系);Newcastle University, UK(英国新castle大学)
NavWM: A Unified Navigation World Model for Foresight-Driven Planning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beihang University(北京航空航天大学);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
WAM-Nav: Asymmetric Latent World-Action Modeling for Unified Visual Navigation
Nanjing University(南京大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);FiveAges;National University of Defense Technology(国防科技大学);Tsinghua University(清华大学);GigaAI(极佳科技)
FocalPolicy: Frequency-Optimized Chunking and Locally Anchored Flow Matching for Coherent Visuomotor Policy
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences(机器人与智能系统国家重点实验室,沈阳自动化研究所,中国科学院);University of the Chinese Academy of Sciences(中国科学院大学);University of Trento(特伦多大学)
FoAM: Foresight-Augmented Multi-Task Imitation Policy for Robotic Manipulation
University of Southern California(南加州大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院)
LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
Tsinghua University(清华大学);Jilin University(吉林大学);Nankai University(南开大学);Peking University(北京大学);Harbin Institute of Technology(哈尔滨工业大学);Zhongguancun Academy(中关村学院);Striding.AI;Infinigence AI
The Aqualoc Dataset: Towards Real-Time Underwater Localization from a Visual-Inertial-Pressure Acquisition System
ONERA, Université Paris Saclay(法国航空航天实验室(ONERA),巴黎萨克雷大学);LIRMM, Université de Montpellier, CNRS(蒙彼利埃计算机、机器人与微电子实验室,蒙彼利埃大学,法国国家科学研究中心);DRASSM, Ministry of Culture(法国文化部水下与海底考古研究局)