Evaluating Real-World Robot Manipulation Policies in Simulation
UC San Diego(加利福尼亚大学圣地亚哥分校);Stanford University(斯坦福大学);UC Berkeley(加利福尼亚大学伯克利分校);Google Deepmind(谷歌DeepMind)
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, C… 展开作者
Xuanlin Li, Kyle Hsu, Jiayuan Gu, Karl Pertsch, Oier Mees, Homer Rich Walke, Chuyuan Fu, Ishikaa Lunawat, Isabel Sieh, Sean Kirmani, Sergey Levine, Jiajun Wu, Chelsea Finn, Hao Su, Quan Vuong, Ted Xiao
The field of robotics has made significant advances towards generalist robot manipulation policies. However, real-world evaluation of such policies is not scalable and faces reproducibility challenges, which are likely to worsen as policies broaden the spectrum of tasks they can perform. We identify control and visual disparities between real and simulated environments as key challenges for reliable simulated evaluation and propose approaches for mitigating these gaps without needing to craft full-fidelity digital twins of real-world environments. We then employ these approaches to create SIMPLER, a collection of simulated environments for manipulation policy evaluation on common real robot setups. Through paired sim-and-real evaluations of manipulation policies, we demonstrate strong correlation between policy performance in SIMPLER environments and in the real world. Additionally, we find that SIMPLER evaluations accurately reflect real-world policy behavior modes such as sensitivity to various distribution shifts. We open-source all SIMPLER environments along with our workflow for creating new environments at https://simpler-env.github.io to facilitate research on general-purpose manipulation policies and simulated evaluation frameworks.
Robust Policies via Mid-Level Visual Representations: An Experimental Study in Manipulation and Navigation
University of California, Berkeley(加州大学伯克利分校);Stanford University(斯坦福大学);New York University(纽约大学);Swiss Federal Institute of Technology Lausanne (EPFL)(洛桑联邦理工学院)
Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Intervention
State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University(机械系统与振动国家重点实验室,机械工程学院,上海交通大学);Shanghai Key Laboratory of Intelligent Robotics, Meta Robotics Institute, Shanghai Jiao Tong University, Shanghai 200240, China(智能机器人上海市重点实验室,元机器人研究院,上海交通大学,上海200240,中国);The University of Tokyo(东京大学)
Should We Learn Contact-Rich Manipulation Policies from Sampling-Based Planners?
New York University(纽约大学);Boston Dynamics AI Institute(波士顿动力AI研究院);Georgia Institute of Technology(佐治亚理工学院);Cornell University(康奈尔大学);Artificial and Natural Intelligence Toulouse Institute(图卢兹人工智能与自然智能研究所)
Benchmarking Structured Policies and Policy Optimization for Real-World Dexterous Object Manipulation
Technical University of Darmstadt(达姆施塔特工业大学);University of Washington(华盛顿大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Cornell University(康奈尔大学);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
LTLDiff: Finite Linear Temporal Logic-Guided Data Generation and Diffusion Policies for Multi-agent Robotic Manipulation
University of Toronto(多伦多大学);A*STAR Centre for Frontier AI Research (A*STAR CFAR), Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局前沿人工智能研究中心(A*STAR CFAR))
Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
School of Computing, National University of Singapore(新加坡国立大学计算机学院);Smart Systems Institute, NUS(NUS智能系统研究所);Guangdong Institute of Intelligence Science and Technology(广东智能科学与技术研究院);Beijing Zhongguancun Academy(北京中关村学院)
Effective Tuning Strategies for Generalist Robot Manipulation Policies
University of Adelaide(阿德莱德大学);Commonwealth Scientific and Industrial Research Organisation (CSIRO)(联邦科学与工业研究组织(CSIRO));Shanghai Jiao Tong University(上海交通大学);TikTok
EBench: Elemental Diagnosis of Generalist Mobile Manipulation Policies
Shanghai AI Laboratory(上海人工智能实验室);Xi’an Jiaotong University(西安交通大学);Institute for AI Industry Research (AIR), Tsinghua University(清华大学智能产业研究院);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University(浙江大学)
Endowing Robots with Longer-term Autonomy by Recovering from External Disturbances in Manipulation through Grounded Anomaly Classification and Recovery Policies
PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation
Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China(计算机科学与技术系,人工智能研究院,清华-博世联合机器学习中心,THBI实验室,BNRist中心,清华大学,北京,100084,中国);Peng Cheng Laboratory, 518108, China(鹏城实验室,518108,中国)