We present CREST, an approach for causal reasoning in simulation to learn the relevant state space for a robot manipulation policy. Our approach conducts interventions using internal models, which are simulations with approximate dynamics and simplified assumptions. These interventions elicit the structure between the state and action spaces, enabling construction of neural network policies with only relevant states as input. These policies are pretrained using the internal model with domain randomization over the relevant states. The policy network weights are then transferred to the target domain (e.g., the real world) for fine tuning. We perform extensive policy transfer experiments in simulation for two representative manipulation tasks: block stacking and crate opening. Our policies are shown to be more robust to domain shifts, more sample efficient to learn, and scale to more complex settings with larger state spaces. We also show improved zero-shot sim-to-real transfer of our policies for the block stacking task.
College of Computer and Information Science, Northeastern University（东北大学计算机与信息科学学院）
In applications of deep reinforcement learning to robotics, it is often the case that we want to learn pose invariant policies: policies that are invariant to changes in the position and orientation of objects in the world. For example, consider a peg-in-hole insertion task. If the agent learns to insert a peg into one hole, we would like that policy to generalize to holes presented in different poses. Unfortunately, this is a challenge using conventional methods. This paper proposes a novel state and action abstraction that is invariant to pose shifts called \textit{deictic image maps} that c…
Arizona State University（亚利桑那州立大学）；Intel AI（英特尔人工智能研究院）；Carnegie Mellon University（卡内基梅隆大学）
Language-conditioned policies allow robots to interpret and execute human instructions. Learning such policies requires a substantial investment with regards to time and compute resources. Still, the resulting controllers are highly device-specific and cannot easily be transferred to a robot with different morphology, capability, appearance or dynamics. In this paper, we propose a sample-efficient approach for training language-conditioned manipulation policies that allows for rapid transfer across different types of robots. By introducing a novel method, namely Hierarchical Modularity, and ad…
Royal Institute of Technology（瑞典皇家理工学院）；ASEA Brown Boveri (ABB) Corporate Research（ABB集团企业研究院）
Deep reinforcement learning (DRL) has been successfully used to solve various robotic manipulation tasks. However, most of the existing works do not address the issue of control stability. This is in sharp contrast to the control theory community where the well-established norm is to prove stability whenever a control law is synthesized. What makes traditional stability analysis difficult for DRL are the uninterpretable nature of the neural network policies and unknown system dynamics. In this work, stability is obtained by deriving an interpretable deep policy structure based on the $\textit{…
Should We Learn Contact-Rich Manipulation Policies from Sampling-Based Planners?
New York University（纽约大学）；Boston Dynamics AI Institute（波士顿动力AI研究院）；Georgia Institute of Technology（佐治亚理工学院）；Cornell University（康奈尔大学）；Artificial and Natural Intelligence Toulouse Institute（图卢兹人工智能与自然智能研究所）
Technical University of Darmstadt（达姆施塔特工业大学）；University of Washington（华盛顿大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Cornell University（康奈尔大学）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
Dexterous manipulation is a challenging and important problem in robotics. While data-driven methods are a promising approach, current benchmarks require simulation or extensive engineering support due to the sample inefficiency of popular methods. We present benchmarks for the TriFinger system, an open-source robotic platform for dexterous manipulation and the focus of the 2020 Real Robot Challenge. The benchmarked methods, which were successful in the challenge, can be generally described as structured policies, as they combine elements of classical robotics and modern policy optimization. T…
LTLDiff: Finite Linear Temporal Logic-Guided Data Generation and Diffusion Policies for Multi-agent Robotic Manipulation
University of Toronto（多伦多大学）；A*STAR Centre for Frontier AI Research (A*STAR CFAR), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局前沿人工智能研究中心（A*STAR CFAR））
Abstracting Robot Manipulation Skills via Mixture-of-Experts Diffusion Policies
School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Smart Systems Institute, NUS（NUS智能系统研究所）；Guangdong Institute of Intelligence Science and Technology（广东智能科学与技术研究院）；Beijing Zhongguancun Academy（北京中关村学院）
EBench: Elemental Diagnosis of Generalist Mobile Manipulation Policies
Shanghai AI Laboratory（上海人工智能实验室）；Xi’an Jiaotong University（西安交通大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学智能产业研究院）；Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiao Tong University（上海交通大学）；Zhejiang University（浙江大学）
MotionTrans: Human VR Data Enable Motion-Level Learning for Robotic Manipulation Policies
Institute for Interdisciplinary Information Sciences（交叉信息学院）；Shanghai Qi Zhi Institute（上海启智研究所）；State Key Laboratory of Multimedia Information Processing School of Computer Science（多媒体信息处理国家重点实验室计算机科学学院）；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）
TACTIC: Understanding Tactile Encoders and Conditioning for Contact-rich Robot Manipulation Policies
University of Technology Nuremberg（纽伦堡工业大学）；Ludwig-Maximilians University Munich（慕尼黑路德维希-马克西米利安大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Deggendorf Institute of Technology（德根多夫理工学院）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；University of Tromso（特罗姆瑟大学）；DLR-German Aerospace Center（德国航空航天中心）
When would Vision-Proprioception Policies Fail in Robotic Manipulation?
Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学光明学院人工智能学院）；Beijing Key Laboratory of Research on Large Models（北京大模型与智能治理研究重点实验室）；School of Artificial Intelligence, Beihang University（北航人工智能学院）
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；Vector Institute（矢量研究所）
Learned visuomotor policies have shown considerable success as an alternative to traditional, hand-crafted frameworks for robotic manipulation. Surprisingly, an extension of these methods to the multiview domain is relatively unexplored. A successful multiview policy could be deployed on a mobile manipulation platform, allowing the robot to complete a task regardless of its view of the scene. In this work, we demonstrate that a multiview policy can be found through imitation learning by collecting data from a variety of viewpoints. We illustrate the general applicability of the method by learn…
KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policies
Hong Kong Embodied AI Lab（香港具身智能实验室）；The Chinese University of Hong Kong（香港中文大学）；University of Electronic Science and Technology of China（电子科技大学）；Shanghai Jiao Tong University（上海交通大学）
Tsinghua University（清华大学）；University of Edinburgh（爱丁堡大学）
In robotic grasping, objects are often occluded in ungraspable configurations such that no pregrasp pose can be found, eg large flat boxes on the table that can only be grasped from the side. Inspired by humans' bimanual manipulation, eg one hand to lift up things and the other to grasp, we address this type of problems by introducing pregrasp manipulation - push and lift actions. We propose a model-free Deep Reinforcement Learning framework to train control policies that utilize visual information and proprioceptive states of the robot to autonomously discover robust pregrasp manipulation. Th…
Rochester Institute of Technology（罗切斯特理工学院）；OSARO Inc（OSARO公司）；Siemens Corporation, Corporate Technology（西门子公司企业技术部）
In this work, we focus on multi-step manipulation tasks that involve long-horizon planning and considers progress reversal. Such tasks interlace high-level reasoning that consists of the expected states that can be attained to achieve an overall task and low-level reasoning that decides what actions will yield these states. We propose a sample efficient Previous Action Conditioned Robotic Manipulation Network (PAC-RoManNet) to learn the action-value functions and predict manipulation action candidates from visual observation of the scene and action-value predictions of the previous action. We …
Engineering school UPSSITECH（UPSSITECH 工程学院）；University of Toulouse III（图卢兹第三大学）；IRIT, Paul Sabatier University（保罗·萨巴捷大学 IRIT 研究所）；CNRS（法国国家科学研究中心）；School of Computer Science（计算机科学学院）；University of Lincoln（林肯大学）
Training robot manipulation policies is a challenging and open problem in robotics and artificial intelligence. In this paper we propose a novel and compact state representation based on the rewards predicted from an image-based task success classifier. Our experiments, using the Pepper robot in simulation with two deep reinforcement learning algorithms on a grab-and-lift task, reveal that our proposed state representation can achieve up to 97% task success using our best policies.
We introduce a sample-efficient method for learning state-dependent stiffness control policies for dexterous manipulation. The ability to control stiffness facilitates safe and reliable manipulation by providing compliance and robustness to uncertainties. Most current reinforcement learning approaches to achieve robotic manipulation have exclusively focused on position control, often due to the difficulty of learning high-dimensional stiffness control policies. This difficulty can be partially mitigated via policy guidance such as imitation learning. However, expert stiffness control demonstra…
OpenViTac: Learning and Benchmarking Visuo-Tactile Policies in a Unified Sim-and-Real Framework
Fudan University（复旦大学）；Shanghai Key Laboratory of Multimodal Embodied AI（上海多模态具身智能重点实验室）；Hefei University of Technology（合肥工业大学）；National University of Singapore（新加坡国立大学）；Neote AI；China Unicom（中国联通）
Learning complex manipulation tasks in realistic, obstructed environments is a challenging problem due to hard exploration in the presence of obstacles and high-dimensional visual observations. Prior work tackles the exploration problem by integrating motion planning and reinforcement learning. However, the motion planner augmented policy requires access to state information, which is often not available in the real-world settings. To this end, we propose to distill a state-based motion planner augmented policy to a visual control policy via (1) visual behavioral cloning to remove the motion p…
Robot manipulation is increasingly poised to interact with humans in co-shared workspaces. Despite increasingly robust manipulation and control algorithms, failure modes continue to exist whenever models do not capture the dynamics of the unstructured environment. To obtain longer-term horizons in robot automation, robots must develop introspection and recovery abilities. We contribute a set of recovery policies to deal with anomalies produced by external disturbances as well as anomaly classification through the use of non-parametric statistics with memoized variational inference with scalabl…
PACT: Self-Evolving Physical Safety Alignment for Diffusion Policies in Embodied Manipulation
Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China（计算机科学与技术系，人工智能研究院，清华-博世联合机器学习中心，THBI实验室，BNRist中心，清华大学，北京，100084，中国）；Peng Cheng Laboratory, 518108, China（鹏城实验室，518108，中国）
Hand-in-the-Loop: Improving VLA Policies for Dexterous Manipulation via Seamless Hand-Arm Intervention
State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University（机械系统与振动国家重点实验室，机械工程学院，上海交通大学）；Shanghai Key Laboratory of Intelligent Robotics, Meta Robotics Institute, Shanghai Jiao Tong University, Shanghai 200240, China（智能机器人上海市重点实验室，元机器人研究院，上海交通大学，上海200240，中国）；The University of Tokyo（东京大学）
