A key component of many robotics model-based planning and control algorithms is physics predictions, that is, forecasting a sequence of states given an initial state and a sequence of controls. This process is slow and a major computational bottleneck for robotics planning algorithms. Parallel-in-time integration methods can help to leverage parallel computing to accelerate physics predictions and thus planning. The Parareal algorithm iterates between a coarse serial integrator and a fine parallel integrator. A key challenge is to devise a coarse model that is computationally cheap but accurate enough for Parareal to converge quickly. Here, we investigate the use of a deep neural network physics model as a coarse model for Parareal in the context of robotic manipulation. In simulated experiments using the physics engine Mujoco as fine propagator we show that the learned coarse model leads to faster Parareal convergence than a coarse physics-based model. We further show that the learned coarse model allows to apply Parareal to scenarios with multiple objects, where the physics-based coarse model is not applicable. Finally, we conduct experiments on a real robot and show that Parareal predictions are close to real-world physics predictions for robotic pushing of multiple objects. Videos are at https://youtu.be/wCh2o1rf-gA.
从粗到细的Q-注意力通过以从粗到细的方式离散化平移空间来实现样本高效的机器人操作,其中分辨率在层次结构中的每一层逐渐增加。虽然有效,但Q-attention遭受“粗糙的模糊性”-当体素化非常粗糙时,如果不首先以更精细的分辨率进行检查,就无法区分外观相似的对象。为了解决这个问题,我们建议将Q-attention设想为一棵树,它可以扩展并用于在每个Q-attention深度处累积前k个体素的值估计。当我们的扩展,Q-attention with Tree Expansion(QTE),在注意力驱动的机器人操作(ARM)系统中取代标准的Q-attention时,我们能够完成更大的任务集;特别是那些遭受“粗糙模糊”的任务。除了在12个RLBench任务中评估我们的方法外,我们还表明,在涉及小对象的现实任务中,性能的提高是显而易见的。
Primary-Fine Decoupling for Action Generation in Robotic Imitation
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China(脑启发智能感知与认知国家重点实验室,中国科学技术大学);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(人工智能研究院,合肥综合性国家科学中心)
Dexterous Soft Hands Linearize Feedback-Control for In-Hand Manipulation
Robotics and Biology Laboratory, Technische Universität Berlin(柏林工业大学机器人与生物实验室);Science of Intelligence, Research Cluster of Excellence, Berlin(柏林卓越研究集群智能科学)
Move-Then-Operate: Behavioral Phasing for Human-Like Robotic Manipulation
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China.(杭州高等研究院,中国科学院大学,中国杭州);University of Science and Technology of China, Hefei, China.(中国科学技术大学,中国合肥);School of Aritificial intelligence, Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing, China(人工智能学院,人工智能研究院,北京科技大学,中国北京)
VERM: Leveraging Foundation Models to Create a Virtual Eye for Efficient 3D Robotic Manipulation
New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别新技术实验室,多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);FiveAges;Shandong University(山东大学)
Articulated Object Manipulation with Coarse-to-fine Affordance for Mitigating the Effect of Point Cloud Noise
Hyperplane Lab, School of CS, Peking University(北京大学计算机学院超平面实验室);National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室);Huawei(华为)
Vi-TacMan: Articulated Object Manipulation via Vision and Touch
Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);School of Psychological and Cognitive Sciences, Peking University(北京大学心理与认知科学学院);State Key Lab of General AI, Peking University(北京大学通用人工智能国家重点实验室);Beijing Key Laboratory of Behavior and Mental Health, Peking University(北京大学行为与心理健康北京市重点实验室);LeapZenith AI Research(跃臻人工智能研究院);Embodied Intelligence Lab, PKU-Wuhan Institute for Artificial Intelligence(北京大学武汉人工智能研究院具身智能实验室)
MPC-based Coarse-to-Fine Motion Planning for Robotic Object Transportation in Cluttered Environments
Department of Electrical and Computer Engineering, University of Kaiserslautern Landau(电气与计算机工程系,凯撒斯劳滕大学);Department of Mechanical Engineering, Dalhousie University(机械工程系,达尔豪斯大学)