Autonomously performing tasks often requires robots to plan high-level discrete actions and continuous low-level motions to realize them. Previous TAMP algorithms have focused mainly on computational performance, completeness, or optimality by making the problem tractable through simplifications and abstractions. However, this comes at the cost of the resulting plans potentially failing to account for the dynamics or complex contacts necessary to reliably perform the task when object manipulation is required. Additionally, approaches that ignore effects of the low-level controllers may not obtain optimal or feasible plan realizations for the real system. We investigate the use of a GPU-parallelized physics simulator to compute realizations of plans with motion controllers, explicitly accounting for dynamics, and considering contacts with the environment. Using cross-entropy optimization, we sample the parameters of the controllers, or actions, to obtain low-cost solutions. Since our approach uses the same controllers as the real system, the robot can directly execute the computed plans. We demonstrate our approach for a set of tasks where the robot is able to exploit the environment's geometry to move an object. Website and code: https://andreumatoses.github.io/research/parallel-realization
FATE: Closed-Loop Feasibility-Aware Task Generation with Active Repair for Physically Grounded Robotic Curricula
School of Aerospace Engineering, Tsinghua University, Beijing, China(航空航天工程系,清华大学,北京,中国);Department of Automation, Tsinghua University, Beijing, China(自动化系,清华大学,北京,中国);School of Integrated Circuits, Tsinghua University, Beijing, China(集成电路学院,清华大学,北京,中国);State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China(通用人工智能国家重点实验室,北京通用人工智能研究院(BIGAI),北京,中国)
Beyond Shallow-Water Photorealism: Physically and Sensor-Grounded Simulation for Deep-Sea Robotics
School of Engineering & Physical Sciences, Heriot-Watt University(赫瑞瓦特大学工程与物理科学学院);Engineering Department, Japan Agency for Marine-Earth Science and Technology(日本海洋研究开发机构工程系)
OmniUMI: Towards Physically Grounded Robot Learning via Human-Aligned Multimodal Interaction
Beijing Academy of Artificial Intelligence(北京人工智能研究院);MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences(自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing Institute of Technology(北京理工大学);Beijing University of Posts and Telecommunications(北京邮电大学);Peking University(北京大学)
TemporalFlow-VLA: Learning Physically Grounded Execution History for Long-Horizon Robot Manipulation
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Zhejiang University(浙江大学);Simon Fraser University(西蒙菲莎大学);AgiBot(云圣智能科技(上海)有限公司)
SVLL: Staged Vision-Language Learning for Physically Grounded Embodied Task Planning
FNii-Shenzhen, CUHKSZ(FNii深圳,CUHKSZ);SSE, CUHKSZ(SSE,CUHKSZ);SAI, CUHKSZ(SAI,CUHKSZ);Shenzhen University(深圳大学);University of Sydney(悉尼大学);Harbin Engineering University(哈尔滨工程大学);Ising AI
ADP: Adversarial Dynamics Priors for Physically Grounded Humanoid Locomotion
Mechatronics, Systems and Control Lab (MSC Lab), Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院机械工程系机电一体化、系统与控制实验室(MSC实验室));Samsung Electronics, Future Robotics AI Group(三星电子未来机器人人工智能集团);School of Electrical Engineering, Hanyang University(汉阳大学电气工程学院);Advanced Robotics Research Center, Korea Institute of Machinery & Materials (KIMM)(韩国机械与材料研究所先进机器人研究中心)
Memory as Plans: World-Action Modeling with Memory-Grounded Planning
Harbin Institute of Technology(哈尔滨工业大学);Nanyang Technological University(南洋理工大学);Shandong University(山东大学);Harbin Institute of Technology (Weihai) Qingdao Research Institute(哈尔滨工业大学(威海)青岛研究院)
Learning to Reason over Scene Graphs: A Case Study of Finetuning GPT-2 into a Robot Language Model for Grounded Task Planning
Technische Universität Darmstadt(达姆施塔特工业大学);Center for Mind, Brain and Behavior, Uni. Marburg and JLU Giessen(马尔堡大学与吉森尤斯图斯·李比希大学心智、大脑与行为研究中心);Amazon Alexa(亚马逊Alexa)
LLM-Grounded Dynamic Task Planning with Hierarchical Temporal Logic for Human-Aware Multi-Robot Handover
The Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳人工智能与机器人社会研究院);Harbin Institute of Technology(哈尔滨工业大学);The Chinese University of Hong Kong-Shenzhen(香港中文大学(深圳));Tsinghua Shenzhen International Graduate School(清华大学深圳国际 Graduate School)