TiPToP: A Modular Open-Vocabulary Robot Manipulation System That Plans
William Shen, Nishanth Kumar, Sahit Chintalapudi, Ryan Lindeborg, Jie Wang, Christopher Watson, Edward Hu, Jing Cao, Dinesh Jayaraman, Leslie Pack Kaelbling, Tomás Lozano-Pérez
We present TiPToP, a modular manipulation system that integrates pretrained foundation models with a GPU-accelerated Task and Motion Planner to solve tasks directly from RGB images and natural language. TiPToP composes perception, planning, and execution modules and requires no robot training data. It can be deployed on a standard DROID setup in under an hour and adapted to new embodiments with minimal effort. We evaluate TiPToP against $π_{0.5}\text{-DROID}$, a state-of-the-art VLA fine-tuned on 350 hours of demonstrations, across two real-world DROID setups (one operated by an external team) and simulation, where TiPToP attains a higher average success rate and faster average completion time. We also evaluate on the MolmoSpaces benchmark, where TiPToP ranks first overall on pick and pick-and-place tasks among methods not trained on in-distribution data. We further show that TiPToP's modularity enables us to trace failures to specific components, revealing where to target improvements. We release TiPToP open-source to serve as a reproducible baseline and to enable further research on modular manipulation systems. Project website and code: https://tiptop-robot.github.io
Reflective VLM Planning for Dual-Arm Desktop Cleaning: Bridging Open-Vocabulary Perception and Precise Manipulation
Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所);Department of Electrical and Computer Engineering, Carnegie Mellon University(卡内基梅隆大学电气与计算机工程系);College of Aeronautics & Engineering, Kent State University(肯特州立大学航空与工程学院)
Open-Vocabulary Spatio-Temporal Scene Graph for Robot Perception and Teleoperation Planning
HI-Robot Lab, School of Automation Science and Engineering, South China University of Technology(华南理工大学自动化科学与工程学院);Institute of AI Industries, Chinese Academy of Science(中国科学院人工智能产业研究所);School of Computer Science, Central China Normal University(中央财经大学计算机科学学院);School of Computer Science and Informatics, University of Liverpool(利物浦大学计算机科学与信息学院)
Uncertainty-Informed Active Perception for Open Vocabulary Object Goal Navigation
Center for Robotics, University of Bonn(波恩大学机器人中心);MAVLab, TU Delft(代尔夫特理工大学MAVLab);Lamarr Institute for Machine Learning and Artificial Intelligence(机器学习与人工智能拉马尔研究所)
FindAnything: Open-Vocabulary and Object-Centric Mapping for Robot Exploration in Any Environment
Technical University of Munich(技术大学慕尼黑);School of Computation, Information and Technology(计算、信息与技术学院);ETH Zurich(苏黎世联邦理工学院);Department of Computing(计算学院);Imperial College London(伦敦帝国理工学院);University of Patras(帕特拉大学);Munich Institute of Robotics and Machine Intelligence (MIRMI)(慕尼黑机器人与机器智能研究所);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心)
PerceptTwin: Semantic Scene Reconstruction for Iterative LLM Planning and Verification
Department of Computer Science and Operations Research, Université de Montréal(蒙特利尔大学计算机科学与运筹学系);Mila - Quebec AI Institute(魁北克人工智能研究所);CIFAR AI Chair(CIFAR人工智能主席)
OpenObject-NAV: Open-Vocabulary Object-Oriented Navigation Based on Dynamic Carrier-Relationship Scene Graph
School of Automation, Beijing Institute of Technology(北京理工大学自动化学院);School of Mechanical Engineering, University of Nottingham Ningbo China(宁波诺丁汉大学机械工程学院)
Jet Propulsion Laboratory(喷气推进实验室);California Institute of Technology(加州理工学院);Swiss Federal Institute of Technology(瑞士联邦理工学院);ETH Zürich(苏黎世联邦理工学院);FieldAI Inc.(FieldAI公司);Stanford University(斯坦福大学);University of California Berkeley(加州大学伯克利分校)
Shaken, Not Stirred: A Novel Dataset for Visual Understanding of Glasses in Human-Robot Bartending Tasks
Department of Applied Informatics, Faculty of Mathematics, Physics and Informatics, Comenius University(应用信息学院,数学物理学院,科门ius大学);Knowledge Technology Group, Department of Informatics, University of Hamburg(知识技术组,信息学院,汉堡大学)