Robotic Manipulation is Vision-to-Geometry Mapping: Vision-Geometry Backbones over Language and Video Models
Sun Yat-sen University(中山大学);Guangdong Key Laboratory of Big Data Analysis(大数据分析与处理广东省重点实验室);X-Era AI Lab(X-Era人工智能实验室);AMAP, Alibaba(阿里妈妈实验室);Guangdong University of Technology(广东工业大学)
At its core, robotic manipulation is a problem of vision-to-geometry mapping ($f(v) \rightarrow G$). Physical actions are fundamentally defined by geometric properties like 3D positions and spatial relationships. Consequently, we argue that the foundation for generalizable robotic control should be a vision-geometry backbone, rather than the widely adopted vision-language or video models. Conventional VLA and video-predictive models rely on backbones pretrained on large-scale 2D image-text or temporal pixel data. While effective, their representations are largely shaped by semantic concepts or 2D priors, which do not intrinsically align with the precise 3D geometric nature required for physical manipulation. Driven by this insight, we propose the Vision-Geometry-Action (VGA) model, which directly conditions action generation on pretrained 3D representations. Specifically, VGA replaces conventional language or video backbones with a pretrained 3D world model, establishing a seamless vision-to-geometry mapping that translates visual inputs directly into physical actions. To further enhance geometric consistency, we introduce Progressive Volumetric Modulation and jointly train action and 3D property prediction to preserve geometric representations. Extensive experiments validate the effectiveness of our approach. Across simulation benchmarks, VGA outperforms leading VLA, 3D-VLA, and WAM baselines, including $π_{0.5}$, OpenVLA-OFT, GeoVLA, and Motus. In real-world deployments, VGA surpasses $π_{0.5}$ under unseen viewpoints and accurately follows language instructions for target grasping. These results highlight that operating on native 3D representations, rather than relying primarily on language or video priors, offers a promising direction toward generalizable physical intelligence. Project page: https://hcplab-sysu.github.io/VisionGeometryActionModel.
RGBManip: Monocular Image-based Robotic Manipulation through Active Object Pose Estimation
Hyperplane Lab, School of CS, Peking University(北京大学计算机学院超平面实验室);National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程学系)
Learning Smooth and Robust Space Robotic Manipulation of Dynamic Target via Inter-frame Correlation
Research Center for Intelligent Robotics, School of Astronautics, Northwestern Polytechnical University(西北工业大学航天学院智能机器人研究中心);School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);School of Aeronautics and Astronautics, Sichuan University(四川大学空天科学与工程学院)
Benchmarking Simulated Robotic Manipulation through a Real World Dataset
Data61, Commonwealth Scientific and Industrial Research Organisation (CSIRO)(联邦科学与工业研究组织(CSIRO)Data61研究所);Queensland University of Technology (QUT)(昆士兰科技大学(QUT))
Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Review
ICube Lab Robotics Department Strasbourg University UMR 7357 CNRS(斯特拉斯堡大学ICube实验室机器人系 UMR 7357 CNRS);ICube Lab UMR 7357 Centre National de la Recherche Scientifique CNRS(ICube实验室 UMR 7357 法国国家科学研究中心)
Research Challenges and Progress in Robotic Grasping and Manipulation Competitions
University of South Florida(南佛罗里达大学);National Institute of Standards and Technology (NIST)(美国国家标准与技术研究院);German Aerospace Center (DLR)(德国航空航天中心);Worcester Polytechnic Institute (WPI)(伍斯特理工学院)
Challenges and Outlook in Robotic Manipulation of Deformable Objects
Cognitive Robotics, TU Delft(代尔夫特理工大学认知机器人);Honda Research Institute, Europe(本田研究院欧洲);LIRMM - Université de Montpellier CNRS(蒙彼利埃大学 CNRS LIRMM);COSMER laboratory EA 7398, Universié de Toulon(土伦大学 COSMER 实验室 EA 7398);The Hong Kong Polytechnic University(香港理工大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Michigan(密歇根大学);Università degli Studi di Napoli Federico II(那不勒斯费德里科二世大学);Osaka University(大阪大学);National Institute of AIST(产业技术综合研究所);Tsinghua University(清华大学);The University of Hong Kong(香港大学);Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所)
DORA: Object Affordance-Guided Reinforcement Learning for Dexterous Robotic Manipulation
TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, Universität Hamburg(技术多模态系统(TAMS),信息学院,汉堡大学);Agile Robots AG(敏捷机器人公司);Technical University of Munich(慕尼黑技术大学)