3D meshes are a fundamental representation widely used in computer science and engineering. In robotics, they are particularly valuable because they capture objects in a form that aligns directly with how robots interact with the physical world, enabling core capabilities such as predicting stable grasps, detecting collisions, and simulating dynamics. Although automatic 3D mesh generation methods have shown promising progress in recent years, potentially offering a path toward real-time robot perception, two critical challenges remain. First, generating high-fidelity meshes is prohibitively slow for real-time use, often requiring tens of seconds per object. Second, mesh generation by itself is insufficient. In robotics, a mesh must be contextually grounded, i.e., correctly segmented from the scene and registered with the proper scale and pose. Additionally, unless these contextual grounding steps remain efficient, they simply introduce new bottlenecks. In this work, we introduce an end-to-end system that addresses these challenges, producing a high-quality, contextually grounded 3D mesh from a single RGB-D image in under one second. Our pipeline integrates open-vocabulary object segmentation, accelerated diffusion-based mesh generation, and robust point cloud registration, each optimized for both speed and accuracy. We demonstrate its effectiveness in a real-world manipulation task, showing that it enables meshes to be used as a practical, on-demand representation for robotics perception and planning.
RL-GSBridge: 3D Gaussian Splatting Based Real2Sim2Real Method for Robotic Manipulation Learning
Shenzhen Research Institute of Shanghai Jiao Tong University(上海交通大学深圳研究院);Shanghai Jiao Tong University(上海交通大学);School of Information and Control Engineering, China University of Mining and Technology(中国矿业大学信息与控制工程学院);University of Cambridge(剑桥大学)
Realtime Simulation of Thin-Shell Deformable Materials using CNN-Based Mesh Embedding
Department of Computer Science and Electrical & Computer Engineering, University of Maryland at College Park(马里兰大学学院公园分校计算机科学与电子与计算机工程系);Department of Computer Science, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校计算机科学系);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning with Gaussian Splatting
Wuhan University(武汉大学);DAMO Academy, Alibaba Group(达摩院,阿里巴巴集团);Hupan Lab(虎扑实验室);The Chinese University of Hong Kong(香港中文大学);Tsinghua University(清华大学);Huazhong University of Science and Technology(华中科技大学);Zhejiang University(浙江大学)
Is Single-View Mesh Reconstruction Ready for Robotics?
Oxford Robotics Institute, University of Oxford(牛津大学机器人研究所);University of Tübingen(图宾根大学);Max Planck Institute for Intelligent Systems & ELLIS Institute Tübingen(马克斯·普朗克智能系统研究所及图宾根ELLIS研究所)
Benchmarking the Effects of Object Pose Estimation and Reconstruction on Robotic Grasping Success
Faculty of Electrical Engineering, Czech Technical University in Prague(捷克技术大学布拉格电子工程系);Czech Institute of Informatics, Robotics and Cybernetics(捷克信息学、机器人学与自动化研究所)
ReJSHand: Efficient Real-Time Hand Pose Estimation and Mesh Reconstruction Using Refined Joint and Skeleton Features
School of Electrical and Information Engineering, Tianjin University(天津大学电气与信息工程学院);College of Sciences, Northeastern University(东北大学理学院);College of Computer Science, Beijing University of Technology(北京工业大学计算机学院);Democritus University of Thrace(色雷斯德谟克利特大学);School of Mechanical Engineering, Tongji University(同济大学机械工程学院);Department of Electronic and Electrical Engineering, Southern University of Science and Technology(南方科技大学电子与电气工程系)
FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulation
Mohamed bin Zayed University of Artificial Intelligence(莫扎德·扎耶德人工智能大学);Technical University of Munich(慕尼黑技术大学);National University of Singapore(新加坡国立大学);Westlake University(西湖大学)
RoboPCA: Pose-centered Affordance Learning from Human Demonstrations for Robot Manipulation
Key Laboratory of AI Safety of CAS, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室、计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学)
GARField: Addressing the visual Sim-to-Real gap in garment manipulation with mesh-attached radiance fields
University of Genova(热那亚大学);Istituto Italiano di Tecnologia(意大利理工学院);Department of Mechanical and Automation Engineering, T- Stone Robotics Institute, The Chinese University of Hong Kong(机械与自动化工程系,T- Stone机器人研究所,香港中文大学)
6D Object Pose Tracking in Internet Videos for Robotic Manipulation
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague(布拉格捷克技术大学捷克信息学、机器人学与控制论研究所);Faculty of Electrical Engineering, Czech Technical University in Prague(布拉格捷克技术大学电气工程学院);H Company(H公司)
Can Single-View Mesh Reconstruction Generalize to Robot Camera Rotation?
Shenzhen Key Laboratory of Robotics and Computer Vision, Southern University of Science and Technology (SUSTech)(南方科技大学深圳市机器人视觉与感知重点实验室);Pengcheng Laboratory(鹏城实验室)
VidAct: Learning Manipulation from In-the-Wild Videos with Object-Centric 3D Awareness
Technical University of Munich(慕尼黑工业大学);Peking University(北京大学);Huawei Heisenberg Research Center(华为海森堡研究中心);Huawei CloudRobo Lab(华为云机器人实验室);Nanjing University(南京大学)