Object segmentation is a key component in the visual system of a robot that performs tasks like grasping and object manipulation, especially in presence of occlusions. Like many other computer vision tasks, the adoption of deep architectures has made available algorithms that perform this task with remarkable performance. However, adoption of such algorithms in robotics is hampered by the fact that training requires large amount of computing time and it cannot be performed on-line. In this work, we propose a novel architecture for object segmentation, that overcomes this problem and provides comparable performance in a fraction of the time required by the state-of-the-art methods. Our approach is based on a pre-trained Mask R-CNN, in which various layers have been replaced with a set of classifiers and regressors that are re-trained for a new task. We employ an efficient Kernel-based method that allows for fast training on large scale problems. Our approach is validated on the YCB-Video dataset which is widely adopted in the computer vision and robotics community, demonstrating that we can achieve and even surpass performance of the state-of-the-art, with a significant reduction (${\sim}6\times$) of the training time. The code to reproduce the experiments is publicly available on GitHub.
Semantic Scene Segmentation for Robotics Applications
Aristotle University of Thessaloniki(塞萨洛尼基亚里士多德大学)
语义场景分割在广泛的机器人应用中起着关键作用,例如,自主导航这些应用都伴随着特定的计算限制,例如,在低功耗GPU上以足够的速度运行,并且还用于高分辨率输入。现有的最先进的分割模型提供了不同设置下的评估结果,主要考虑高功率GPU。在本文中,我们研究了最成功的语义场景分割模型的行为,在部署(推理)速度方面,在各种设置(GPU,输入大小等)。in the context上下文of robotics机器人applications应用.这项工作的目标是提供一个比较研究,目前国家的最先进的分割模型,以选择最符合机器人应用程序的要求。
Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London(伦敦大学学院威康/EPSRC介入与外科科学中心);Department of Computer Science, University College London(伦敦大学学院计算机科学系)
Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video
The Chinese University of Hong Kong(香港中文大学);Shun Hing Institute of Advanced Engineering, CUHK(香港中文大学晨兴先进工程研究所);T Stone Robotics Institute, CUHK(香港中文大学T Stone机器人研究所)
Intuitive Inc.(直觉外科公司);Konika Minola Inc.(柯尼卡美能达公司);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心);Digital Surgery Ltd.(数字外科有限公司);Norwegian University of Science and Technology(挪威科技大学);Indian Institute of Technology Madras(印度理工学院马德拉斯分校);OTH Regensburg(雷根斯堡应用技术大学);Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) UCL(伦敦大学学院威康/EPSRC介入与外科科学中心);IRCAD;National Taiwan University(国立台湾大学);Yale University(耶鲁大学);Johns Hopkins University(约翰斯·霍普金斯大学);Rediminds Inc.(Rediminds公司);German Cancer Research Center (DKFZ)(德国癌症研究中心);Daegu Gyeongbuk Institute of Science and Technology(大邱庆北科学技术院)
Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations
The Chinese University of Hong Kong(香港中文大学);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University College London(伦敦大学学院);Zhejiang Lab(之江实验室)
ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools
University College London(伦敦大学学院);Katholieke Universiteit Leuven(天主教鲁汶大学);University College London Hospitals(伦敦大学学院医院);Universitair Ziekenhuis Leuven(鲁汶大学医院);Wellcome / EPSRC Centre for Surgical and Interventional Sciences, UCL, London, UK(惠康/英国工程与物理科学研究委员会外科与介入科学中心(伦敦大学学院));Surgical Robot Vision Group, CMIC, Dept. Computer Science, University College London(伦敦大学学院计算机科学系CMIC外科机器人视觉小组);Translational Imaging Group, CMIC, Dept. Med. Phys. Biomed. Eng., University College London(伦敦大学学院医学物理与生物医学工程系CMIC转化成像小组)
Attention-Guided Lightweight Network for Real-Time Segmentation of Robotic Surgical Instruments
State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所复杂系统管理与控制国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);CAS Center for Excellence in Brain Science and Intelligence Technology(中国科学院脑科学与智能技术卓越创新中心)
TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery
The Chinese University of Hong Kong(香港中文大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University College London(伦敦大学学院)
Efficient Global-Local Memory for Real-time Instrument Segmentation of Robotic Surgical Video
Xiamen University(厦门大学);The Chinese University of Hong Kong(香港中文大学);Zhongshan Hospital affiliated to Xiamen University(厦门大学附属中山医院);The Hong Kong Polytechnic University(香港理工大学)
One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video
The Chinese University of Hong Kong(香港中文大学);T stone Robotics Institute(T石头机器人研究所);Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems(粤港澳人机协同智能系统联合实验室);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
LiPS: Lightweight Panoptic Segmentation for Resource-Constrained Robotics
Université Paris-Saclay, CEA LIST(巴黎-萨克雷大学,CEA LIST);U2IS, ENSTA, Institut Polytechnique de Paris(U2IS、ENSTA、巴黎理工学院);University of Bonn, Center for Robotics(波恩大学,机器人中心)
Real-Time Instrument Segmentation in Robotic Surgery using Auxiliary Supervised Deep Adversarial Learning
NUS Graduate School for Integrative Sciences and Engineering (NGS)(新加坡国立大学综合科学与工程研究生院(NGS));National University of Singapore(新加坡国立大学);University of Moratuwa(莫拉图瓦大学);NIT Trichy(印度国立理工学院蒂鲁吉拉伯利分校)
Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications
Humanoid Robots Lab, University of Bonn(波恩大学仿人机器人实验室);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习和人工智能研究所);Center for Robotics, Bonn, Germany(波恩机器人中心)
More than Segmentation: Benchmarking SAM 3 for Segmentation, 3D Perception, and Reconstruction in Robotic Surgery
The Chinese University of Hong Kong(香港中文大学);The Hong Kong University of Science and Technology(香港科学与技术大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Technical University of Munich(慕尼黑技术大学)
Intuitive Surgical Inc.(直觉外科公司);Massachusetts Institute of Technology(麻省理工学院);University of Bern(伯尔尼大学);University of Alberta(阿尔伯塔大学);Georgia Institute of Technology(佐治亚理工学院);University of Washington(华盛顿大学);NVIDIA GmbH.(英伟达有限公司);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心);Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS) UCL(伦敦大学学院韦尔科姆/工程和物理科学研究委员会介入与外科科学中心);Lyft Inc.(来福车公司);Shenzhen Institute of Advanced Technology(深圳先进技术研究院);Beijing Institute of Technology(北京理工大学);Technical University of Munich(慕尼黑工业大学);German Cancer Research Center(德国癌症研究中心);Kings College London(伦敦国王学院)
Technical Report for ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Leveraging DINOv3 for Robust Outdoor Scene Understanding in Field Robotics
Daegu Gyeongbuk Institute of Science and Technology (DGIST)(大邱庆北科学技术院);Massachusetts Institute of Technology (MIT)(麻省理工学院)
Bimodal SegNet: Instance Segmentation Fusing Events and RGB Frames for Robotic Grasping
Faculty of Science, Engineering and Computing, Kingston University(科学、工程与计算学院,金斯顿大学);Aerospace Engineering Department, Khalifa University of Science and Technology(航空航天工程系,哈利法大学科技学院);Dubai Future Labs - RLab(迪拜未来实验室-RLab)