We consider the challenging problem of zero-shot video object segmentation (VOS). That is, segmenting and tracking multiple moving objects within a video fully automatically, without any manual initialization. We treat this as a grouping problem by exploiting object proposals and making a joint inference about grouping over both space and time. We propose a network architecture for tractably performing proposal selection and joint grouping. Crucially, we then show how to train this network with reinforcement learning so that it learns to perform the optimal non-myopic sequence of grouping decisions to segment the whole video. Unlike standard supervised techniques, this also enables us to directly optimize for the non-differentiable overlap-based metrics used to evaluate VOS. We show that the proposed method, which we call ALBA outperforms the previous stateof-the-art on three benchmarks: DAVIS 2017 [2], FBMS [20] and Youtube-VOS [27].
Imperial College London（伦敦帝国学院）；University College London（伦敦大学学院）；Samsung AI Center Cambridge（三星剑桥人工智能中心）
For real-time semantic video segmentation, most recent works utilised a dynamic framework with a key scheduler to make online key/non-key decisions. Some works used a fixed key scheduling policy, while others proposed adaptive key scheduling methods based on heuristic strategies, both of which may lead to suboptimal global performance. To overcome this limitation, we model the online key decision process in dynamic video segmentation as a deep reinforcement learning problem and learn an efficient and effective scheduling policy from expert information about decision history and from the proces…
Deep neural network (DNN) based approaches have been widely investigated and deployed in medical image analysis. For example, fully convolutional neural networks (FCN) achieve the state-of-the-art performance in several applications of 2D/3D medical image segmentation. Even the baseline neural network models (U-Net, V-Net, etc.) have been proven to be very effective and efficient when the training process is set up properly. Nevertheless, to fully exploit the potentials of neural networks, we propose an automated searching approach for the optimal training strategy with reinforcement learning.…
Recognition of surgical gesture is crucial for surgical skill assessment and efficient surgery training. Prior works on this task are based on either variant graphical models such as HMMs and CRFs, or deep learning models such as Recurrent Neural Networks and Temporal Convolutional Networks. Most of the current approaches usually suffer from over-segmentation and therefore low segment-level edit scores. In contrast, we present an essentially different methodology by modeling the task as a sequential decision-making process. An intelligent agent is trained using reinforcement learning with hier…
This paper proposes an automatic image co-segmentation algorithm based on deep reinforcement learning (RL). Existing co-segmentation tasks mainly rely on deep learning methods, and the obtained foreground edges are often rough. In order to obtain more precise foreground edges, we use deep RL to solve this problem and achieve the finer segmentation. To our best knowledge, this is the first work to apply RL methods to co-segmentation. We define the problem as a Markov Decision Process (MDP) and optimize it by RL with asynchronous advantage actor-critic (A3C). The RL image co-segmentation network…
Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）
Existing automatic 3D image segmentation methods usually fail to meet the clinic use. Many studies have explored an interactive strategy to improve the image segmentation performance by iteratively incorporating user hints. However, the dynamic process for successive interactions is largely ignored. We here propose to model the dynamic process of iterative interactive image segmentation as a Markov decision process (MDP) and solve it with reinforcement learning (RL). Unfortunately, it is intractable to use single-agent RL for voxel-wise prediction due to the large exploration space. To reduce …
Xi’an Jiaotong-Liverpool University（西交利物浦大学）；National University of Singapore（新加坡国立大学）
In this paper, we aim to tackle the task of semi-supervised video object segmentation across a sequence of frames where only the ground-truth segmentation of the first frame is provided. The challenges lie in how to online update the segmentation model initialized from the first frame adaptively and accurately, even in presence of multiple confusing instances or large object motion. The existing approaches rely on selecting the region of interest for model update, which however, is rough and inflexible, leading to performance degradation. To overcome this limitation, we propose a novel approac…
Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland University（萨尔兰大学）
Segmentation of nanoscale electron microscopy (EM) images is crucial but still challenging in connectomics research. One reason for this is that none of the existing segmentation methods are error-free, so they require proofreading, which is typically implemented as an interactive, semi-automatic process via manual intervention. Herein, we propose a fully automatic proofreading method based on reinforcement learning that mimics the human decision process of detection, classification, and correction of segmentation errors. We systematically design the proposed system by combining multiple reinf…
Nanjing University（南京大学）；Medical School of Nanjing University（南京大学医学院）；National Institute of Healthcare Data Science, Nanjing University（南京大学国家健康数据科学研究院）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）
Conventional data augmentation realized by performing simple pre-processing operations (\eg, rotation, crop, \etc) has been validated for its advantage in enhancing the performance for medical image segmentation. However, the data generated by these conventional augmentation methods are random and sometimes harmful to the subsequent segmentation. In this paper, we developed a novel automatic learning-based data augmentation method for medical image segmentation which models the augmentation task as a trial-and-error procedure using deep reinforcement learning (DRL). In our method, we innovativ…
VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation
School of Automation, Southeast University, Nanjing, China；Baidu Inc, Beijing, China；College of Information Science \& Electronic Engineering, Zhejiang University, Hangzhou, China
Future advancements in robot autonomy and sophistication of robotics tasks rest on robust, efficient, and task-dependent semantic understanding of the environment. Semantic segmentation is the problem of simultaneous segmentation and categorization of a partition of sensory data. The majority of current approaches tackle this using multi-class segmentation and labeling in a Conditional Random Field (CRF) framework or by generating multiple object hypotheses and combining them sequentially. In practical settings, the subset of semantic labels that are needed depend on the task and particular sc…
FineRS: Fine-grained Reasoning and Segmentation of Small Objects with Reinforcement Learning
Dalian University of Technology（大连理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际graduate school）
SPARS: Self-Play Adversarial Reinforcement Learning for Segmentation of Liver Tumours
Department of Medical Physics and Biomedical Engineering, University College London（医学物理与生物医学工程系，伦敦大学学院）；UCL Hawkes Institute, University College London（UCL哈维斯研究所，伦敦大学学院）
Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Tongji University（同济大学）
Interactive segmentation has recently been explored to effectively and efficiently harvest high-quality segmentation masks by iteratively incorporating user hints. While iterative in nature, most existing interactive segmentation methods tend to ignore the dynamics of successive interactions and take each interaction independently. We here propose to model iterative interactive image segmentation with a Markov decision process (MDP) and solve it with reinforcement learning (RL) where each voxel is treated as an agent. Considering the large exploration space for voxel-wise prediction and the de…
Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate Segmentation in TRUS Images
University College London（伦敦大学学院）；UCL Hospital NHS Foundation Trust（伦敦大学学院医院NHS基金会信托）；Wellcome EPSRC Centre for Interventional and Surgical Sciences（惠康EPSRC介入与外科科学中心）；Centre for Medical Image Computing（医学影像计算中心）；Department of Medical Physics and Biomedical Engineering（医学物理与生物医学工程系）；Division of Surgery & Interventional Science（外科与介入科学部）；Department of Urology（泌尿外科）
XJTLU（西交利物浦大学）；University of Liverpool（利物浦大学）；Beijing Jiaotong University（北京交通大学）
In this paper, the main task we aim to tackle is the multi-instance semi-supervised video object segmentation across a sequence of frames where only the first-frame box-level ground-truth is provided. Detection-based algorithms are widely adopted to handle this task, and the challenges lie in the selection of the matching method to predict the result as well as to decide whether to update the target template using the newly predicted result. The existing methods, however, make these selections in a rough and inflexible way, compromising their performance. To overcome this limitation, we propos…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Combinatorial optimization is frequently used in computer vision. For instance, in applications like semantic segmentation, human pose estimation and action recognition, programs are formulated for solving inference in Conditional Random Fields (CRFs) to produce a structured output that is consistent with visual features of the image. However, solving inference in CRFs is in general intractable, and approximation methods are computationally demanding and limited to unary, pairwise and hand-crafted forms of higher order potentials. In this paper, we show that we can learn program heuristics, i.…
Beijing University of Posts and Telecommunications（北京邮电大学）
Medical image segmentation is one of the important tasks of computer-aided diagnosis in medical image analysis. Since most medical images have the characteristics of blurred boundaries and uneven intensity distribution, through existing segmentation methods, the discontinuity within the target area and the discontinuity of the target boundary are likely to lead to rough or even erroneous boundary delineation. In this paper, we propose a new iterative refined interactive segmentation method for medical images based on agent reinforcement learning, which focuses on the problem of target segmenta…
Image segmentation needs both local boundary position information and global object context information. The performance of the recent state-of-the-art method, fully convolutional networks, reaches a bottleneck due to the neural network limit after balancing between the two types of information simultaneously in an end-to-end training style. To overcome this problem, we divide the semantic image segmentation into temporal subtasks. First, we find a possible pixel position of some object boundary; then trace the boundary at steps within a limited length until the whole object is outlined. We pr…
Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
We present a new technique for deep reinforcement learning that automatically detects moving objects and uses the relevant information for action selection. The detection of moving objects is done in an unsupervised way by exploiting structure from motion. Instead of directly learning a policy from raw images, the agent first learns to detect and segment moving objects by exploiting flow information in video sequences. The learned representation is then used to focus the policy of the agent on the moving objects. Over time, the agent identifies which objects are critical for decision making an…
Edinburgh Napier University（爱丁堡龙比亚大学）；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Several imaging applications (vessels, retina, plant roots, road networks from satellites) require the accurate segmentation of thin structures for subsequent analysis. Discontinuities (gaps) in the extracted foreground may hinder down-stream image-based analysis of biomarkers, organ structure and topology. In this paper, we propose a general post-processing technique to recover such gaps in large-scale segmentation masks. We cast this problem as a blind inpainting task, where the regions of missing lines in the segmentation masks are not known to the algorithm, which we solve with an adversar…
Reinforcement Learning for Unsupervised Domain Adaptation in Spatio-Temporal Echocardiography Segmentation
Department of Computer Science, University of Sherbrooke（谢布鲁克大学计算机科学系）；INSA, Universite Claude Bernard Lyon 1, CNRS UMR 5220, Inserm U1206, CREATIS（里昂1大学INSA、CNRS UMR 5220、Inserm U1206、CREATIS）；Dep. of Software and Information Technology Engineering, École de technologie supérieure（蒙特利尔工程学院软件与信息技术工程系）；Institut Universitaire de France (IUF)（法国国家科学院（IUF））
Amodal recognition is the ability of the system to detect occluded objects. Most SOTA Visual Recognition systems lack the ability to perform amodal recognition. Few studies have achieved amodal recognition through passive prediction or embodied recognition approaches. However, these approaches suffer from challenges in real-world applications, such as dynamic obstacles. We propose SeekNet, an improved optimization method for amodal recognition through embodied visual recognition. Additionally, we implement SeekNet for social robots, where there are multiple interactions with crowded pedestrian…
RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning
Division of Hematology, The Ohio State University Comprehensive Cancer Center（血液科，俄亥俄州立大学综合癌症中心）；Department of Radiology, Mahidol University（医学放射科，玛希多大学）；Department of Radiology, Mettapracharak Hospital（医学放射科，Mettapracharak医院）；Department of Radiation Oncology, The Ohio State University Comprehensive Cancer Center（放射肿瘤科，俄亥俄州立大学综合癌症中心）
MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning
Chinese University of Hong Kong, Hong Kong SAR, China（香港中文大学）；Hunyuan Group, Tencent（腾讯洪音集团）；Institute of Automation, the Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所）；Dalian University of Technology, Dalian, China（大连理工大学）；Stanford University, Stanford, USA（斯坦福大学）
Salient object segmentation aims at distinguishing various salient objects from backgrounds. Despite the lack of semantic consistency, salient objects often have obvious texture and location characteristics in local area. Based on this priori, we propose a novel Local Context Attention Network (LCANet) to generate locally reinforcement feature maps in a uniform representational architecture. The proposed network introduces an Attentional Correlation Filter (ACF) module to generate explicit local attention by calculating the correlation feature map between coarse prediction and global context. …
In this paper, we present a state-of-the-art reinforcement learning method for autonomous driving. Our approach employs temporal difference learning in a Bayesian framework to learn vehicle control signals from sensor data. The agent has access to images from a forward facing camera, which are preprocessed to generate semantic segmentation maps. We trained our system using both ground truth and estimated semantic segmentation input. Based on our observations from a large set of experiments, we conclude that training the system on ground truth input data leads to better performance than trainin…
Synergistic Vision-Language Reinforcement Enables Scalable On-Demand Analysis across Diverse Clinical Tasks
Department of Electronic and Computer Engineering, HKUST（香港科技大学电子与计算机工程系）；Department of Radiology, Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University（中山大学放射科、广东省恶性肿瘤表观遗传与基因调控重点实验室、中山纪念医院）；Department of Computer Science and Engineering, HKUST（香港科技大学计算机科学与工程系）
