High-quality training data play a key role in image segmentation tasks. Usually, pixel-level annotations are expensive, laborious and time-consuming for the large volume of training data. To reduce labelling cost and improve segmentation quality, interactive segmentation methods have been proposed, which provide the result with just a few clicks. However, their performance does not meet the requirements of practical segmentation tasks in terms of speed and accuracy. In this work, we propose EdgeFlow, a novel architecture that fully utilizes interactive information of user clicks with edge-guided flow. Our method achieves state-of-the-art performance without any post-processing or iterative optimization scheme. Comprehensive experiments on benchmarks also demonstrate the superiority of our method. In addition, with the proposed method, we develop an efficient interactive segmentation tool for practical data annotation tasks. The source code and tool is avaliable at https://github.com/PaddlePaddle/PaddleSeg.
Leipzig University（莱比锡大学）；IfADo - Leibniz Research Centre for Working Environment and Human Factors（莱布尼茨工作环境与人因研究中心）；INRIA Paris（法国国家信息与自动化研究所巴黎中心）；Sorbonne Université（索邦大学）
We present a novel approach that combines machine learning based interactive image segmentation using supervoxels with a clustering method for the automated identification of similarly colored images in large data sets which enables a guided reuse of classifiers. Our approach solves the problem of significant color variability prevalent and often unavoidable in biological and medical images which typically leads to deteriorated segmentation and quantification accuracy thereby greatly reducing the necessary training effort. This increase in efficiency facilitates the quantification of much larg…
University of Campinas（坎皮纳斯大学）；Université Gustave Eiffel（古斯塔夫·埃菲尔大学）
Despite the progress of interactive image segmentation methods, high-quality pixel-level annotation is still time-consuming and laborious - a bottleneck for several deep learning applications. We take a step back to propose interactive and simultaneous segment annotation from multiple images guided by feature space projection. This strategy is in stark contrast to existing interactive segmentation methodologies, which perform annotation in the image domain. We show that feature space annotation achieves competitive results with state-of-the-art methods in foreground segmentation datasets: iCoS…
Learning-based methods for visual segmentation have made progress on particular types of segmentation tasks, but are limited by the necessary supervision, the narrow definitions of fixed tasks, and the lack of control during inference for correcting errors. To remedy the rigidity and annotation burden of standard approaches, we address the problem of few-shot segmentation: given few image and few pixel supervision, segment any images accordingly. We propose guided networks, which extract a latent task representation from any amount of supervision, and optimize our architecture end-to-end for f…
SpinalSAM-R1: A Vision-Language Multimodal Interactive System for Spine CT Segmentation
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（人工智能学院，南京航空航天大学）；Department of Orthopedics, Qilu Hospital, Shandong University（骨科部，齐鲁医院，山东大学）；Key Laboratory of Qingdao in Medicine and Engineering（医学与工程青岛重点实验室）
Korea University（高丽大学）；Chungnam National University（忠南大学）
We propose a novel guided interactive segmentation (GIS) algorithm for video objects to improve the segmentation accuracy and reduce the interaction time. First, we design the reliability-based attention module to analyze the reliability of multiple annotated frames. Second, we develop the intersection-aware propagation module to propagate segmentation results to neighboring frames. Third, we introduce the GIS mechanism for a user to select unsatisfactory frames quickly with less effort. Experimental results demonstrate that the proposed algorithm provides more accurate segmentation results at…
Shenzhen Institute of Artificial Intelligence and Robotics for Society, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳） 深圳市人工智能与机器人研究院）；School of Information Science and Engineering, Northeastern University, Shenyang（东北大学 信息科学与工程学院）；Maxvision Technology Corp., Shenzhen（万Vision科技公司（深圳））
University of Electronic Science and Technology of China（电子科技大学）；King’s College London（伦敦国王学院）；University Hospitals KU Leuven（鲁汶大学附属医院）；University College London（伦敦大学学院）；SenseTime Research（商汤科技研究院）
Segmentation of the fetal brain from stacks of motion-corrupted fetal MRI slices is important for motion correction and high-resolution volume reconstruction. Although Convolutional Neural Networks (CNNs) have been widely used for automatic segmentation of the fetal brain, their results may still benefit from interactive refinement for challenging slices. To improve the efficiency of interactive refinement process, we propose an Uncertainty-Guided Interactive Refinement (UGIR) framework. We first propose a grouped convolution-based CNN to obtain multiple automatic segmentation predictions with…
The target of automatic video summarization is to create a short skim of the original long video while preserving the major content/events. There is a growing interest in the integration of user queries into video summarization or query-driven video summarization. This video summarization method predicts a concise synopsis of the original video based on the user query, which is commonly represented by the input text. However, two inherent problems exist in this query-driven way. First, the text query might not be enough to describe the exact and diverse needs of the user. Second, the user cann…
Prompt-Guided Interactive Segmentation of Interstitial Lung Disease in Thoracic CT
University of Bern（伯尔尼大学）；Graduate School for Cellular and Biomedical Sciences（细胞与生物医学科学研究生院）；Bern University Hospital（伯尔尼大学医院）；Lucern Cantonal Hospital（卢塞恩州立医院）；Lausanne University Hospital (CHUV) and University of Lausanne（洛桑大学医院（CHUV）与洛桑大学）
Neurons have a polarized structure, including dendrites and axons, and compartment-specific functions can be affected by dwelling mitochondria. It is known that the morphology of mitochondria is closely related to the functions of neurons and neurodegenerative diseases. Even though several deep learning methods have been developed to automatically analyze the morphology of mitochondria, the application of existing methods to actual analysis still encounters several difficulties. Since the performance of pre-trained deep learning model may vary depending on the target data, re-training of the m…
We present a deep learning method for the interactive video object segmentation. Our method is built upon two core operations, interaction and propagation, and each operation is conducted by Convolutional Neural Networks. The two networks are connected both internally and externally so that the networks are trained jointly and interact with each other to solve the complex video object segmentation problem. We propose a new multi-round training scheme for the interactive video object segmentation so that the networks can learn how to understand the user's intention and update incorrect estimati…
In this contribution, a semi-automatic segmentation algorithm for (medical) image analysis is presented. More precise, the approach belongs to the category of interactive contouring algorithms, which provide real-time feedback of the segmentation result. However, even with interactive real-time contouring approaches there are always cases where the user cannot find a satisfying segmentation, e.g. due to homogeneous appearances between the object and the background, or noise inside the object. For these difficult cases the algorithm still needs additional user support. However, this additional …
Optimization-Guided Diffusion for Interactive Scene Generation
Beijing Institute of Technology（北京理工大学）；OpenDriveLab at The University of Hong Kong（香港大学OpenDriveLab）；Nanyang Technological University（南洋理工大学）；NVIDIA Research（英伟达研究院）；Yinwang Intelligent Tech. Co. Ltd.（银网智能科技有限公司）
I2E: From Image Pixels to Actionable Interactive Environments for Text-Guided Image Editing
Huazhong University of Science and Technology（华中科技大学）；Kuaishou Technology（快手科技）；Central China Normal University（华中师范大学）；Tsinghua University（清华大学）；Shanghai AI Laboratory（上海人工智能实验室）
Toward Reliable AR-Guided Surgical Navigation: Interactive Deformation Modeling with Data-Driven Biomechanics and Prompts
Department of Computer Science and Engineering, The Chinese University of Hong Kong, HKSAR, China（计算机科学与工程系，香港中文大学（深圳））；Center of Smart Health, School of Nursing, The Hong Kong Polytechnic University, HKSAR, China（智能健康中心，香港理工大学）；Department of Hepatobiliary Surgery, The Third Affiliated Hospital of Southern Medical University, Guangzhou, China（肝胆外科部，南方医科大学第三附属医院）
Korea University（高丽大学）；Korea Basic Science Institute（韩国基础科学研究院）；Korea Artificial Organ Center（韩国人工器官中心）；ICT Convergence Technology for Health and Safety（健康安全信息通信融合技术）；Department of Electronics and Information Engineering（电子与信息工程系）；Bioimaging Research Team（生物成像研究团队）
Purpose: To acquire the real-time interactive temperature map for aqueous and adipose tissue, the problems of long acquisition and processing time must be addressed. To overcome these major challenges, this paper proposes a cascaded convolutional neural network (CNN) framework and multi-echo gradient echo (meGRE) with a single reference variable flip angle (srVFA). Methods: To optimize the echo times for each method, MR images are acquired using a meGRE sequence; meGRE images with two flip angles (FAs) and meGRE images with a single FA are acquired during the pretreatment and treatment stages,…
EvRWKV: A Continuous Interactive RWKV Framework for Effective Event-Guided Low-Light Image Enhancement
Anhui Provincial International Joint Research center for Advanced technology in Medical imaging（安徽省国际联合先进医学影像技术研究中心）；School of Artificial Intelligence（人工智能学院）；Anhui University（安徽大学）；State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology（光电信息采集与防护技术国家重点实验室）；Anhui Provincial Key Laboratory of Secure Artificial Intelligence（安徽省安全人工智能重点实验室）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（粤港澳大湾区人工智能与数字经济实验室（深圳））
Time-variant Image Inpainting via Interactive Distribution Transition Estimation
University of Alberta（阿尔伯塔大学）；IHPC and CFAR, A*STAR（IHPC和CFAR，A*STAR）；Nanyang Technological University（南洋理工大学）；University of South Carolina（南卡罗来纳大学）；School of Artificial Intelligence, Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Ministry of Education, Jilin University（人工智能学院，知识驱动的人机智能工程研究中心，教育部，吉林大学）；College of Intelligence and Computing, Tianjin University（智能与计算学院，天津大学）；University of Tokyo（东京大学）
POCI-Diff: 3D-Layout Guided Diffusion for Controllable Synthetic Surveillance Data Generation
DISI, University of Trento（DISI，特伦托大学）；Toretei S.r.l.（Toretei公司）；Visual AI Lab, MIT-IBM Watson AI Lab（视觉人工智能实验室，MIT-IBM沃森人工智能实验室）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
CellPilot: A unified approach to automatic and interactive segmentation in histopathology
School of Computation and Information Technology, Technical University of Munich（慕尼黑工业大学计算与信息技术学院）；Helmholtz Zentrum München - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心——德国环境健康研究中心）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）
Bielefeld University（比勒费尔德大学）；Queensland University of Technology (QUT)（昆士兰科技大学）
This work discusses a learning approach to mask rewarding objects in images using sparse reward signals from an imitation learning dataset. For that, we train an Hourglass network using only feedback from a critic model. The Hourglass network learns to produce a mask to decrease the critic's score of a high score image and increase the critic's score of a low score image by swapping the masked areas between these two images. We trained the model on an imitation learning dataset from the NeurIPS 2020 MineRL Competition Track, where our model learned to mask rewarding objects in a complex intera…
Scene-Generalizable Interactive Segmentation of Radiance Fields
Harbin Institute of Technology（哈尔滨工业大学）；Y-tech, Kuaishou Technology（快手科技Y-tech）；Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）；Dartmouth College（达特茅斯学院）
Kent State University（肯特州立大学）；Stony Brook University（石溪大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）
Existing interactive visualization tools for deep learning are mostly applied to the training, debugging, and refinement of neural network models working on natural images. However, visual analytics tools are lacking for the specific application of x-ray image classification with multiple structural attributes. In this paper, we present an interactive system for domain scientists to visually study the multiple attributes learning models applied to x-ray scattering images. It allows domain scientists to interactively explore this important type of scientific images in embedded spaces that are d…
