Encoder-decoder architectures are widely adopted for medical image segmentation tasks. With the lateral skip connection, the models can obtain and fuse both semantic and resolution information in deep layers to achieve more accurate segmentation performance. However, in many applications (e.g., blurry boundary images), these models often cannot precisely locate complex boundaries and segment tiny isolated parts. To solve this challenging problem, we firstly analyze why simple skip connections are not enough to help accurately locate indistinct boundaries and argue that it is due to the fuzzy information in the skip connection provided in the encoder layers. Then we propose a semantic-guided encoder feature learning strategy to learn both high resolution and rich semantic encoder features so that we can more accurately locate the blurry boundaries, which can also enhance the network by selectively learning discriminative features. Besides, we further propose a soft contour constraint mechanism to model the blurry boundary detection. Experimental results on real clinical datasets show that our proposed method can achieve state-of-the-art segmentation accuracy, especially for the blurry regions. Further analysis also indicates that our proposed network components indeed contribute to the improvement of performance. Experiments on additional datasets validate the generalization ability of our proposed method.
Automated segmentation of brain tumors from 3D magnetic resonance images (MRIs) is necessary for the diagnosis, monitoring, and treatment planning of the disease. Manual delineation practices require anatomical knowledge, are expensive, time consuming and can be inaccurate due to human error. Here, we describe a semantic segmentation network for tumor subregion segmentation from 3D MRIs based on encoder-decoder architecture. Due to a limited training dataset size, a variational auto-encoder branch is added to reconstruct the input image itself in order to regularize the shared decoder and impo…
Huazhong University of Science and Technology（华中科技大学）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
We consider the problem of task-agnostic feature upsampling in dense prediction where an upsampling operator is required to facilitate both region-sensitive tasks like semantic segmentation and detail-sensitive tasks such as image matting. Existing upsampling operators often can work well in either type of the tasks, but not both. In this work, we present FADE, a novel, plug-and-play, and task-agnostic upsampling operator. FADE benefits from three design choices: i) considering encoder and decoder features jointly in upsampling kernel generation; ii) an efficient semi-shift convolutional opera…
Med-DisSeg: Dispersion-Driven Representation Learning for Fine-Grained Medical Image Segmentation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；Southeast University（东南大学）；Nanjing University（南京大学）；University of Sydney（悉尼大学）；University of Wollongong（伍伦贡大学）；Shandong University（山东大学）；Shandong Chest Hospital（山东省胸科医院）
Accurate image segmentation plays a crucial role in medical image analysis, yet it faces great challenges of various shapes, diverse sizes, and blurry boundaries. To address these difficulties, square kernel-based encoder-decoder architecture has been proposed and widely used, but its performance remains still unsatisfactory. To further cope with these challenges, we present a novel double-branch encoder architecture. Our architecture is inspired by two observations: 1) Since the discrimination of features learned via square convolutional kernels needs to be further improved, we propose to uti…
Semantic segmentation, which refers to pixel-wise classification of an image, is a fundamental topic in computer vision owing to its growing importance in robot vision and autonomous driving industries. It provides rich information about objects in the scene such as object boundary, category, and location. Recent methods for semantic segmentation often employ an encoder-decoder structure using deep convolutional neural networks. The encoder part extracts feature of the image using several filters and pooling operations, whereas the decoder part gradually recovers the low-resolution feature map…
Bournemouth University（伯恩茅斯大学）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；BCN Medtech（巴塞罗那医疗科技公司）；Universitat Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚高等研究院）
Polyps represent an early sign of the development of Colorectal Cancer. The standard procedure for their detection consists of colonoscopic examination of the gastrointestinal tract. However, the wide range of polyp shapes and visual appearances, as well as the reduced quality of this image modality, turn their automatic identification and segmentation with computational tools into a challenging computer vision task. In this work, we present a new strategy for the delineation of gastrointestinal polyps from endoscopic images based on a direct extension of common encoder-decoder networks for se…
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
Ultrasound imaging is generally employed for real-time investigation of internal anatomy of the human body for disease identification. Delineation of the anatomical boundary of organs and pathological lesions is quite challenging due to the stochastic nature of speckle intensity in the images, which also introduces visual fatigue for the observer. This paper introduces a fully convolutional neural network based method to segment organ and pathologies in ultrasound volume by learning the spatial-relationship between closely related classes in the presence of stochastically varying speckle inten…
An Edge-aware Prompt-enhanced SAM for Ultrasound Image Segmentation
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University（航空信息安全与可信计算重点实验室，教育部，武汉大学计算机科学与工程学院）；National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University（多媒體軟體國家工程研究中心，人工智能研究院，武漢大學計算機科學學院，湖北省多媒體與網絡通信工程重點實驗室）
针对超声图像分割中 SAM 边界描绘不佳问题，提出 EP-SAM，通过图像编码器多块特征提取和边缘感知监督，生成高质量提示引导模型，在多个基准测试中表现优于现有基于 SAM 的方法。
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；NVIDIA（英伟达）
Textures and edges contribute different information to image recognition. Edges and boundaries encode shape information, while textures manifest the appearance of regions. Despite the success of Convolutional Neural Networks (CNNs) in computer vision and medical image analysis applications, predominantly only texture abstractions are learned, which often leads to imprecise boundary delineations. In medical imaging, expert manual segmentation often relies on organ boundaries; for example, to manually segment a liver, a medical practitioner usually identifies edges first and subsequently fills i…
Shanghai Jiao Tong University（上海交通大学）；Imsight Medical Technology Co. Ltd.（深圳医诺智能科技有限公司）；The Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）
Segmentation of colorectal cancerous regions from 3D Magnetic Resonance (MR) images is a crucial procedure for radiotherapy which conventionally requires accurate delineation of tumour boundaries at an expense of labor, time and reproducibility. While deep learning based methods serve good baselines in 3D image segmentation tasks, small applicable patch size limits effective receptive field and degrades segmentation performance. In addition, Regions of interest (RoIs) localization from large whole volume 3D images serves as a preceding operation that brings about multiple benefits in terms of …
German Aerospace Center（德国航空航天中心）；ETH Zurich（苏黎世联邦理工学院）；TU München（慕尼黑工业大学）
We present an end-to-end trainable deep convolutional neural network (DCNN) for semantic segmentation with built-in awareness of semantically meaningful boundaries. Semantic segmentation is a fundamental remote sensing task, and most state-of-the-art methods rely on DCNNs as their workhorse. A major reason for their success is that deep networks learn to accumulate contextual information over very large windows (receptive fields). However, this success comes at a cost, since the associated loss of effecive spatial resolution washes out high-frequency details and leads to blurry object boundari…
Feedforward fully convolutional neural networks currently dominate in semantic segmentation of 3D point clouds. Despite their great success, they suffer from the loss of local information at low-level layers, posing significant challenges to accurate scene segmentation and precise object boundary delineation. Prior works either address this issue by post-processing or jointly learn object boundaries to implicitly improve feature encoding of the networks. These approaches often require additional modules which are difficult to integrate into the original architecture. To improve the segmentatio…
Shanghai Normal University（上海师范大学）；Jiangxi University of Finance and Economics（江西财经大学）
Inspired by the recent success of fully convolutional networks (FCN) in semantic segmentation, we propose a deep smoke segmentation network to infer high quality segmentation masks from blurry smoke images. To overcome large variations in texture, color and shape of smoke appearance, we divide the proposed network into a coarse path and a fine path. The first path is an encoder-decoder FCN with skip structures, which extracts global context information of smoke and accordingly generates a coarse segmentation mask. To retain fine spatial details of smoke, the second path is also designed as an …
AFFSegNet: Adaptive Feature Fusion Segmentation Network for Microtumors and Multi-Organ Segmentation
Southern University of Science and Technology（南方科技大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；University of Macau（澳门大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
FDNet: Feature Decoupled Segmentation Network for Tooth CBCT Image
Hangzhou Dianzi University（杭州电子科技大学）；College of Media Engineering, Communication University of Zhejiang（浙江传媒学院媒体工程学院）；Shandong University（山东大学）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
针对牙齿 CBCT 中伪影复杂、边界模糊的问题，本文提出 FDNet，用 LF-Wavelet 增强全局语义并结合 SAM 编码器细化边界，在基准上取得最优分割性能。
NVIDIA（英伟达）；University of California, Los Angeles（加州大学洛杉矶分校）
Automated segmentation of kidneys and kidney tumors is an important step in quantifying the tumor's morphometrical details to monitor the progression of the disease and accurately compare decisions regarding the kidney tumor treatment. Manual delineation techniques are often tedious, error-prone and require expert knowledge for creating unambiguous representation of kidneys and kidney tumors segmentation. In this work, we propose an end-to-end boundary aware fully Convolutional Neural Networks (CNNs) for reliable kidney and kidney tumor semantic segmentation from arterial phase abdominal 3D CT…
RF-HiT: Rectified Flow Hierarchical Transformer for General Medical Image Segmentation
Computer Science department Mohamed Khider University Biskra, Algeria；ISASI, CNR \& University of Salento 73100 Lecce, Italy；AI University of the Basque Country (UPV/EHU), Spain；Sorbonne Center for Artificial Intelligence, Sorbonne University Abu Dhabi, UAE
Image inpainting methods have shown significant improvements by using deep neural networks recently. However, many of these techniques often create distorted structures or blurry textures inconsistent with surrounding areas. The problem is rooted in the encoder layers' ineffectiveness in building a complete and faithful embedding of the missing regions. To address this problem, two-stage approaches deploy two separate networks for a coarse and fine estimate of the inpainted image. Some approaches utilize handcrafted features like edges or contours to guide the reconstruction process. These met…
Indian Institute of Information Technology Allahabad（阿拉哈巴德印度信息技术学院）
Deep learning has brought the most profound contribution towards biomedical image segmentation to automate the process of delineation in medical imaging. To accomplish such task, the models are required to be trained using huge amount of annotated or labelled data that highlights the region of interest with a binary mask. However, efficient generation of the annotations for such huge data requires expert biomedical analysts and extensive manual effort. It is a tedious and expensive task, while also being vulnerable to human error. To address this problem, a self-supervised learning framework, …
Barlow-Swin: Toward a novel siamese-based segmentation architecture using Swin-Transformers
HSLU, Lucerne University of Applied Sciences and Arts（卢塞恩应用科学与艺术大学）；Technical University of Munich（慕尼黑技术大学）；University College London (UCL)（伦敦大学学院）；TOELT LLC AI lab（TOELT LLC人工智能实验室）
Adapting SAM to Nuclei Instance Segmentation and Classification via Cooperative Fine-Grained Refinement
organization= College of Computer；Data Science , addressline= Fuzhou University , city= Fuzhou , country= China；organization= School of Mechanical Engineering；Automation , addressline= Fuzhou University , city= Fuzhou , country= China；organization= College of Physics；Information Engineering , addressline= Fuzhou University , city= Fuzhou , country= China
Localization-Infused Vision-Language Semantic Fusion for Text-Guided Medical Image Segmentation
Air Force Engineering University（空军工程大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Sydney（悉尼大学）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）；Zhongguancun Institute of Artificial Intelligence（中关村人工智能研究院）
Shanghai Jiao Tong University（上海交通大学）；Shandong Cancer Hospital Affiliated to Shandong University（山东大学附属山东省肿瘤医院）；Deepwise AI Lab（深睿医疗人工智能实验室）
Segmentation of nasopharyngeal carcinoma (NPC) from Magnetic Resonance Images (MRI) is a crucial prerequisite for NPC radiotherapy. However, manually segmenting of NPC is time-consuming and labor-intensive. Additionally, single-modality MRI generally cannot provide enough information for its accurate delineation. Therefore, a multi-modality MRI fusion network (MMFNet) based on three modalities of MRI (T1, T2 and contrast-enhanced T1) is proposed to complete accurate segmentation of NPC. The backbone of MMFNet is designed as a multi-encoder-based network, consisting of several encoders to captu…
NeuroTS-Net: Multi-Class Semantic Segmentation of Pediatric Brain Tumors in Multi-Modal MRI
Lucian Blaga University of Sibiu（锡比乌卢奇安·布拉加大学）；University of Arizona（亚利桑那大学）；Pediatric Clinical Hospital of Sibiu（锡比乌儿科临床医院）；Johns Hopkins University（约翰斯·霍普金斯大学）；MedLife Polisano Hospital（MedLife Polisano医院）
TReND: Transformer derived features and Regularized NMF for neonatal functional network Delineation
University of Pennsylvania（宾夕法尼亚大学）；Children’s Hospital of Philadelphia（费城儿童医院）；Peking University（北京大学）；Beijing Children’s Hospital（北京儿童医院）；Beijing Normal University（北京师范大学）
CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学系）
Deep convolutional networks (CNNs) have exhibited their potential in image inpainting for producing plausible results. However, in most existing methods, e.g., context encoder, the missing parts are predicted by propagating the surrounding convolutional features through a fully connected layer, which intends to produce semantically plausible but blurry result. In this paper, we introduce a special shift-connection layer to the U-Net architecture, namely Shift-Net, for filling in missing regions of any shape with sharp structures and fine-detailed textures. To this end, the encoder feature of t…
Mobile-Seed: Joint Semantic Segmentation and Boundary Detection for Mobile Robots
Wuhan University（武汉大学）；Technical University of Munich（慕尼黑工业大学）；Nanyang Technological University（南洋理工大学）；Institute of Infocomm Research (I2R)（资讯通信研究院）；National University of Defense Technology（国防科技大学）
Is the medical image segmentation problem solved? A survey of current developments and future directions
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）；University of Pennsylvania（宾夕法尼亚大学）；Cleveland Clinic（克利夫兰诊所）；Mayo Clinic（梅奥诊所）；Brigham and Women’s Hospital, Harvard Medical School（哈佛医学院布莱根妇女医院）
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
The accurate segmentation of organs-at-risk (OARs) in head and neck CT images is a critical step for radiation therapy of head and neck cancer patients. However, manual delineation for numerous OARs is time-consuming and laborious, even for expert oncologists. Moreover, manual delineation results are susceptible to high intra- and inter-variability. To this end, we propose a novel dual shape guided network (DSGnet) to automatically delineate nine important OARs in head and neck CT images. To deal with the large shape variation and unclear boundary of OARs in CT images, we represent the organ s…
DEFN: Dual-Encoder Fourier Group Harmonics Network for Three-Dimensional Indistinct-Boundary Object Segmentation
Sir Run Run Shaw Hospital, Affiliated to Zhejiang University School of Medicine（浙江大学医学院附属邵逸夫医院）；Hangzhou Dianzi University（杭州电子科技大学）；University of Cambridge（剑桥大学）；School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽皇后学院电子工程与计算机科学学院）
CFFormer: Cross CNN-Transformer Channel Attention and Spatial Feature Fusion for Improved Segmentation of Heterogeneous Medical Images
School of Computer Science, University of Nottingham Ningbo China（Nottingham 中国 Nottingham 理工学院）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, China（深圳先进技术研究所，中国科学院）；School of Mathematical and Computational Sciences, Massey University（数学与计算科学学院，马斯sey 大学）；University of Nottingham, United Kingdom（Nottingham 大学，英国）
