This work explores a hybrid approach to segmentation as an alternative to a purely data-driven approach. We introduce an end-to-end U-Net based network called DU-Net, which uses additional frequency preserving features, namely the Scattering Coefficients (SC), for medical image segmentation. SC are translation invariant and Lipschitz continuous to deformations which help DU-Net outperform other conventional CNN counterparts on four datasets and two segmentation tasks: Optic Disc and Optic Cup in color fundus images and fetal Head in ultrasound images. The proposed method shows remarkable improvement over the basic U-Net with performance competitive to state-of-the-art methods. The results indicate that it is possible to use a lighter network trained with fewer images (without any augmentation) to attain good segmentation results.
Indian Institute of Information Technology Allahabad（阿拉哈巴德印度信息技术学院）
In the clinical settings, during digital examination of histopathological slides, the pathologist annotate the slides by marking the rough boundary around the suspected tumour region. The marking or annotation is generally represented as a polygonal boundary that covers the extent of the tumour in the slide. These polygonal markings are difficult to imitate through CAD techniques since the tumour regions are heterogeneous and hence segmenting them would require exhaustive pixel wise ground truth annotation. Therefore, for CAD analysis, the ground truths are generally annotated by pathologist e…
Many loss functions have been derived from cross-entropy loss functions such as large-margin softmax loss and focal loss. The large-margin softmax loss makes the classification more rigorous and prevents overfitting. The focal loss alleviates class imbalance in object detection by down-weighting the loss of well-classified examples. Recent research has shown that these two loss functions derived from cross entropy have valuable applications in the field of image segmentation. However, to the best of our knowledge, there is no unified formulation that combines these two loss functions so that t…
A large number of retinal vessel analysis methods based on image segmentation have emerged in recent years. However, existing methods depend on cumbersome backbones, such as VGG16 and ResNet-50, benefiting from their powerful feature extraction capabilities but suffering from high computational costs. In this paper, we propose a novel neural network (HybridNetSeg) dedicated to solving this drawback while further improving overall performance. Considering deformable convolution can extract complex and variable structural information, and larger kernel in mixed depthwise convolution makes contri…
TBConvL-Net: A Hybrid Deep Learning Architecture for Robust Medical Image Segmentation
COMSATS University Islamabad (CUI)（COMSATS大学伊斯兰堡校区）；University of New South Wales（新南威尔士大学）；University of Engineering and Technology (UET) Lahore（拉合尔工程技术大学）
本文提出混合架构 TBConvL-Net，融合 CNN、双卷积 LSTM 与 ViT 建模局部、长程及时序上下文，并采用复合损失提升边界与鲁棒性，在十种公开数据集上优于现有方法。
Sichuan Normal University（四川师范大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Transformers have achieved remarkable success in medical image analysis owing to their powerful capability to use flexible self-attention mechanism. However, due to lacking intrinsic inductive bias in modeling visual structural information, they generally require a large-scale pre-training schedule, limiting the clinical applications over expensive small-scale medical data. To this end, we propose a parameter-efficient transformer to explore intrinsic inductive bias via position information for medical image segmentation. Specifically, we empirically investigate how different position encoding…
Due to the lack of expertise for medical image annotation, the investigation of label-efficient methodology for medical image segmentation becomes a heated topic. Recent progresses focus on the efficient utilization of weak annotations together with few strongly-annotated labels so as to achieve comparable segmentation performance in many unprofessional scenarios. However, these approaches only concentrate on the supervision inconsistency between strongly- and weakly-annotated instances but ignore the instance inconsistency inside the weakly-annotated instances, which inevitably leads to perfo…
This paper proposes a generative ScatterNet hybrid deep learning (G-SHDL) network for semantic image segmentation. The proposed generative architecture is able to train rapidly from relatively small labeled datasets using the introduced structural priors. In addition, the number of filters in each layer of the architecture is optimized resulting in a computationally efficient architecture. The G-SHDL network produces state-of-the-art classification performance against unsupervised and semi-supervised learning on two image datasets. Advantages of the G-SHDL network over supervised methods are d…
Xiamen University（厦门大学）；Beihang University（北京航空航天大学）；University of Hull（赫尔大学）；Chinese Academy of Sciences（中国科学院）
Medical image segmentation is one of the most fundamental tasks concerning medical information analysis. Various solutions have been proposed so far, including many deep learning-based techniques, such as U-Net, FC-DenseNet, etc. However, high-precision medical image segmentation remains a highly challenging task due to the existence of inherent magnification and distortion in medical images as well as the presence of lesions with similar density to normal tissues. In this paper, we propose TFCNs (Transformers for Fully Convolutional denseNets) to tackle the problem by introducing ResLinear-Tr…
SegTransVAE: Hybrid CNN -- Transformer with Regularization for medical image segmentation
VinBrain JSC.（VinBrain联合股份公司）；University of Science, Ho Chi Minh City（胡志明市科学大学）；Vietnam National University, Ho Chi Minh City（胡志明市越南国家大学）；Vin University（文大大学）
Rutgers University（罗格斯大学）；The College of New Jersey（新泽西学院）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
We study few-shot semantic segmentation that aims to segment a target object from a query image when provided with a few annotated support images of the target class. Several recent methods resort to a feature masking (FM) technique to discard irrelevant feature activations which eventually facilitates the reliable prediction of segmentation mask. A fundamental limitation of FM is the inability to preserve the fine-grained spatial details that affect the accuracy of segmentation mask, especially for small target objects. In this paper, we develop a simple, effective, and efficient approach to …
Chongqing University of Posts and Telecommunications（重庆邮电大学）；NTT Media Intelligence Laboratories（NTT媒体情报实验室）；Georgia Institute of Technology（佐治亚理工学院）
Weakly-supervised instance segmentation, which could greatly save labor and time cost of pixel mask annotation, has attracted increasing attention in recent years. The commonly used pipeline firstly utilizes conventional image segmentation methods to automatically generate initial masks and then use them to train an off-the-shelf segmentation network in an iterative way. However, the initial generated masks usually contains a notable proportion of invalid masks which are mainly caused by small object instances. Directly using these initial masks to train segmentation model is harmful for the p…
The Chinese University of Hong Kong（香港中文大学）；Imsight Medical Technology, Inc.（伊赛智能医疗科技有限公司）
Liver cancer is one of the leading causes of cancer death. To assist doctors in hepatocellular carcinoma diagnosis and treatment planning, an accurate and automatic liver and tumor segmentation method is highly demanded in the clinical practice. Recently, fully convolutional neural networks (FCNs), including 2D and 3D FCNs, serve as the back-bone in many volumetric image segmentation. However, 2D convolutions can not fully leverage the spatial information along the third dimension while 3D convolutions suffer from high computational cost and GPU memory consumption. To address these issues, we …
Multi-Modal Brain Tumor Segmentation via 3D Multi-Scale Self-attention and Cross-attention
University of Electronic Science and Technology of China（电子科技大学）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
College of Mathematics and Statistic, Henan University（河南大学数学与统计学院）
The performance of image segmentation highly relies on the original inputting image. When the image is contaminated by some noises or blurs, we can not obtain the efficient segmentation result by using direct segmentation methods. In order to efficiently segment the contaminated image, this paper proposes a two step method based on the hybrid total variation model with a box constraint and the K-means clustering method. In the first step, the hybrid model is based on the weighted convex combination between the total variation functional and the high-order total variation as the regularization …
Image thresholding has played an important role in image segmentation. This paper presents a hybrid approach for image segmentation based on the thresholding by fuzzy c-means (THFCM) algorithm for image segmentation. The goal of the proposed approach is to find a discerner cluster able to find an automatic threshold. The algorithm is formulated by applying the standard FCM clustering algorithm to the frequencies (y-values) on the smoothed histogram. Hence, the frequencies of an image can be used instead of the conventional whole data of image. The cluster that has the highest peak which repres…
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
Semantic segmentation in very high resolution (VHR) aerial images is one of the most challenging tasks in remote sensing image understanding. Most of the current approaches are based on deep convolutional neural networks (DCNNs). However, standard convolution with local receptive fields fails in modeling global dependencies. Prior researches have indicated that attention-based methods can capture long-range dependencies and further reconstruct the feature maps for better representation. Nevertheless, limited by the mere perspective of spacial and channel attention and huge computation complexi…
Rutgers University（罗格斯大学）；SenseBrain（感知大脑公司）；Shanghai AI Laboratory（上海人工智能实验室）；Centre for Perceptual and Interactive Intelligence（感知与交互智能中心）
Transformer architecture has emerged to be successful in a number of natural language processing tasks. However, its applications to medical vision remain largely unexplored. In this study, we present UTNet, a simple yet powerful hybrid Transformer architecture that integrates self-attention into a convolutional neural network for enhancing medical image segmentation. UTNet applies self-attention modules in both encoder and decoder for capturing long-range dependency at different scales with minimal overhead. To this end, we propose an efficient self-attention mechanism along with relative pos…
Recently, transformer-based image segmentation methods have achieved notable success against previous solutions. While for video domains, how to effectively model temporal context with the attention of object instances across frames remains an open problem. In this paper, we propose an online video instance segmentation framework with a novel instance-aware temporal fusion method. We first leverages the representation, i.e., a latent code in the global context (instance code) and CNN feature maps to represent instance- and pixel-level features. Based on this representation, we introduce a crop…
DyGLNet: Hybrid Global-Local Feature Fusion with Dynamic Upsampling for Medical Image Segmentation
College of Information Science and Engineering, Henan University of Technology（河南理工大学信息科学与工程学院）；School of Science, Sun Yat-sen University（中山大学科学学院）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院智能信息处理重点实验室）
HybridMIM: A Hybrid Masked Image Modeling Framework for 3D Medical Image Segmentation
Medical College of Tianjin University（天津大学医学院）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The University of Hong Kong（香港大学）；Haihe Hospital, Tianjin University（天津大学海河医院）；College of Computing and Intelligence, Tianjin University（天津大学计算与智能学院）
In this article a novel algorithm for color image segmentation has been developed. The proposed algorithm based on combining two existing methods in such a novel way to obtain a significant method to partition the color image into significant regions. On the first phase, the traditional Otsu method for gray channel image segmentation were applied for each of the R,G, and B channels separately to determine the suitable automatic threshold for each channel. After that, the new modified channels are integrated again to formulate a new color image. The resulted image suffers from some kind of dist…
HiDiff: Hybrid Diffusion Framework for Medical Image Segmentation
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University（脑启发智能科学与技术研究院，复旦大学）；Shanghai Key Lab of Intelligent Information Processing, School of Computer Science, Fudan University（上海智能信息处理重点实验室，复旦大学计算机学院）
The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；Zhejiang University（浙江大学）；The University of Sydney（悉尼大学）；Nanyang Technological University（南洋理工大学）
Cascade is a classic yet powerful architecture that has boosted performance on various tasks. However, how to introduce cascade to instance segmentation remains an open question. A simple combination of Cascade R-CNN and Mask R-CNN only brings limited gain. In exploring a more effective approach, we find that the key to a successful instance segmentation cascade is to fully leverage the reciprocal relationship between detection and segmentation. In this work, we propose a new framework, Hybrid Task Cascade (HTC), which differs in two important aspects: (1) instead of performing cascaded refine…
Facebook（脸书）；Google（谷歌）；The University of Hong Kong（香港大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）
State-of-the-art results of semantic segmentation are established by Fully Convolutional neural Networks (FCNs). FCNs rely on cascaded convolutional and pooling layers to gradually enlarge the receptive fields of neurons, resulting in an indirect way of modeling the distant contextual dependence. In this work, we advocate the use of spatially recurrent layers (i.e. ReNet layers) which directly capture global contexts and lead to improved feature representations. We demonstrate the effectiveness of ReNet layers by building a Naive deep ReNet (N-ReNet), which achieves competitive performance on …
Weakly-Supervised Semantic Segmentation (WSSS) methods with image-level labels generally train a classification network to generate the Class Activation Maps (CAMs) as the initial coarse segmentation labels. However, current WSSS methods still perform far from satisfactorily because their adopted CAMs 1) typically focus on partial discriminative object regions and 2) usually contain useless background regions. These two problems are attributed to the sole image-level supervision and aggregation of global information when training the classification networks. In this work, we propose the visual…
HQ-UNet: A Hybrid Quantum-Classical U-Net with a Quantum Bottleneck for Remote Sensing Image Segmentation
Space Applications Centre, Indian Space Research Organisation (ISRO), India（印度空间研究组织（ISRO）空间应用中心）；New-Brunswick Honors College, Rutgers University, New Jersey（新泽西罗格斯大学新不伦瑞克荣誉学院）；Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay（印度理工学院孟买资源工程研究中心）
RUFNet: Query-Guided Support Mask Refinement and Uncertainty Fusion based on Hybrid Mamba for Few-Shot Brain Tumor Segmentation
Department of Language Science and Technology, The Hong Kong Polytechnic University（香港理工大学语言科学与技术系）；School of Artificial Intelligence, Chongqing University of Technology（重庆理工大学人工智能学院）；Affiliated Lianyungang Hospital of Xuzhou Medical University（徐州医科大学附属连云港医院）；College of Mechatronics and Control Engineering, Shenzhen University（深圳大学机电与控制工程学院）
Norwegian University of Science and Technology（挪威科技大学）；St. Olavs Hospital, Trondheim University Hospital（圣奥拉夫医院，特隆赫姆大学医院）；SINTEF Digital（挪威科技工业研究院数字研究所）
Histopathological cancer diagnostics has become more complex, and the increasing number of biopsies is a challenge for most pathology laboratories. Thus, development of automatic methods for evaluation of histopathological cancer sections would be of value. In this study, we used 624 whole slide images (WSIs) of breast cancer from a Norwegian cohort. We propose a cascaded convolutional neural network design, called H2G-Net, for semantic segmentation of gigapixel histopathological images. The design involves a detection stage using a patch-wise method, and a refinement stage using a convolution…
MaxViT-UNet: Multi-Axis Attention for Medical Image Segmentation
Pakistan Institute of Engineering & Applied Sciences（巴基斯坦工程与应用科学学院）；PIEAS Artificial Intelligence Center (PAIC)（PIEAS人工智能中心）；Center for Mathematical Sciences（数学科学中心）
HBFormer: A Hybrid-Bridge Transformer for Microtumor and Miniature Organ Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；Guangdong University of Technology（广东技术大学）；University of Macau（澳门大学）；Huizhou University（惠州大学）
Technion - Israel Institute of Technology（以色列理工学院）
Deep learning approaches to generic (non-semantic) segmentation have so far been indirect and relied on edge detection. This is in contrast to semantic segmentation, where DNNs are applied directly. We propose an alternative approach called Deep Generic Segmentation (DGS) and try to follow the path used for semantic segmentation. Our main contribution is a new method for learning a pixel-wise representation that reflects segment relatedness. This representation is combined with a CRF to yield the segmentation algorithm. We show that we are able to learn meaningful representations that improve …
