Convolutional neural networks (CNNs) and Transformer-based models are being widely applied in medical image segmentation thanks to their ability to extract high-level features and capture important aspects of the image. However, there is often a trade-off between the need for high accuracy and the desire for low computational cost. A model with higher parameters can theoretically achieve better performance but also result in more computational complexity and higher memory usage, and thus is not practical to implement. In this paper, we look for a lightweight U-Net-based model which can remain the same or even achieve better performance, namely U-Lite. We design U-Lite based on the principle of Depthwise Separable Convolution so that the model can both leverage the strength of CNNs and reduce a remarkable number of computing parameters. Specifically, we propose Axial Depthwise Convolutions with kernels 7x7 in both the encoder and decoder to enlarge the model receptive field. To further improve the performance, we use several Axial Dilated Depthwise Convolutions with filters 3x3 for the bottleneck as one of our branches. Overall, U-Lite contains only 878K parameters, 35 times less than the traditional U-Net, and much more times less than other modern Transformer-based models. The proposed model cuts down a large amount of computational complexity while attaining an impressive performance on medical segmentation tasks compared to other state-of-the-art architectures. The code will be available at: https://github.com/duong-db/U-Lite.
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；NVIDIA（英伟达）
Textures and edges contribute different information to image recognition. Edges and boundaries encode shape information, while textures manifest the appearance of regions. Despite the success of Convolutional Neural Networks (CNNs) in computer vision and medical image analysis applications, predominantly only texture abstractions are learned, which often leads to imprecise boundary delineations. In medical imaging, expert manual segmentation often relies on organ boundaries; for example, to manually segment a liver, a medical practitioner usually identifies edges first and subsequently fills i…
University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Lab（上海人工智能实验室）；SenseTime（商汤科技）
Recently, deep learning with Convolutional Neural Networks (CNNs) and Transformers has shown encouraging results in fully supervised medical image segmentation. However, it is still challenging for them to achieve good performance with limited annotations for training. In this work, we present a very simple yet efficient framework for semi-supervised medical image segmentation by introducing the cross teaching between CNN and Transformer. Specifically, we simplify the classical deep co-training from consistency regularization to cross teaching, where the prediction of a network is used as the …
Convolutional Neural Networks (CNNs) have been successful in solving tasks in computer vision including medical image segmentation due to their ability to automatically extract features from unstructured data. However, CNNs are sensitive to rotation and affine transformation and their success relies on huge-scale labeled datasets capturing various input variations. This network paradigm has posed challenges at scale because acquiring annotated data for medical segmentation is expensive, and strict privacy regulations. Furthermore, visual representation learning with CNNs has its own flaws, e.g…
In this research, we propose a tiny image segmentation model, L^3U-net, that works on low-resource edge devices in real-time. We introduce a data folding technique that reduces inference latency by leveraging the parallel convolutional layer processing capability of the CNN accelerators. We also deploy the proposed model to such a device, MAX78000, and the results show that L^3U-net achieves more than 90% accuracy over two different segmentation datasets with 10 fps.
LV-UNet: A Lightweight and Vanilla Model for Medical Image Segmentation
College of Control Science and Engineering（控制科学与工程学院）；Zhejiang University（浙江大学）；College of Computer Science and Technology（计算机科学与技术学院）；Zhejiang University of Technology（浙江工业大学）；The First Clinical Medical College（第一临床医学院）；Guangdong Medical University（广东医科大学）
Xiamen University（厦门大学）；Beihang University（北京航空航天大学）；University of Hull（赫尔大学）；Chinese Academy of Sciences（中国科学院）
Medical image segmentation is one of the most fundamental tasks concerning medical information analysis. Various solutions have been proposed so far, including many deep learning-based techniques, such as U-Net, FC-DenseNet, etc. However, high-precision medical image segmentation remains a highly challenging task due to the existence of inherent magnification and distortion in medical images as well as the presence of lesions with similar density to normal tissues. In this paper, we propose TFCNs (Transformers for Fully Convolutional denseNets) to tackle the problem by introducing ResLinear-Tr…
ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation
Huazhong University of Science and Technology（华中科技大学）；Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属协和医院）
LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image Segmentation
School of Computer Science, Peking University（北京大学计算机学院）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Key Laboratory of High Confidence Software Technologies, Ministry of Education（教育部高可信软件技术重点实验室）；National Engineering Research Center for Software Engineering, Peking University（北京大学国家软件工程研究中心）
SegTransVAE: Hybrid CNN -- Transformer with Regularization for medical image segmentation
VinBrain JSC.（VinBrain联合股份公司）；University of Science, Ho Chi Minh City（胡志明市科学大学）；Vietnam National University, Ho Chi Minh City（胡志明市越南国家大学）；Vin University（文大大学）
Tsinghua University（清华大学）；University of San Francisco（旧金山大学）；Boston University（波士顿大学）；University of Southern California（南加州大学）
Convolutional neural networks (CNNs) have been successfully applied to medical image classification, segmentation, and related tasks. Among the many CNNs architectures, U-Net and its improved versions based are widely used and achieve state-of-the-art performance these years. These improved architectures focus on structural improvements and the size of the convolution kernel is generally fixed. In this paper, we propose a module that combines the benefits of multiple kernel sizes and we apply the proposed module to U-Net and its variants. We test our module on three segmentation benchmark data…
Fully-connected Conditional Random Field (CRF) is often used as post-processing to refine voxel classification results by encouraging spatial coherence. In this paper, we propose a new end-to-end training method called Posterior-CRF. In contrast with previous approaches which use the original image intensity in the CRF, our approach applies 3D, fully connected CRF to the posterior probabilities from a CNN and optimizes both CNN and CRF together. The experiments on white matter hyperintensities segmentation demonstrate that our method outperforms CNN, post-processing CRF and different end-to-en…
Macquarie University（麦考瑞大学）；University of Technology Sydney（悉尼科技大学）；Shanghai AI Laboratory（上海人工智能实验室）
Automatic tumor or lesion segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on Convolutional Neural Networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because, although the human visual system can detect symmetries in 2D images effectively, regular CNNs can only exploit translation invariance, overlooking further inherent symmetries existing in medical images such as rotations and reflections. To solve this problem, we propose a novel group e…
MLRU++: Multiscale Lightweight Residual UNETR++ with Attention for Efficient 3D Medical Image Segmentation
AI Research Lab, Department of Computer Science（人工智能研究实验室、计算机科学系）；Biomedical & Translational Sciences, Sanford School of Medicine（生物医学与转化科学、桑福德医学院）；University Of South Dakota（南达科他大学）
De-LightSAM: Modality-Decoupled Lightweight SAM for Generalizable Medical Segmentation
School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学宁波校区计算机科学学院）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；School of School of Engineering & Physical Sciences, University of Lincoln（林肯大学工程与物理科学学院）；Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港科学与创新研究院）；School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院）；The Hong Kong Polytechnic University（香港理工大学）
Convolutional neural network (CNN) based methods have achieved great successes in medical image segmentation, but their capability to learn global representations is still limited due to using small effective receptive fields of convolution operations. Transformer based methods are capable of modelling long-range dependencies of information for capturing global representations, yet their ability to model local context is lacking. Integrating CNN and Transformer to learn both local and global representations while exploring multi-scale features is instrumental in further improving medical image…
Resonant Beam Charging (RBC) is a wireless charging technology which supports multi-watt power transfer over meter-level distance. The features of safety, mobility and simultaneous charging capability enable RBC to charge multiple mobile devices safely at the same time. To detect the devices that need to be charged, a Mask R-CNN based dection model is proposed in previous work. However, considering the constraints of the RBC system, it's not easy to apply Mask R-CNN in lightweight hardware-embedded devices because of its heavy model and huge computation. Thus, we propose a machine learning det…
We propose adversarial constrained-CNN loss, a new paradigm of constrained-CNN loss methods, for weakly supervised medical image segmentation. In the new paradigm, prior knowledge is encoded and depicted by reference masks, and is further employed to impose constraints on segmentation outputs through adversarial learning with reference masks. Unlike pseudo label methods for weakly supervised segmentation, such reference masks are used to train a discriminator rather than a segmentation network, and thus are not required to be paired with specific images. Our new paradigm not only greatly facil…
Nanjing University of Posts and Telecommunications（南京邮电大学）；Soochow University（苏州大学）；Shanghai University（上海大学）；Nanjing University of Science and Technology（南京理工大学）；Kyushu Institute of Technology（九州工业大学）
In the past decade, convolutional neural networks (CNNs) have shown prominence for semantic segmentation. Although CNN models have very impressive performance, the ability to capture global representation is still insufficient, which results in suboptimal results. Recently, Transformer achieved huge success in NLP tasks, demonstrating its advantages in modeling long-range dependency. Recently, Transformer has also attracted tremendous attention from computer vision researchers who reformulate the image processing tasks as a sequence-to-sequence prediction but resulted in deteriorating local fe…
Medical image segmentation - the prerequisite of numerous clinical needs - has been significantly prospered by recent advances in convolutional neural networks (CNNs). However, it exhibits general limitations on modeling explicit long-range relation, and existing cures, resorting to building deep encoders along with aggressive downsampling operations, leads to redundant deepened networks and loss of localized details. Hence, the segmentation task awaits a better solution to improve the efficiency of modeling global contexts while maintaining a strong grasp of low-level details. In this paper, …
In this article, we look into some essential aspects of convolutional neural networks (CNNs) with the focus on medical image segmentation. First, we discuss the CNN architecture, thereby highlighting the spatial origin of the data, voxel-wise classification and the receptive field. Second, we discuss the sampling of input-output pairs, thereby highlighting the interaction between voxel-wise classification, patch size and the receptive field. Finally, we give a historical overview of crucial changes to CNN architectures for classification and segmentation, giving insights in the relation betwee…
Duke University（杜克大学）；Vanderbilt University（范德堡大学）
Lesion detection in digital breast tomosynthesis (DBT) is an important and a challenging problem characterized by a low prevalence of images containing tumors. Due to the label scarcity problem, large deep learning models and computationally intensive algorithms are likely to fail when applied to this task. In this paper, we present a practical yet lightweight backbone to improve the accuracy of tumor detection. Specifically, we propose a novel modification of visual transformer (ViT) on image feature patches to connect the feature patches of a tumor with healthy backgrounds of breast images a…
The University of Adelaide（阿德莱德大学）；Northwestern Polytechnical University（西北工业大学）
Convolutional neural networks (CNNs) have been the de facto standard for nowadays 3D medical image segmentation. The convolutional operations used in these networks, however, inevitably have limitations in modeling the long-range dependency due to their inductive bias of locality and weight sharing. Although Transformer was born to address this issue, it suffers from extreme computational and spatial complexities in processing high-resolution 3D feature maps. In this paper, we propose a novel framework that efficiently bridges a {\bf Co}nvolutional neural network and a {\bf Tr}ansformer {\bf (…
MSLAU-Net: A Hybrid CNN-Transformer Network for Medical Image Segmentation
College of Computer Science and Engineering（计算机科学与工程学院）；College of Artificial Intelligence（人工智能学院）；Department of Pharmacy（药学部）；Department of Prosthodontics（修复学部）；Chongqing Key Laboratory of Oral Diseases（重庆口腔疾病重点实验室）；Chongqing Municipal Key Laboratory of Oral Biomedical Engineering of Higher Education（重庆市口腔生物医学工程高等教育重点实验室）；School of Computer Science and Engineering（计算机科学与工程学院）；Southeast University（东南大学）
BiSe-Unet: A Lightweight Dual-path U-Net with Attention-refined Context for Real-time Medical Image Segmentation
Electrical and Computer Engineering, University of Central Florida（佛罗里达中央大学电气与计算机工程系）；Department of Medicine, University of Central Florida College of Medicine（佛罗里达中央大学医学院医学系）
Deep neural networks have been a prevailing technique in the field of medical image processing. However, the most popular convolutional neural networks (CNNs) based methods for medical image segmentation are imperfect because they model long-range dependencies by stacking layers or enlarging filters. Transformers and the self-attention mechanism are recently proposed to effectively learn long-range dependencies by modeling all pairs of word-to-word attention regardless of their positions. The idea has also been extended to the computer vision field by creating and treating image patches as emb…
CNN-Transformer Rectified Collaborative Learning for Medical Image Segmentation
School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；Key Laboratory for Ubiquitous Network and Service Software of Liaoning Province（辽宁省泛在网络与服务软件重点实验室）；DUT-RU International School of Information Science and Engineering, Dalian University of Technology（大连理工大学-鲁东大学国际信息科学与工程学院）；Affiliated Zhongshan Hospital of Dalian University（大连大学附属中山医院）
Binghamton University, SUNY（纽约州立大学宾汉姆顿分校）；University of Missouri-Kansas City（密苏里大学堪萨斯城分校）；New York State University Police（纽约州大学警察局）
Edge computing efficiently extends the realm of information technology beyond the boundary defined by cloud computing paradigm. Performing computation near the source and destination, edge computing is promising to address the challenges in many delay-sensitive applications, like real-time human surveillance. Leveraging the ubiquitously connected cameras and smart mobile devices, it enables video analytics at the edge. In recent years, many smart video surveillance approaches are proposed for object detection and tracking by using Artificial Intelligence (AI) and Machine Learning (ML) algorith…
Recent advances of semantic image segmentation greatly benefit from deeper and larger Convolutional Neural Network (CNN) models. Compared to image segmentation in the wild, properties of both medical images themselves and of existing medical datasets hinder training deeper and larger models because of overfitting. To this end, we propose a novel two-stream UNET architecture for automatic end-to-end medical image segmentation, in which intensity value and gradient vector flow (GVF) are two inputs for each stream, respectively. We demonstrate that two-stream CNNs with more low-level features gre…
BEFUnet: A Hybrid CNN-Transformer Architecture for Precise Medical Image Segmentation
School of Electrical Engineering, Iran University of Science and Technology（伊朗科学技术大学电气工程学院）；University College of Nabi Akram（纳比·阿克拉姆大学学院）；Department of Electrical Engineering, Qazvin Branch, Iran（伊朗加兹温分校电气工程系）
X-Ray image enhancement, along with many other medical image processing applications, requires the segmentation of images into bone, soft tissue, and open beam regions. We apply a machine learning approach to this problem, presenting an end-to-end solution which results in robust and efficient inference. Since medical institutions frequently do not have the resources to process and label the large quantity of X-Ray images usually needed for neural network training, we design an end-to-end solution for small datasets, while achieving state-of-the-art results. Our implementation produces an over…
Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep network architecture automatically. We specifically focus on designing neural architectures for medical image segmentation task. Our proposed method is based on a policy gradient reinforcement learning for which the reward function is assigned a segmentation evaluation utility (i.…
ScribFormer: Transformer Makes CNN Work Better for Scribble-based Medical Image Segmentation
Xiamen University（厦门大学）；University of Washington（华盛顿大学）；Peking University（北京大学）；University of Hull（赫尔大学）；Chinese University of Hong Kong（香港中文大学）；Hong Kong Institute of Medical Engineering（香港医学工程研究所）；Hong Kong Centre for Cerebro-cardiovascular Health Engineering（香港心脑血管健康工程中心）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能医疗科技有限公司）；Shanghai Clinical Research and Trial Center（上海临床研究与试验中心）
Cross-modality image estimation involves the generation of images of one medical imaging modality from that of another modality. Convolutional neural networks (CNNs) have been shown to be useful in identifying, characterising and extracting image patterns. Generative adversarial networks (GANs) use CNNs as generators and estimated images are discriminated as true or false based on an additional network. CNNs and GANs within the image estimation framework may be considered more generally as deep learning approaches, since imaging data tends to be large, leading to a larger number of network wei…
The George Washington University Medical Center（乔治华盛顿大学医学中心）
Recently, deep learning has become much more popular in computer vision area. The Convolution Neural Network (CNN) has brought a breakthrough in images segmentation areas, especially, for medical images. In this regard, U-Net is the predominant approach to medical image segmentation task. The U-Net not only performs well in segmenting multimodal medical images generally, but also in some tough cases of them. However, we found that the classical U-Net architecture has limitation in several aspects. Therefore, we applied modifications: 1) designed efficient CNN architecture to replace encoder an…
Institute of Mechatronic Systems, Leibniz Universität Hannover（汉诺威莱布尼茨大学机电系统研究所）
We present deformable unsupervised medical image registration using a randomly-initialized deep convolutional neural network (CNN) as regularization prior. Conventional registration methods predict a transformation by minimizing dissimilarities between an image pair. The minimization is usually regularized with manually engineered priors, which limits the potential of the registration. By learning transformation priors from a large dataset, CNNs have achieved great success in deformable registration. However, learned methods are restricted to domain-specific data and the required amounts of me…
University of Sydney（悉尼大学）；Shanghai Jiao Tong University（上海交通大学）；UNSW（新南威尔士大学）
Gigapixel medical images provide massive data, both morphological textures and spatial information, to be mined. Due to the large data scale in histology, deep learning methods play an increasingly significant role as feature extractors. Existing solutions heavily rely on convolutional neural networks (CNNs) for global pixel-level analysis, leaving the underlying local geometric structure such as the interaction between cells in the tumor microenvironment unexplored. The topological structure in medical images, as proven to be closely related to tumor evolution, can be well characterized by gr…
Macquarie University（麦考瑞大学）；University of Technology Sydney（悉尼科技大学）；The University of Sydney（悉尼大学）
Automatic tumor segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because regular CNNs can only exploit translation invariance, ignoring further inherent symmetries existing in medical images such as rotations and reflections. To mitigate this shortcoming, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning mor…
The rapid advancements in machine learning, graphics processing technologies and the availability of medical imaging data have led to a rapid increase in the use of deep learning models in the medical domain. This was exacerbated by the rapid advancements in convolutional neural network (CNN) based architectures, which were adopted by the medical imaging community to assist clinicians in disease diagnosis. Since the grand success of AlexNet in 2012, CNNs have been increasingly used in medical image analysis to improve the efficiency of human clinicians. In recent years, three-dimensional (3D) …
