TTT-Unet: Enhancing U-Net with Test-Time Training Layers for Biomedical Image Segmentation
Lehigh University（里海大学）；Northeastern University（东北大学）；University of Notre Dame（圣母大学）；University of Georgia（佐治亚大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
Rong Zhou, Zhengqing Yuan, Zhiling Yan, Weixiang Sun, Kai Zhang, Yiwei Li, Yanfang Ye, Xiang Li, Lifang He, Lichao Sun
Biomedical image segmentation is crucial for accurately diagnosing and analyzing various diseases. However, Convolutional Neural Networks (CNNs) and Transformers, the most commonly used architectures for this task, struggle to effectively capture long-range dependencies due to the inherent locality of CNNs and the computational complexity of Transformers. To address this limitation, we introduce TTT-Unet, a novel framework that integrates Test-Time Training (TTT) layers into the traditional U-Net architecture for biomedical image segmentation. TTT-Unet dynamically adjusts model parameters during the testing time, enhancing the model's ability to capture both local and long-range features. We evaluate TTT-Unet on multiple medical imaging datasets, including 3D abdominal organ segmentation in CT and MR images, instrument segmentation in endoscopy images, and cell segmentation in microscopy images. The results demonstrate that TTT-Unet consistently outperforms state-of-the-art CNN-based and Transformer-based segmentation models across all tasks. The code is available at https://github.com/rongzhou7/TTT-Unet.
xLSTM-UNet can be an Effective 2D & 3D Medical Image Segmentation Backbone with Vision-LSTM (ViL) better than its Mamba Counterpart
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；KOKONI, Moxin (Huzhou) Tech. Co., LTD.（墨芯（湖州）科技有限公司KOKONI）；School of Information Engineering, Huzhou University（湖州师范学院信息工程学院）；Information Systems Technology and Design Pillar, Singapore University of Technology and Design（新加坡科技设计大学信息系统技术与设计 pillar）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）
New York University（纽约大学）；Manipal Institute of Technology, Manipal Academy of Higher Education（马尼帕尔高等教育学院马尼帕尔理工学院）；Columbia University（哥伦比亚大学）
Biomedical image segmentation is one of the fastest growing fields which has seen extensive automation through the use of Artificial Intelligence. This has enabled widespread adoption of accurate techniques to expedite the screening and diagnostic processes which would otherwise take several days to finalize. In this paper, we present an end-to-end pipeline to segment lungs from chest X-ray images, training the neural network model on the Japanese Society of Radiological Technology (JSRT) dataset, using UNet to enable faster processing of initial screening for various lung disorders. The pipel…
Indian Institute of Information Technology Allahabad（阿拉哈巴德印度信息技术学院）
Deep learning has brought the most profound contribution towards biomedical image segmentation to automate the process of delineation in medical imaging. To accomplish such task, the models are required to be trained using huge amount of annotated or labelled data that highlights the region of interest with a binary mask. However, efficient generation of the annotations for such huge data requires expert biomedical analysts and extensive manual effort. It is a tedious and expensive task, while also being vulnerable to human error. To address this problem, a self-supervised learning framework, …
Currently, developments of deep learning techniques are providing instrumental to identify, classify, and quantify patterns in medical images. Segmentation is one of the important applications in medical image analysis. In this regard, U-Net is the predominant approach to medical image segmentation tasks. However, we found that those U-Net based models have limitations in several aspects, for example, millions of parameters in the U-Net consuming considerable computation resource and memory, lack of global information, and missing some tough objects. Therefore, we applied two modifications to …
Beijing Zoezen Robot Co Ltd（北京佐伊森机器人有限公司）；Beihang University（北京航空航天大学）
Medical image segmentation have drawn massive attention as it is important in biomedical image analysis. Good segmentation results can assist doctors with their judgement and further improve patients' experience. Among many available pipelines in medical image analysis, Unet is one of the most popular neural networks as it keeps raw features by adding concatenation between encoder and decoder, which makes it still widely used in industrial field. In the mean time, as a popular model which dominates natural language process tasks, transformer is now introduced to computer vision tasks and have …
Artificial Intelligence Products Group, Intel Corporation（英特尔公司人工智能产品组）；Novartis Institutes for Biomedical Research（诺华生物医学研究所）；Novartis Institutes for Biomedical Research, Basel, Switzerland（诺华生物医学研究所（瑞士巴塞尔））；Novartis Institutes for Biomedical Research, Emeryville, CA, USA（诺华生物医学研究所（美国加利福尼亚州埃默里维尔））
Existing approaches to train neural networks that use large images require to either crop or down-sample data during pre-processing, use small batch sizes, or split the model across devices mainly due to the prohibitively limited memory capacity available on GPUs and emerging accelerators. These techniques often lead to longer time to convergence or time to train (TTT), and in some cases, lower model accuracy. CPUs, on the other hand, can leverage significant amounts of memory. While much work has been done on parallelizing neural network training on multiple CPUs, little attention has been gi…
Engineering School of ITTCS Ural Federal University（乌拉尔联邦大学ITTCS工程学院）；Institute of Natural Sciences Ural Federal University（乌拉尔联邦大学自然科学研究所）；Laboratory of Translational Medicine and Bioinformatics Institute of Immunology and Physiology（免疫学与生理学研究所转化医学与生物信息学实验室）；Laboratory of Immunophysiology and Immunopharmacology Institute of Immunology and Physiology（免疫学与生理学研究所免疫生理学与免疫药理学实验室）
Deep neural networks show high accuracy in theproblem of semantic and instance segmentation of biomedicaldata. However, this approach is computationally expensive. Thecomputational cost may be reduced with network simplificationafter training or choosing the proper architecture, which providessegmentation with less accuracy but does it much faster. In thepresent study, we analyzed the accuracy and performance ofUNet and ENet architectures for the problem of semantic imagesegmentation. In addition, we investigated the ENet architecture by replacing of some convolution layers with box-convolutio…
Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）；Shri Guruji Gobind Singhji Institute of Engineering and Technology（什里古鲁吉·戈宾德·辛格吉工程技术学院）
Colorectal cancer is a leading cause of death worldwide. However, early diagnosis dramatically increases the chances of survival, for which it is crucial to identify the tumor in the body. Since its imaging uses high-resolution techniques, annotating the tumor is time-consuming and requires particular expertise. Lately, methods built upon Convolutional Neural Networks(CNNs) have proven to be at par, if not better in many biomedical segmentation tasks. For the task at hand, we propose another CNN-based approach, which uses atrous convolutions and residual connections besides the conventional fi…
Southern University of Science and Technology（南方科技大学）
Nonunion is one of the challenges faced by orthopedics clinics for the technical difficulties and high costs in photographing interosseous capillaries. Segmenting vessels and filling capillaries are critical in understanding the obstacles encountered in capillary growth. However, existing datasets for blood vessel segmentation mainly focus on the large blood vessels of the body, and the lack of labeled capillary image datasets greatly limits the methodological development and applications of vessel segmentation and capillary filling. Here, we present a benchmark dataset, named IFCIS-155, consi…
We propose a deep clustering architecture alongside image segmentation for medical image analysis. The main idea is based on unsupervised learning to cluster images on severity of the disease in the subject's sample, and this image is then segmented to highlight and outline regions of interest. We start with training an autoencoder on the images for segmentation. The encoder part from the autoencoder branches out to a clustering node and segmentation node. Deep clustering using Kmeans clustering is performed at the clustering branch and a lightweight model is used for segmentation. Each of the…
Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis
Institute of Computer Science, University of Tartu（塔尔图大学计算机科学研究所）；Center for Artificial Intelligence, Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学人工智能中心）；STACC
Cell image analysis is crucial in Alzheimer's research to detect the presence of A$β$ protein inhibiting cell function. Deep learning speeds up the process by making only low-level data sufficient for fruitful inspection. We first found Unet is most suitable in augmented microscopy by comparing performance in multi-class semantics segmentation. We develop the augmented microscopy method to capture nuclei in a brightfield image and the transformer using Unet model to convert an input image into a sequence of topological information. The performance regarding Intersection-over-Union is consisten…
Segmentation has been a major task in neuroimaging. A large number of automated methods have been developed for segmenting healthy and diseased brain tissues. In recent years, deep learning techniques have attracted a lot of attention as a result of their high accuracy in different segmentation problems. We present a new deep learning based segmentation method, DeepMRSeg, that can be applied in a generic way to a variety of segmentation tasks. The proposed architecture combines recent advances in the field of biomedical image segmentation and computer vision. We use a modified UNet architectur…
City College New York（纽约市立学院）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
Purpose: Conventional automated segmentation of the head anatomy in MRI distinguishes different brain and non-brain tissues based on image intensities and prior tissue probability maps (TPM). This works well for normal head anatomies, but fails in the presence of unexpected lesions. Deep convolutional neural networks leverage instead spatial patterns and can learn to segment lesions, but often ignore prior probabilities. Approach: We add three sources of prior information to a three-dimensional convolutional network, namely, spatial priors with a TPM, morphological priors with conditional rand…
