Existing polyp segmentation models from colonoscopy images often fail to provide reliable segmentation results on datasets from different centers, limiting their applicability. Our objective in this study is to create a robust and well-generalized segmentation model named PrototypeLab that can assist in polyp segmentation. To achieve this, we incorporate various lighting modes such as White light imaging (WLI), Blue light imaging (BLI), Linked color imaging (LCI), and Flexible spectral imaging color enhancement (FICE) into our new segmentation model, that learns to create prototypes for each class of object present in the images. These prototypes represent the characteristic features of the objects, such as their shape, texture, color. Our model is designed to perform effectively on out-of-distribution (OOD) datasets from multiple centers. We first generate a coarse mask that is used to learn prototypes for the main object class, which are then employed to generate the final segmentation mask. By using prototypes to represent the main class, our approach handles the variability present in the medical images and generalize well to new data since prototype capture the underlying distribution of the data. PrototypeLab offers a promising solution with a dice coefficient of $\geq$ 90\% and mIoU $\geq$ 85\% with a near real-time processing speed for polyp segmentation. It achieved superior performance on OOD datasets compared to 16 state-of-the-art image segmentation architectures, potentially improving clinical outcomes. Codes are available at https://github.com/xxxxx/PrototypeLab.
School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Aberdeen Institute of Data Science and Artificial Intelligence, South China Normal University（华南师范大学阿伯丁数据科学与人工智能学院）
Polyp segmentation is a crucial step towards computer-aided diagnosis of colorectal cancer. However, most of the polyp segmentation methods require pixel-wise annotated datasets. Annotated datasets are tedious and time-consuming to produce, especially for physicians who must dedicate their time to their patients. We tackle this issue by proposing a novel framework that can be trained using only weakly annotated images along with exploiting unlabeled images. To this end, we propose three ideas to address this problem, more specifically our contributions are: 1) a novel sparse foreground loss th…
Robert Gordon University（罗伯特戈登大学）；Griffith University（格里菲斯大学）
Colorectal cancer (CRC) is the first cause of death in many countries. CRC originates from a small clump of cells on the lining of the colon called polyps, which over time might grow and become malignant. Early detection and removal of polyps are therefore necessary for the prevention of colon cancer. In this paper, we introduce an ensemble of medical polyp segmentation algorithms. Based on an observation that different segmentation algorithms will perform well on different subsets of examples because of the nature and size of training sets they have been exposed to and because of method-intri…
We propose a new convolution neural network called HarDNet-MSEG for polyp segmentation. It achieves SOTA in both accuracy and inference speed on five popular datasets. For Kvasir-SEG, HarDNet-MSEG delivers 0.904 mean Dice running at 86.7 FPS on a GeForce RTX 2080 Ti GPU. It consists of a backbone and a decoder. The backbone is a low memory traffic CNN called HarDNet68, which has been successfully applied to various CV tasks including image classification, object detection, multi-object tracking and semantic segmentation, etc. The decoder part is inspired by the Cascaded Partial Decoder, known …
Peking University（北京大学）；SenseTime Research（商汤科技研究院）；The University of Sydney（悉尼大学）；Wuhan University（武汉大学）；Hubei Luojia Laboratory（湖北珞珈实验室）；JD Explore Academy（京东探索研究院）
The Depth-aware Video Panoptic Segmentation (DVPS) is a new challenging vision problem that aims to predict panoptic segmentation and depth in a video simultaneously. The previous work solves this task by extending the existing panoptic segmentation method with an extra dense depth prediction and instance tracking head. However, the relationship between the depth and panoptic segmentation is not well explored -- simply combining existing methods leads to competition and needs carefully weight balancing. In this paper, we present PolyphonicFormer, a vision transformer to unify these sub-tasks u…
We present a neural network architecture for medical image segmentation of diabetic foot ulcers and colonoscopy polyps. Diabetic foot ulcers are caused by neuropathic and vascular complications of diabetes mellitus. In order to provide a proper diagnosis and treatment, wound care professionals need to extract accurate morphological features from the foot wounds. Using computer-aided systems is a promising approach to extract related morphological features and segment the lesions. We propose a convolution neural network called HarDNet-DFUS by enhancing the backbone and replacing the decoder of …
University of Science, VNU-HCM（胡志明市国家大学科学大学）；John von Neumann Institute, VNU-HCM（胡志明市国家大学约翰·冯·诺依曼研究所）；Vietnam National University, Ho Chi Minh city（越南国家大学胡志明市分校）
The Medico: Multimedia Task 2020 focuses on developing an efficient and accurate computer-aided diagnosis system for automatic segmentation [3]. We participate in task 1, Polyps segmentation task, which is to develop algorithms for segmenting polyps on a comprehensive dataset. In this task, we propose methods combining Residual module, Inception module, Adaptive Convolutional neural network with U-Net model, and PraNet for semantic segmentation of various types of polyps in endoscopic images. We select 5 runs with different architecture and parameters in our methods. Our methods show potential…
SimulaMet（西穆拉梅特研究所）；UIT The Arctic University of Norway（挪威北极大学）；Augere Medical AS（奥热医疗公司）；University of Oslo（奥斯陆大学）；Kristiania University College（克里斯蒂安尼亚大学学院）；Oslo Metropolitan University（奥斯陆城市大学）；Oslo University Hospital（奥斯陆大学医院）
Pixel-wise image segmentation is a highly demanding task in medical-image analysis. In practice, it is difficult to find annotated medical images with corresponding segmentation masks. In this paper, we present Kvasir-SEG: an open-access dataset of gastrointestinal polyp images and corresponding segmentation masks, manually annotated by a medical doctor and then verified by an experienced gastroenterologist. Moreover, we also generated the bounding boxes of the polyp regions with the help of segmentation masks. We demonstrate the use of our dataset with a traditional segmentation approach and …
Convolutional Neural Networks (CNNs) are propelling advances in a range of different computer vision tasks such as object detection and object segmentation. Their success has motivated research in applications of such models for medical image analysis. If CNN-based models are to be helpful in a medical context, they need to be precise, interpretable, and uncertainty in predictions must be well understood. In this paper, we develop and evaluate recent advances in uncertainty estimation and model interpretability in the context of semantic segmentation of polyps from colonoscopy images. We evalu…
Bournemouth University（伯恩茅斯大学）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；BCN Medtech（巴塞罗那医疗科技公司）；Universitat Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚高等研究院）
Polyps represent an early sign of the development of Colorectal Cancer. The standard procedure for their detection consists of colonoscopic examination of the gastrointestinal tract. However, the wide range of polyp shapes and visual appearances, as well as the reduced quality of this image modality, turn their automatic identification and segmentation with computational tools into a challenging computer vision task. In this work, we present a new strategy for the delineation of gastrointestinal polyps from endoscopic images based on a direct extension of common encoder-decoder networks for se…
Università di Bologna（博洛尼亚大学）；University of Padua（帕多瓦大学）
Semantic segmentation has a wide array of applications ranging from medical-image analysis, scene understanding, autonomous driving and robotic navigation. This work deals with medical image segmentation and in particular with accurate polyp detection and segmentation during colonoscopy examinations. Several convolutional neural network architectures have been proposed to effectively deal with this task and with the problem of segmenting objects at different scale input. The basic architecture in image segmentation consists of an encoder and a decoder: the first uses convolutional filters to e…
University of Cambridge（剑桥大学）；Cancer Research UK Cambridge Centre（英国癌症研究中心剑桥中心）
Background: Colonoscopy remains the gold-standard screening for colorectal cancer. However, significant miss rates for polyps have been reported, particularly when there are multiple small adenomas. This presents an opportunity to leverage computer-aided systems to support clinicians and reduce the number of polyps missed. Method: In this work we introduce the Focus U-Net, a novel dual attention-gated deep neural network, which combines efficient spatial and channel-based attention into a single Focus Gate module to encourage selective learning of polyp features. The Focus U-Net further incorp…
Cancer is a disease that occurs as a result of the uncontrolled division and proliferation of cells. Colon cancer is one of the most common types of cancer in the world. Polyps that can be seen in the large intestine can cause cancer if not removed with early intervention. Deep learning and image segmentation techniques are used to minimize the number of polyps that goes unnoticed by the experts during these interventions. Although these techniques perform well in terms of accuracy, they require too many parameters. We propose a new model to address this problem. Our proposed model requires fe…
Colonoscopy is widely recognised as the gold standard procedure for the early detection of colorectal cancer (CRC). Segmentation is valuable for two significant clinical applications, namely lesion detection and classification, providing means to improve accuracy and robustness. The manual segmentation of polyps in colonoscopy images is time-consuming. As a result, the use of deep learning (DL) for automation of polyp segmentation has become important. However, DL-based solutions can be vulnerable to overfitting and the resulting inability to generalise to images captured by different colonosc…
Xi’an Jiaotong-Liverpool University（西交利物浦大学）；University of Liverpool（利物浦大学）
Colonoscopy, currently the most efficient and recognized colon polyp detection technology, is necessary for early screening and prevention of colorectal cancer. However, due to the varying size and complex morphological features of colonic polyps as well as the indistinct boundary between polyps and mucosa, accurate segmentation of polyps is still challenging. Deep learning has become popular for accurate polyp segmentation tasks with excellent results. However, due to the structure of polyps image and the varying shapes of polyps, it easy for existing deep learning models to overfitting the c…
SimulaMet；UiT The Arctic University of Norway（挪威北极大学）；Oslo Metropolitan University（奥斯陆城市大学）；Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Bærum Hospital（贝鲁姆医院）；Augere Medical AS（Augere Medical 公司）；Sahlgrenska University Hospital-Mölndal Hospital（萨尔格伦斯卡大学医院-默恩达尔医院）；Sahlgrenska Academy, University of Gothenburg（哥德堡大学萨尔格伦斯卡学院）
Deep learning in gastrointestinal endoscopy can assist to improve clinical performance and be helpful to assess lesions more accurately. To this extent, semantic segmentation methods that can perform automated real-time delineation of a region-of-interest, e.g., boundary identification of cancer or precancerous lesions, can benefit both diagnosis and interventions. However, accurate and real-time segmentation of endoscopic images is extremely challenging due to its high operator dependence and high-definition image quality. To utilize automated methods in clinical settings, it is crucial to de…
SimulaMet（西姆拉大都会研究所）；Augere Medical AS（奥格医疗公司）；UiT The Arctic University of Norway（挪威UiT北极大学）；University of Oslo（奥斯陆大学）；Oslo Metropolitan University（奥斯陆城市大学）
Accurate computer-aided polyp detection and segmentation during colonoscopy examinations can help endoscopists resect abnormal tissue and thereby decrease chances of polyps growing into cancer. Towards developing a fully automated model for pixel-wise polyp segmentation, we propose ResUNet++, which is an improved ResUNet architecture for colonoscopic image segmentation. Our experimental evaluations show that the suggested architecture produces good segmentation results on publicly available datasets. Furthermore, ResUNet++ significantly outperforms U-Net and ResUNet, two key state-of-the-art d…
Validating polyp and instrument segmentation methods in colonoscopy through Medico 2020 and MedAI 2021 Challenges
Northwestern University（西北大学）；Jadavpur University（贾达普大学）；Technische Universität Hamburg（汉堡工业大学）；SimulaMet（西穆拉梅特研究所）；Julius-Maximilian University of Würzburg（维尔茨堡大学）；Australian National University（澳大利亚国立大学）；Gachon University（嘉泉大学）；University of Oxford（牛津大学）；University of Bergen（卑尔根大学）；University of Oslo（奥斯陆大学）；Swedish Medical Center（瑞典医疗中心）；National University of Computer and Emerging Sciences（国家计算机与新兴科学大学）；University of Science, VNU-HCM（胡志明市国家大学科学大学）
S$^2$ME: Spatial-Spectral Mutual Teaching and Ensemble Learning for Scribble-supervised Polyp Segmentation
The Chinese University of Hong Kong（香港中文大学）；National University of Singapore（新加坡国立大学）；Hubei University of Technology（湖北工业大学）；University College London（伦敦大学学院）
University of Lincoln（林肯大学）；Lingnan Normal University（岭南师范学院）
Besides the complex nature of colonoscopy frames with intrinsic frame formation artefacts such as light reflections and the diversity of polyp types/shapes, the publicly available polyp segmentation training datasets are limited, small and imbalanced. In this case, the automated polyp segmentation using a deep neural network remains an open challenge due to the overfitting of training on small datasets. We proposed a simple yet effective polyp segmentation pipeline that couples the segmentation (FCN) and classification (CNN) tasks. We find the effectiveness of interactive weight transfer betwe…
Polyp-DAM: Polyp segmentation via depth anything model
Nanjing University of Science and Technology（南京理工大学）；University of Science and Technology of China（中国科学技术大学）；Sun Yat-sen University（中山大学）；National University of Singapore（新加坡国立大学）
Polyp-E: Benchmarking the Robustness of Deep Segmentation Models via Polyp Editing
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；The Fifth Medical Center, Chinese PLA General Hospital（中国人民解放军总医院第五医学中心）；Children’s Hospital of Zhejiang University School of Medicine（浙江大学医学院附属儿童医院）
Induce to Empower: Improving Lightweight Baselines via Foundation Model Induction for Generalized Polyp Segmentation
Malaviya National Institute of Technology Jaipur（斋浦尔马尔维亚国家技术学院）；Côte d’Azur University（蔚蓝海岸大学）；Khalifa University（哈里发大学）；Biomedical Perception & Intelligence Lab, University of South Dakota（南达科他大学的生物医学感知与智能实验室）
Wireless Capsule Endoscopy (WCE) is a relatively new technology to record the entire GI trace, in vivo. The large amounts of frames captured during an examination cause difficulties for physicians to review all these frames. The need for reducing the reviewing time using some intelligent methods has been a challenge. Polyps are considered as growing tissues on the surface of intestinal tract not inside of an organ. Most polyps are not cancerous, but if one becomes larger than a centimeter, it can turn into cancer by great chance. The WCE frames provide the early stage possibility for detection…
One Polyp Identifies All: One-Shot Polyp Segmentation with SAM via Cascaded Priors and Iterative Prompt Evolution
Chinese University of Hong Kong（香港中文大学）；Stanford University（斯坦福大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；Hong Kong University of Science and Technology(Guangzhou)（香港科学与技术大学（广州））；Southern University of Science and Technology（南方科技大学）
ASGNet: Adaptive Spectrum Guidance Network for Automatic Polyp Segmentation
PCA Laboratory, Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology（PCA实验室、教育部高维信息智能感知与系统重点实验室、计算机科学与工程学院、南京理工大学）；School of Information Science and Engineering, East China University of Science and Technology（信息科学与工程学院、东华大学）
Video capsule endoscopy (VCE) is used widely nowadays for visualizing the gastrointestinal (GI) tract. Capsule endoscopy exams are prescribed usually as an additional monitoring mechanism and can help in identifying polyps, bleeding, etc. To analyze the large scale video data produced by VCE exams automatic image processing, computer vision, and learning algorithms are required. Recently, automatic polyp detection algorithms have been proposed with various degrees of success. Though polyp detection in colonoscopy and other traditional endoscopy procedure based images is becoming a mature field…
CMFDNet: Cross-Mamba and Feature Discovery Network for Polyp Segmentation
School of Computer Science and Technology, Changsha University of Science and Technology（长沙理工大学计算机科学与技术学院）；SCOT – Optimal Sourcing Systems Amazon.com Services LLC（SCOT——最优采购系统亚马逊公司）；School of Mechanical and Electrical Engineering, China University of Mining and Technology（中国矿业大学机械与电子工程学院）
The Chinese University of Hong Kong(Shenzhen)（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Third Affiliated Hospital of Shenzhen University(Luohu Hospital Group)（深圳大学第三附属医院（罗湖医院集团））；University of Science and Technology of China（中国科学技术大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Accurate polyp segmentation is of great importance for colorectal cancer diagnosis. However, even with a powerful deep neural network, there still exists three big challenges that impede the development of polyp segmentation. (i) Samples collected under different conditions show inconsistent colors, causing the feature distribution gap and overfitting issue; (ii) Due to repeated feature downsampling, small polyps are easily degraded; (iii) Foreground and background pixels are imbalanced, leading to a biased training. To address the above issues, we propose the Shallow Attention Network (SANet)…
Polyp segmentation is still known as a difficult problem due to the large variety of polyp shapes, scanning and labeling modalities. This prevents deep learning model to generalize well on unseen data. However, Transformer-based approach recently has achieved some remarkable results on performance with the ability of extracting global context better than CNN-based architecture and yet lead to better generalization. To leverage this strength of Transformer, we propose a new model with encoder-decoder architecture named LAPFormer, which uses a hierarchical Transformer encoder to better extract g…
Dalian University of Technology（大连理工大学）；Peng Cheng Laboratory（鹏城实验室）
More than 90\% of colorectal cancer is gradually transformed from colorectal polyps. In clinical practice, precise polyp segmentation provides important information in the early detection of colorectal cancer. Therefore, automatic polyp segmentation techniques are of great importance for both patients and doctors. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. However, both the two operations easily generate plenty of redundant information, which will weaken the complementarity betwe…
PSTNet: Enhanced Polyp Segmentation with Multi-scale Alignment and Frequency Domain Integration
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院）山东省计算中心（国家超级计算济南中心））；Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science（山东省计算机网络重点实验室，山东省计算机科学基础研究中心）；AI Lab, Deepwise Healthcare（Deepwise Healthcare人工智能实验室）
Hanoi University of Science and Technology（河内理工大学）；Vietnam National University of Agriculture（越南国立农业大学）；Vietnam National University Ho Chi Minh City（越南胡志明市国立大学）
Identifying polyps is challenging for automatic analysis of endoscopic images in computer-aided clinical support systems. Models based on convolutional networks (CNN), transformers, and their combinations have been proposed to segment polyps with promising results. However, those approaches have limitations either in modeling the local appearance of the polyps only or lack of multi-level features for spatial dependency in the decoding process. This paper proposes a novel network, namely ColonFormer, to address these limitations. ColonFormer is an encoder-decoder architecture capable of modelin…
