Deep learning has shown excellent performance in analysing medical images. However, datasets are difficult to obtain due privacy issues, standardization problems, and lack of annotations. We address these problems by producing realistic synthetic images using a combination of 3D technologies and generative adversarial networks. We use zero annotations from medical professionals in our pipeline. Our fully unsupervised method achieves promising results on five real polyp segmentation datasets. As a part of this study we release Synth-Colon, an entirely synthetic dataset that includes 20000 realistic colon images and additional details about depth and 3D geometry: https://enric1994.github.io/synth-colon
This paper aims to contribute in bench-marking the automatic polyp segmentation problem using generative adversarial networks framework. Perceiving the problem as an image-to-image translation task, conditional generative adversarial networks are utilized to generate masks conditioned by the images as inputs. Both generator and discriminator are convolution neural networks based. The model achieved 0.4382 on Jaccard index and 0.611 as F2 score.
University of Adelaide（阿德莱德大学）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）；South Australian Health and Medical Research Institute（南澳大利亚健康与医学研究所）；Nagoya University（名古屋大学）
The automatic detection of frames containing polyps from a colonoscopy video sequence is an important first step for a fully automated colonoscopy analysis tool. Typically, such detection system is built using a large annotated data set of frames with and without polyps, which is expensive to be obtained. In this paper, we introduce a new system that detects frames containing polyps as anomalies from a distribution of frames from exams that do not contain any polyps. The system is trained using a one-class training set consisting of colonoscopy frames without polyps -- such training set is con…
Oslo University Hospital（奥斯陆大学医院）；Norwegian University of Science and Technology（挪威科技大学）；Mokpo National University（木浦国立大学）
Synthetic polyp generation is a good alternative to overcome the privacy problem of medical data and the lack of various polyp samples. In this study, we propose a deep learning-based polyp image generation framework that generates synthetic polyp images that are similar to real ones. We suggest a framework that converts a given polyp image into a negative image (image without a polyp) using a simple conditional GAN architecture and then converts the negative image into a new-looking polyp image using the same network. In addition, by using the controllable polyp masks, polyps with various cha…
Diff-VPS: Video Polyp Segmentation via a Multi-task Diffusion Network with Adversarial Temporal Reasoning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；The Hong Kong University of Science and Technology（香港科技大学）
This paper contributes to automating medical image segmentation by proposing generative adversarial network-based models to segment both polyps and instruments in endoscopy images. A major contribution of this work is to provide explanations for the predictions using a layer-wise relevance propagation approach designating which input image pixels are relevant to the predictions and to what extent. On the polyp segmentation task, the models achieved 0.84 of accuracy and 0.46 on Jaccard index. On the instrument segmentation task, the models achieved 0.96 of accuracy and 0.70 on Jaccard index. Th…
Robust Polyp Detection and Diagnosis through Compositional Prompt-Guided Diffusion Models
Zhejiang University（浙江大学）；Shanghai Institute for Advanced Study of Zhejiang University（浙江大学上海高等研究院）；Endoscopy Center, Zhongshan Hospital, Fudan University（复旦大学中山医院内窥镜中心）；Shanghai Collaborative Innovation Center of Endoscopy（上海内窥镜协同创新中心）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医疗研究中心）；Shanghai Key Laboratory of MICCAI（上海MICCAI重点实验室）；Alliance Manchester Business School, The University of Manchester（曼彻斯特大学联盟商学院）；Data Science Institute, Imperial College London（伦敦帝国学院数据科学研究所）
University of Bergen（卑尔根大学）；SimulaMet（西穆拉梅特）；Nepal Medical College（尼泊尔医学院）；Kathmandu University（加德满都大学）；UiT The Arctic University of Norway（挪威北极大学）；University of Oslo（奥斯陆大学）；Oslo Metropolitan University（奥斯陆城市大学）
Polyps are the predecessors to colorectal cancer which is considered as one of the leading causes of cancer-related deaths worldwide. Colonoscopy is the standard procedure for the identification, localization, and removal of colorectal polyps. Due to variability in shape, size, and surrounding tissue similarity, colorectal polyps are often missed by the clinicians during colonoscopy. With the use of an automatic, accurate, and fast polyp segmentation method during the colonoscopy, many colorectal polyps can be easily detected and removed. The ``Medico automatic polyp segmentation challenge'' p…
Sun Yat-sen University（中山大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；Pazhou Lab（琶洲实验室）；The Sixth Affiliated Hospital, Sun Yat-sen University（中山大学附属第六医院）
Automatic and accurate polyp segmentation plays an essential role in early colorectal cancer diagnosis. However, it has always been a challenging task due to 1) the diverse shape, size, brightness and other appearance characteristics of polyps, 2) the tiny contrast between concealed polyps and their surrounding regions. To address these problems, we propose a lesion-aware dynamic network (LDNet) for polyp segmentation, which is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. Specifically, the designed segmentation head is condi…
Indira Gandhi National Open University（英迪拉·甘地国立开放大学）
Colorectal cancer is one of fatal cancer worldwide. Colonoscopy is the standard treatment for examination, localization, and removal of colorectal polyps. However, it has been shown that the miss-rate of colorectal polyps during colonoscopy is between 6 to 27%. The use of an automated, accurate, and real-time polyp segmentation during colonoscopy examinations can help the clinicians to eliminate missing lesions and prevent further progression of colorectal cancer. The ``Medico automatic polyp segmentation challenge'' provides an opportunity to study polyp segmentation and build a fast segmenta…
University of Electronic Science and Technology of China（电子科技大学）
Colorectal cancer (CRC) is one of the most common fatal cancer in the world. Polypectomy can effectively interrupt the progression of adenoma to adenocarcinoma, thus reducing the risk of CRC development. Colonoscopy is the primary method to find colonic polyps. However, due to the different sizes of polyps and the unclear boundary between polyps and their surrounding mucosa, it is challenging to segment polyps accurately. To address this problem, we design a Boundary Distribution Guided Network (BDG-Net) for accurate polyp segmentation. Specifically, under the supervision of the ideal Boundary…
Colorectal cancer (CRC) is one of the most commonly diagnosed cancers and a leading cause of cancer deaths in the United States. Colorectal polyps that grow on the intima of the colon or rectum is an important precursor for CRC. Currently, the most common way for colorectal polyp detection and precancerous pathology is the colonoscopy. Therefore, accurate colorectal polyp segmentation during the colonoscopy procedure has great clinical significance in CRC early detection and prevention. In this paper, we propose a novel end-to-end deep learning framework for the colorectal polyp segmentation. …
Hanoi University of Science and Technology（河内科技大学）；Hanoi Medical University（河内医科大学）；The Institute of Gastroenterology and Hepatology（胃肠病学与肝病学研究所）；Vietnam National University of Agriculture（越南国立农业大学）
Colorectal cancer is among the most common malignancies and can develop from high-risk colon polyps. Colonoscopy is an effective screening tool to detect and remove polyps, especially in the case of precancerous lesions. However, the missing rate in clinical practice is relatively high due to many factors. The procedure could benefit greatly from using AI models for automatic polyp segmentation, which provide valuable insights for improving colon polyp detection. However, precise segmentation is still challenging due to variations of polyps in size, shape, texture, and color. This paper propos…
Hanoi University of Science and Technology（河内科技大学）；Hanoi Medical University（河内医科大学）；The Institute of Gastroenterology and Hepatology（胃肠肝病学研究所）；University of Medicine and Pharmacy, Hue University（顺化大学医药大学）；Vietnam National University of Agriculture（越南国立农业大学）
Automatic polyp segmentation has proven to be immensely helpful for endoscopy procedures, reducing the missing rate of adenoma detection for endoscopists while increasing efficiency. However, classifying a polyp as being neoplasm or not and segmenting it at the pixel level is still a challenging task for doctors to perform in a limited time. In this work, we propose a fine-grained formulation for the polyp segmentation problem. Our formulation aims to not only segment polyp regions, but also identify those at high risk of malignancy with high accuracy. In addition, we present a UNet-based neur…
University of Kansas（堪萨斯大学）；Ryerson University（瑞尔森大学）
Colonoscopy is a procedure to detect colorectal polyps which are the primary cause for developing colorectal cancer. However, polyp segmentation is a challenging task due to the diverse shape, size, color, and texture of polyps, shuttle difference between polyp and its background, as well as low contrast of the colonoscopic images. To address these challenges, we propose a feature enhancement network for accurate polyp segmentation in colonoscopy images. Specifically, the proposed network enhances the semantic information using the novel Semantic Feature Enhance Module (SFEM). Furthermore, ins…
AgentPolyp: Accurate Polyp Segmentation via Image Enhancement Agent
School of Mathematics, Shandong University（山东大学数学学院）；School of Information Science and Engineering, Shandong Normal University（山东师范大学信息科学与工程学院）；Yeshiva University（叶史瓦大学）；School of cyber science and technology, Sun Yat-sen University（中山大学网络空间安全学院）
Inception Institute of Artificial Intelligence（人工智能起源研究所）；Wuhan University（武汉大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Colonoscopy is an effective technique for detecting colorectal polyps, which are highly related to colorectal cancer. In clinical practice, segmenting polyps from colonoscopy images is of great importance since it provides valuable information for diagnosis and surgery. However, accurate polyp segmentation is a challenging task, for two major reasons: (i) the same type of polyps has a diversity of size, color and texture; and (ii) the boundary between a polyp and its surrounding mucosa is not sharp. To address these challenges, we propose a parallel reverse attention network (PraNet) for accur…
We present a neural network architecture for medical image segmentation of diabetic foot ulcers and colonoscopy polyps. Diabetic foot ulcers are caused by neuropathic and vascular complications of diabetes mellitus. In order to provide a proper diagnosis and treatment, wound care professionals need to extract accurate morphological features from the foot wounds. Using computer-aided systems is a promising approach to extract related morphological features and segment the lesions. We propose a convolution neural network called HarDNet-DFUS by enhancing the backbone and replacing the decoder of …
Indian Statistical Institute（印度统计研究所）；Østfold University College（东福尔德大学学院）；SimulaMet；UiT The Arctic University of Norway（挪威北极大学）；University of Oxford（牛津大学）
Colonoscopy is a gold standard procedure but is highly operator-dependent. Efforts have been made to automate the detection and segmentation of polyps, a precancerous precursor, to effectively minimize missed rate. Widely used computer-aided polyp segmentation systems actuated by encoder-decoder have achieved high performance in terms of accuracy. However, polyp segmentation datasets collected from varied centers can follow different imaging protocols leading to difference in data distribution. As a result, most methods suffer from performance drop and require re-training for each specific dat…
SimulaMet（西穆拉梅特研究所）；UIT The Arctic University of Norway（挪威北极大学）；University of Oxford（牛津大学）；Oslo Metropolitan University（奥斯陆城市大学）
Colonoscopy is the gold standard for examination and detection of colorectal polyps. Localization and delineation of polyps can play a vital role in treatment (e.g., surgical planning) and prognostic decision making. Polyp segmentation can provide detailed boundary information for clinical analysis. Convolutional neural networks have improved the performance in colonoscopy. However, polyps usually possess various challenges, such as intra-and inter-class variation and noise. While manual labeling for polyp assessment requires time from experts and is prone to human error (e.g., missed lesions)…
SimulaMet（西穆拉梅特研究所）；UIT The Arctic University of Norway（挪威北极大学）；Augere Medical AS（奥热医疗公司）；University of Oslo（奥斯陆大学）；Kristiania University College（克里斯蒂安尼亚大学学院）；Oslo Metropolitan University（奥斯陆城市大学）；Oslo University Hospital（奥斯陆大学医院）
Pixel-wise image segmentation is a highly demanding task in medical-image analysis. In practice, it is difficult to find annotated medical images with corresponding segmentation masks. In this paper, we present Kvasir-SEG: an open-access dataset of gastrointestinal polyp images and corresponding segmentation masks, manually annotated by a medical doctor and then verified by an experienced gastroenterologist. Moreover, we also generated the bounding boxes of the polyp regions with the help of segmentation masks. We demonstrate the use of our dataset with a traditional segmentation approach and …
Indira Gandhi National Open University（英迪拉·甘地国立开放大学）；Indian Statistical Institute（印度统计研究所）；Northwestern University（西北大学）
The detection and removal of precancerous polyps through colonoscopy is the primary technique for the prevention of colorectal cancer worldwide. However, the miss rate of colorectal polyp varies significantly among the endoscopists. It is well known that a computer-aided diagnosis (CAD) system can assist endoscopists in detecting colon polyps and minimize the variation among endoscopists. In this study, we introduce a novel deep learning architecture, named MKDCNet, for automatic polyp segmentation robust to significant changes in polyp data distribution. MKDCNet is simply an encoder-decoder n…
SimulaMet（西穆拉大都会研究所）；UiT The Arctic University of Norway（挪威UiT北极大学）；Oslo Metropolitan University（奥斯陆城市大学）；Augere Medical AS（奥杰雷医疗公司）；Sahlgrenska University Hospital（萨尔格伦斯卡大学医院）；Bærum Hospital（贝鲁姆医院）
Colorectal cancer is the third most common cause of cancer worldwide. According to Global cancer statistics 2018, the incidence of colorectal cancer is increasing in both developing and developed countries. Early detection of colon anomalies such as polyps is important for cancer prevention, and automatic polyp segmentation can play a crucial role for this. Regardless of the recent advancement in early detection and treatment options, the estimated polyp miss rate is still around 20\%. Support via an automated computer-aided diagnosis system could be one of the potential solutions for the over…
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers
Nankai University（南开大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Inception Institute of Artificial Intelligence（Inception人工智能研究所）；Institute of High Performance Computing（高性能计算研究所）；Terminus Group（特斯联科技集团）；UCAS-Terminus AI Lab（中国科学院大学-特斯联人工智能实验室）
Indira Gandhi National Open University（英迪拉·甘地国立开放大学）；Larkin Community Hospital Palm Springs Campus（拉金社区医院棕榈泉院区）；Long Island Jewish Valley Stream（长岛犹太谷溪医院）；Feinberg School of Medicine, Northwestern University（西北大学范伯格医学院）
Colorectal cancer (CRC) is one of the most common causes of cancer and cancer-related mortality worldwide. Performing colon cancer screening in a timely fashion is the key to early detection. Colonoscopy is the primary modality used to diagnose colon cancer. However, the miss rate of polyps, adenomas and advanced adenomas remains significantly high. Early detection of polyps at the precancerous stage can help reduce the mortality rate and the economic burden associated with colorectal cancer. Deep learning-based computer-aided diagnosis (CADx) system may help gastroenterologists to identify po…
Oslo University Hospital（奥斯陆大学医院）；University of Oslo（奥斯陆大学）；Norwegian University of Science and Technology（挪威科技大学）；OmniVision Technologies Norway AS（OmniVision技术挪威公司）；LG CNS
Automatic polyp detection and segmentation are highly desirable for colon screening due to polyp miss rate by physicians during colonoscopy, which is about 25%. However, this computerization is still an unsolved problem due to various polyp-like structures in the colon and high interclass polyp variations in terms of size, color, shape, and texture. In this paper, we adapt Mask R-CNN and evaluate its performance with different modern convolutional neural networks (CNN) as its feature extractor for polyp detection and segmentation. We investigate the performance improvement of each feature extr…
Northwestern University（西北大学）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）
Colonoscopy is considered the most effective screening test to detect colorectal cancer (CRC) and its precursor lesions, i.e., polyps. However, the procedure experiences high miss rates due to polyp heterogeneity and inter-observer dependency. Hence, several deep learning powered systems have been proposed considering the criticality of polyp detection and segmentation in clinical practices. Despite achieving improved outcomes, the existing automated approaches are inefficient in attaining real-time processing speed. Moreover, they suffer from a significant performance drop when evaluated on i…
National Institute of Standards and Technology（美国国家标准与技术研究院）；Colorado State University（科罗拉多州立大学）；University of Colorado（科罗拉多大学）；Hanoi University of Science and Technology（河内科技大学）；Hanoi Medical University（河内医科大学）；Institute of Gastroenterology and Hepatology（胃肠病学与肝病学研究所）；University of Medicine and Pharmacy, Hue University（顺化大学医学与药学院）；Vietnam National University of Agriculture（越南国立农业大学）
In recent years, computer-aided automatic polyp segmentation and neoplasm detection have been an emerging topic in medical image analysis, providing valuable support to colonoscopy procedures. Attentions have been paid to improving the accuracy of polyp detection and segmentation. However, not much focus has been given to latency and throughput for performing these tasks on dedicated devices, which can be crucial for practical applications. This paper introduces a novel deep neural network architecture called BlazeNeo, for the task of polyp segmentation and neoplasm detection with an emphasis …
University of Science, VNU-HCM（胡志明市国家大学科学大学）；John von Neumann Institute, VNU-HCM（胡志明市国家大学约翰·冯·诺依曼研究所）；Vietnam National University, Ho Chi Minh city（越南国家大学胡志明市分校）
The Medico: Multimedia Task 2020 focuses on developing an efficient and accurate computer-aided diagnosis system for automatic segmentation [3]. We participate in task 1, Polyps segmentation task, which is to develop algorithms for segmenting polyps on a comprehensive dataset. In this task, we propose methods combining Residual module, Inception module, Adaptive Convolutional neural network with U-Net model, and PraNet for semantic segmentation of various types of polyps in endoscopic images. We select 5 runs with different architecture and parameters in our methods. Our methods show potential…
Colorectal cancer (CRC) is the second leading cause of cancer-related death worldwide. Excision of polyps during colonoscopy helps reduce mortality and morbidity for CRC. Powered by deep learning, computer-aided diagnosis (CAD) systems can detect regions in the colon overlooked by physicians during colonoscopy. Lacking high accuracy and real-time speed are the essential obstacles to be overcome for successful clinical integration of such systems. While literature is focused on improving accuracy, the speed parameter is often ignored. Toward this critical need, we intend to develop a novel real…
ASPS: Augmented Segment Anything Model for Polyp Segmentation
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Northwestern Polytechnical University（西北工业大学）；Beijing Technology and Business University（北京工商大学）
针对 SAM 在息肉分割中忽略局部细节且分布外性能差的问题，提出 ASPS 方法，通过 CFA 融合 CNN 与 ViT 特征、UPR 利用 IoU 分数缓解不确定性，有效提升分割性能。
Beijing University of Posts and Telecommunications（北京邮电大学）
Polyp segmentation is of great importance in the early diagnosis and treatment of colorectal cancer. Since polyps vary in their shape, size, color, and texture, accurate polyp segmentation is very challenging. One promising way to mitigate the diversity of polyps is to model the contextual relation for each pixel such as using attention mechanism. However, previous methods only focus on learning the dependencies between the position within an individual image and ignore the contextual relation across different images. In this paper, we propose Duplex Contextual Relation Network (DCRNet) to cap…
Topo-VM-UNetV2: Encoding Topology into Vision Mamba UNet for Polyp Segmentation
Department of Computer Science, University of Texas Rio Grande Valley（德克萨斯理工大学里奥格兰德分校计算机科学系）；Department of Mathematical Sciences, The University of Texas at Dallas（德克萨斯大学达拉斯分校数学科学系）
NepAL Applied Mathematics and Informatics Institute for Research (NAAMII)（尼泊尔应用数学与信息学研究所（NAAMII））；Northwestern University（西北大学）；University of Oxford（牛津大学）；NIHR Oxford Biomedical Research Centre（英国国家健康研究所牛津生物医学研究中心）
Colonoscopy is a gold standard procedure but is highly operator-dependent. Automated polyp segmentation, a precancerous precursor, can minimize missed rates and timely treatment of colon cancer at an early stage. Even though there are deep learning methods developed for this task, variability in polyp size can impact model training, thereby limiting it to the size attribute of the majority of samples in the training dataset that may provide sub-optimal results to differently sized polyps. In this work, we exploit size-related and polyp number-related features in the form of text attention duri…
Colorectal cancer is a one of the highest causes of cancer-related death, especially in men. Polyps are one of the main causes of colorectal cancer and early diagnosis of polyps by colonoscopy could result in successful treatment. Diagnosis of polyps in colonoscopy videos is a challenging task due to variations in the size and shape of polyps. In this paper we proposed a polyp segmentation method based on convolutional neural network. Performance of the method is enhanced by two strategies. First, we perform a novel image patch selection method in the training phase of the network. Second, in …
Cancer is a disease that occurs as a result of the uncontrolled division and proliferation of cells. Colon cancer is one of the most common types of cancer in the world. Polyps that can be seen in the large intestine can cause cancer if not removed with early intervention. Deep learning and image segmentation techniques are used to minimize the number of polyps that goes unnoticed by the experts during these interventions. Although these techniques perform well in terms of accuracy, they require too many parameters. We propose a new model to address this problem. Our proposed model requires fe…
Australian National University（澳大利亚国立大学）；Minjiang University（闽江学院）；Johns Hopkins University（约翰斯·霍普金斯大学）；ETH Zürich（苏黎世联邦理工学院）；UCLA（加州大学洛杉矶分校）；NPU（西北工业大学）
We present the first comprehensive video polyp segmentation (VPS) study in the deep learning era. Over the years, developments in VPS are not moving forward with ease due to the lack of large-scale fine-grained segmentation annotations. To address this issue, we first introduce a high-quality frame-by-frame annotated VPS dataset, named SUN-SEG, which contains 158,690 colonoscopy frames from the well-known SUN-database. We provide additional annotations with diverse types, i.e., attribute, object mask, boundary, scribble, and polygon. Second, we design a simple but efficient baseline, dubbed PN…
