Weakly-supervised learning under image-level labels supervision has been widely applied to semantic segmentation of medical lesions regions. However, 1) most existing models rely on effective constraints to explore the internal representation of lesions, which only produces inaccurate and coarse lesions regions; 2) they ignore the strong probabilistic dependencies between target lesions dataset (e.g., enteroscopy images) and well-to-annotated source diseases dataset (e.g., gastroscope images). To better utilize these dependencies, we present a new semantic lesions representation transfer model for weakly-supervised endoscopic lesions segmentation, which can exploit useful knowledge from relevant fully-labeled diseases segmentation task to enhance the performance of target weakly-labeled lesions segmentation task. More specifically, a pseudo label generator is proposed to leverage seed information to generate highly-confident pseudo pixel labels by incorporating class balance and super-pixel spatial prior. It can iteratively include more hard-to-transfer samples from weakly-labeled target dataset into training set. Afterwards, dynamically searched feature centroids for same class among different datasets are aligned by accumulating previously-learned features. Meanwhile, adversarial learning is also employed in this paper, to narrow the gap between the lesions among different datasets in output space. Finally, we build a new medical endoscopic dataset with 3659 images collected from more than 1100 volunteers. Extensive experiments on our collected dataset and several benchmark datasets validate the effectiveness of our model.
Deep learning based neural networks have gained popularity for a variety of biomedical imaging applications. In the last few years several works have shown the use of these methods for colon cancer detection and the early results have been promising. These methods can potentially be utilized to assist doctor's and may help in identifying the number of lesions or abnormalities in a diagnosis session. From our literature survey we found out that there is a lack of publicly available labeled data. Thus, as part of this work, we have aimed at open sourcing a dataset which contains annotations of p…
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Pancreas Centre, First Affiliated Hospital, Nanjing Medical University（南京医科大学第一附属医院胰腺中心）；Department of General Surgery, First Affiliated Hospital, Nanjing Medical University（南京医科大学第一附属医院普外科）；Big Data Computing Center of Southeast University（东南大学大数据计算中心）
Universitat Pompeu Fabra（庞培法布拉大学）；University of Adelaide（阿德莱德大学）；Catalan Institution for Research and Advanced Studies (ICREA)（加泰罗尼亚研究与高级研究所（ICREA））
We study the impact of different loss functions on lesion segmentation from medical images. Although the Cross-Entropy (CE) loss is the most popular option when dealing with natural images, for biomedical image segmentation the soft Dice loss is often preferred due to its ability to handle imbalanced scenarios. On the other hand, the combination of both functions has also been successfully applied in this kind of tasks. A much less studied problem is the generalization ability of all these losses in the presence of Out-of-Distribution (OoD) data. This refers to samples appearing in test time t…
Registration-Grounded Spectral Fusion for Unregistered WLI/NBI Endoscopic Lesion Segmentation
The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院）；School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Chinese PLA General Hospital（中国人民解放军总医院）；University of Arkansas at Little Rock（阿肯色大学小石城分校）；Indiana University-Purdue University Indianapolis（印第安纳大学-普渡大学印第安纳波利斯分校）
Weakly-supervised learning has attracted growing research attention on medical lesions segmentation due to significant saving in pixel-level annotation cost. However, 1) most existing methods require effective prior and constraints to explore the intrinsic lesions characterization, which only generates incorrect and rough prediction; 2) they neglect the underlying semantic dependencies among weakly-labeled target enteroscopy diseases and fully-annotated source gastroscope lesions, while forcefully utilizing untransferable dependencies leads to the negative performance. To tackle above issues, …
University of Chinese Academy of Sciences（中国科学院大学）；Huaqiao University（华侨大学）；University of Arkansas at Little Rock（阿肯色大学小石城分校）
Unsupervised domain adaptation has attracted growing research attention on semantic segmentation. However, 1) most existing models cannot be directly applied into lesions transfer of medical images, due to the diverse appearances of same lesion among different datasets; 2) equal attention has been paid into all semantic representations instead of neglecting irrelevant knowledge, which leads to negative transfer of untransferable knowledge. To address these challenges, we develop a new unsupervised semantic transfer model including two complementary modules (i.e., T_D and T_F ) for endoscopic l…
Politecnico di Milano（米兰理工大学）；CNRS-Université de Strasbourg（法国国家科学研究中心-斯特拉斯堡大学）；University of Calabria（卡拉布里亚大学）；Istituto Europeo di Oncologia (IRCCS)（欧洲肿瘤研究所（IRCCS））
Ureteroscopy and cystoscopy are the gold standard methods to identify and treat tumors along the urinary tract. It has been reported that during a normal procedure a rate of 10-20 % of the lesions could be missed. In this work we study the implementation of 3 different Convolutional Neural Networks (CNNs), using a 2-steps training strategy, to classify images from the urinary tract with and without lesions. A total of 6,101 images from ureteroscopy and cystoscopy procedures were collected. The CNNs were trained and tested using transfer learning in a two-steps fashion on 3 datasets. The datase…
Tecnologico de Monterrey（蒙特雷理工学院）；University of Leeds（利兹大学）
Prevalence of gastrointestinal (GI) cancer is growing alarmingly every year leading to a substantial increase in the mortality rate. Endoscopic detection is providing crucial diagnostic support, however, subtle lesions in upper and lower GI are quite hard to detect and cause considerable missed detection. In this work, we leverage deep learning to develop a framework to improve the localization of difficult to detect lesions and minimize the missed detection rate. We propose an end to end student-teacher learning setup where class probabilities of a trained teacher model on one class with larg…
A Point-Neighborhood Learning Framework for Nasal Endoscope Image Segmentation
School of Intelligent Engineering, Sun Yat-Sen University-Shenzhen Campus（中山大学深圳校区智能工程学院）；Department of Otolaryngology, The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院耳鼻喉科）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）
Focus-to-Perceive Representation Learning: A Cognition-Inspired Hierarchical Framework for Endoscopic Video Analysis
Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education, Xiangtan University（湘潭大学教育部智能计算与信息处理重点实验室）；School of Computer Science, Xiangtan University（湘潭大学计算机科学学院）；Key Laboratory for Artificial Intelligence and International Communication, Hunan Normal University（湖南师范大学人工智能与国际传播重点实验室）
SPRMamba: Surgical Phase Recognition for Endoscopic Submucosal Dissection with Mamba
School of Biomedical Engineering（生物医学工程学院）；Division of Gastroenterology and Hepatology, Shanghai Institute of Digestive Disease, NHC Key Laboratory of Digestive Diseases, Renji Hospital（消化内科与肝病科、上海消化疾病研究所、国家消化疾病临床医学研究中心、仁济医院）；Aier Institute of Digital Ophthalmology and Visual Science（爱尔数字眼科与视觉科学研究院）；College of Media Engineering（媒体工程学院）
SimulaMet；UiT The Arctic University of Norway（挪威北极大学）；Oslo Metropolitan University（奥斯陆城市大学）；Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Bærum Hospital（贝鲁姆医院）；Augere Medical AS（Augere Medical 公司）；Sahlgrenska University Hospital-Mölndal Hospital（萨尔格伦斯卡大学医院-默恩达尔医院）；Sahlgrenska Academy, University of Gothenburg（哥德堡大学萨尔格伦斯卡学院）
Deep learning in gastrointestinal endoscopy can assist to improve clinical performance and be helpful to assess lesions more accurately. To this extent, semantic segmentation methods that can perform automated real-time delineation of a region-of-interest, e.g., boundary identification of cancer or precancerous lesions, can benefit both diagnosis and interventions. However, accurate and real-time segmentation of endoscopic images is extremely challenging due to its high operator dependence and high-definition image quality. To utilize automated methods in clinical settings, it is crucial to de…
Small intestinal capsule endoscopy is the mainstream method for inspecting small intestinal lesions,but a single small intestinal capsule endoscopy will produce 60,000 - 120,000 images, the majority of which are similar and have no diagnostic value. It takes 2 - 3 hours for doctors to identify lesions from these images. This is time-consuming and increase the probability of misdiagnosis and missed diagnosis since doctors are likely to experience visual fatigue while focusing on a large number of similar images for an extended period of time.In order to solve these problems, we proposed a simil…
While data-driven approaches excel at many image analysis tasks, the performance of these approaches is often limited by a shortage of annotated data available for training. Recent work in semi-supervised learning has shown that meaningful representations of images can be obtained from training with large quantities of unlabeled data, and that these representations can improve the performance of supervised tasks. Here, we demonstrate that an unsupervised jigsaw learning task, in combination with supervised training, results in up to a 9.8% improvement in correctly classifying lesions in colono…
A Client-server Deep Federated Learning for Cross-domain Surgical Image Segmentation
NepAl Applied Mathematics and Informatics Institute (NAAMII)（尼泊尔应用数学与信息学研究所（NAAMII））；University of Leeds（利兹大学）；University of Liverpool（利物浦大学）；University College London（伦敦大学学院）；University of Aberdeen（阿伯丁大学）
Polyp-Gen: Realistic and Diverse Polyp Image Generation for Endoscopic Dataset Expansion
Department of Electronic Engineering, Chinese University of Hong Kong（香港中文大学电子工程系）；Centre for Artificial Intelligence and Robotics (CAIR), Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（香港科学院人工智能与机器人中心）；Department of Electrical Engineering, The City University of Hong Kong（香港城市大学电子工程系）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Sixth Affiliated Hospital, Sun Yat-sen University（中山大学第六附属医院）
SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data
UCL Hawkes Institute, University College London（伦敦大学学院霍克斯学院）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；Medtronic（美敦力）；Department of Computer Science, University Liverpool（利物浦大学计算机科学系）
ERCPMP-Gx: Endoscopic Image and Video Dataset for Morphological, Histopathological, and Genomic Characterization of Colorectal Polyposis
Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）；Familial & Hereditary Cancers Institute（家族与遗传性癌症研究所）；Shiraz University of Medical Sciences（设拉子医科大学）；DataBioX Research（DataBioX研究院）
GIM-ENDO: A Multimodal Endoscopic Image and Video Dataset for Gastric Intestinal Metaplasia Morphology and Pathology
Gastroenterology and Liver Disease Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学胃肠病与肝病研究中心，胃肠病与肝病研究所）；Iran University of Medical Sciences（伊朗医科大学）；Shiraz University of Medical Sciences（设拉子医科大学）；Gastroenterology Department, Amiralam Hospital, Tehran University of Medical Sciences（德黑兰医科大学阿米拉拉姆医院胃肠病科）；Department of Computer Engineering, Science and Research Branch, Islamic Azad University（伊斯兰阿扎德大学科学与研究分校计算机工程系）
Video streams are utilised to guide minimally-invasive surgery and diagnostic procedures in a wide range of procedures, and many computer assisted techniques have been developed to automatically analyse them. These approaches can provide additional information to the surgeon such as lesion detection, instrument navigation, or anatomy 3D shape modeling. However, the necessary image features to recognise these patterns are not always reliably detected due to the presence of irregular light patterns such as specular highlight reflections. In this paper, we aim at removing specular highlights from…
Simple is what you need for efficient and accurate medical image segmentation
Department of Ultrasound, Shenzhen Maternity and Child Healthcare Hospital, Women and Children's Medical Center, Southern Medical University, Shenzhen, Guangdong Province, China（超声科，深圳母婴保健医院，妇女儿童医疗中心，南方医科大学，深圳，广东省，中国）
Multi-Scale Structural-aware Exposure Correction for Endoscopic Imaging
School of Engineering and Sciences, Tecnologico de Monterrey（蒙特雷理工学院工程与科学学院）；Hôpital Ambroise Paré (AP-HP)（昂布鲁瓦兹-帕雷医院（巴黎公共医疗集团））；Universidad Panamericana（泛美大学）
Politecnico di Milano（米兰理工大学）；Université de Strasbourg（斯特拉斯堡大学）；Hospital Papa Giovanni XXIII（教皇约翰二十三世医院）
Objective: Accurate visual classification of bladder tissue during Trans-Urethral Resection of Bladder Tumor (TURBT) procedures is essential to improve early cancer diagnosis and treatment. During TURBT interventions, White Light Imaging (WLI) and Narrow Band Imaging (NBI) techniques are used for lesion detection. Each imaging technique provides diverse visual information that allows clinicians to identify and classify cancerous lesions. Computer vision methods that use both imaging techniques could improve endoscopic diagnosis. We address the challenge of tissue classification when annotation…
Clinically-guided Data Synthesis for Laryngeal Lesion Detection
Biomedical Robotics Laboratory, Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy Department of Informatics, Bioengineering, Robotics, Systems Engineering, University of Genova, Genoa, Italy Departament de Matem\`atiques i Inform\`atica, Universitat de Barcelona, Spain Computer Vision Center, Bellaterra, Spain Instituci\`o Catalana de Recerca i Estudis Avan cats (ICREA), Barcelona, Spain
SegDINO: Introducing Multi-Scale Structure into DINO for Efficient Medical Image Segmentation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）；Imperial College London（帝国理工学院）
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection
The Chinese University of Hong Kong（香港中文大学）；Shanghai AI Laboratory（上海人工智能实验室）；The Qilu Hospital of Shandong University（山东大学齐鲁医院）；National University o（国立大学）
Institute for Medical Engineering and Science（医学工程与科学研究所）；Massachusetts Institute of Technology（麻省理工学院）；Lyft Inc.（来福特公司）；Neuromation OU（神经自动化有限公司）；University of Michigan（密歇根大学）
Accurate detection and localization for angiodysplasia lesions is an important problem in early stage diagnostics of gastrointestinal bleeding and anemia. Gold-standard for angiodysplasia detection and localization is performed using wireless capsule endoscopy. This pill-like device is able to produce thousand of high enough resolution images during one passage through gastrointestinal tract. In this paper we present our winning solution for MICCAI 2017 Endoscopic Vision SubChallenge: Angiodysplasia Detection and Localization its further improvements over the state-of-the-art results using sev…
Interpretable histopathology-based prediction of disease relevant features in Inflammatory Bowel Disease biopsies using weakly-supervised deep learning
Deep learning-based Computer-Aided Diagnosis (CAD) has attracted appealing attention in academic researches and clinical applications. Nevertheless, the Convolutional Neural Networks (CNNs) diagnosis system heavily relies on the well-labeled lesion dataset, and the sensitivity to the variation of data distribution also restricts the potential application of CNNs in CAD. Unsupervised Domain Adaptation (UDA) methods are developed to solve the expensive annotation and domain gaps problem and have achieved remarkable success in medical image analysis. Yet existing UDA approaches only adapt knowled…
From Generalist to Specialist: A Context-Fusion Framework for Endoscopic Polyp Reporting with a Frozen VLM
Zhejiang University（浙江大学）；Shanghai Institute for Advanced Study, Zhejiang University（浙江大学上海高等研究院）；Shanghai Key Laboratory of MICCAI（上海市MICCAI重点实验室）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心）；Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University（复旦大学附属中山医院内镜中心及内镜研究所）；Shanghai Collaborative Innovation Center of Endoscopy（上海市内镜协同创新中心）；Alliance Manchester Business School, The University of Manchester（曼彻斯特大学联盟曼彻斯特商学院）
Sri Sivasubramaniya Nadar College of Engineering（斯里西瓦苏布拉马尼亚·纳达尔工程学院）
Annotated images and ground truth for the diagnosis of rare and novel diseases are scarce. This is expected to prevail, considering the small number of affected patient population and limited clinical expertise to annotate images. Further, the frequently occurring long-tailed class distributions in skin lesion and other disease classification datasets cause conventional training approaches to lead to poor generalization due to biased class priors. Few-shot learning, and meta-learning in general, aim to overcome these issues by aiming to perform well in low data regimes. This paper focuses on i…
Enhancing Diagnostic Precision in Gastric Bleeding through Automated Lesion Segmentation: A Deep DuS-KFCM Approach
University of Macau（澳门大学）；Guangdong Institute of Intelligence Science and Technology（广东省智能科学与技术研究院）；The Chinese University of Hong Kong（香港中文大学）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Chongqing Technology and Business University（重庆工商大学）；Institute of Artificial Intelligence, Chongqing Technology and Business University（重庆工商大学人工智能研究院）；Hebei University of Engineering（河北工程大学）
Endo-CLIP: Progressive Self-Supervised Pre-training on Raw Colonoscopy Records
Digital Medical Research Center, School of Basic Medical Sciences, Fudan University, Shanghai, China（上海复旦大学基础医学学院数字医学研究中心）；University College London, London, UK（伦敦大学学院）；Shanghai Key Laboratory of MICCAI, Shanghai, China（上海MICCAI重点实验室）；Endoscopy Center and Endoscopy Research Institute, Zhongshan Hospital, Fudan University, Shanghai, China（复旦大学中山医院内窥镜中心和内窥镜研究所）；Shanghai Collaborative Innovation Center of Endoscopy, Shanghai, China（上海内窥镜协同创新中心）；Shanghai Institute for Advanced Study of Zhejiang University, Shanghai, China（浙江大学上海高级研究院）；Alliance Manchester Business School, The University of Manchester, Manchester, UK（曼彻斯特大学曼彻斯特商业学校）；Data Science Institute, Imperial College London, London, UK（伦敦帝国理工学院数据科学研究院）
Pretrained, Curriculum-Tuned, and Ensembled: A Tracer-Aware Interactive Segmentation Pipeline for AutoPET V
The Netherlands Cancer Institute（荷兰癌症研究所）；Radboud University Medical Centre（拉德堡德大学医学中心）；Macao Polytechnic University（澳门理工大学）；Amsterdam University Medical Center（阿姆斯特丹大学医学中心）
SimulaMet（西姆拉大都会研究所）；Augere Medical AS（奥杰医疗公司）；UiT The Arctic University of Norway（挪威UiT北极大学）；University of Oslo（奥斯陆大学）；Oslo Metropolitan University（奥斯陆城市大学）；Sahlgrenska University Hospital（萨尔格伦斯卡大学医院）；Bærum Hospital（贝鲁姆医院）
Colonoscopy is considered the gold standard for detection of colorectal cancer and its precursors. Existing examination methods are, however, hampered by high overall miss-rate, and many abnormalities are left undetected. Computer-Aided Diagnosis systems based on advanced machine learning algorithms are touted as a game-changer that can identify regions in the colon overlooked by the physicians during endoscopic examinations, and help detect and characterize lesions. In previous work, we have proposed the ResUNet++ architecture and demonstrated that it produces more efficient results compared …
The University of Hong Kong（香港大学）；Xiamen University（厦门大学）
Self-supervised representation learning has been extremely successful in medical image analysis, as it requires no human annotations to provide transferable representations for downstream tasks. Recent self-supervised learning methods are dominated by noise-contrastive estimation (NCE, also known as contrastive learning), which aims to learn invariant visual representations by contrasting one homogeneous image pair with a large number of heterogeneous image pairs in each training step. Nonetheless, NCE-based approaches still suffer from one major problem that is one homogeneous pair is not eno…
University College London（伦敦大学学院）；Scuola Superiore Sant’Anna（圣安娜高等研究院）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Toshiba Corporate Research and Development Center（东芝企业研发中心）；The Royal Infirmary of Edinburgh（爱丁堡皇家医院）；University of Turin（都灵大学）
Early diagnosis is essential for the successful treatment of bowel cancers including colorectal cancer (CRC) and capsule endoscopic imaging with robotic actuation can be a valuable diagnostic tool when combined with automated image analysis. We present a deep learning rooted detection and segmentation framework for recognizing lesions in colonoscopy and capsule endoscopy images. We restructure established convolution architectures, such as VGG and ResNets, by converting them into fully-connected convolution networks (FCNs), fine-tune them and study their capabilities for polyp segmentation and…
