Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy
SimulaMet；UIT The Arctic University of Norway（挪威北极大学）；University of Oxford（牛津大学）；Simula Research Laboratory（Simula研究实验室）；Oslo Metropolitan University（奥斯陆城市大学）；Sintef Digital（SINTEF数字研究所）；Augere Medical AS（Augere Medical公司）；Sahlgrenska University Hospital-Mölndal（萨尔格伦斯卡大学医院-默恩达尔）；Bærum Hospital（贝鲁姆医院）；Karolinska University Hospital（卡罗林斯卡大学医院）
Debesh Jha, Sharib Ali, Krister Emanuelsen, Steven A. Hicks, VajiraThambawita, Enrique Garcia-Ceja, Michael A. Riegler, Thomas de Lange, Peter T. Schmidt, Håvar… 展开作者
Debesh Jha, Sharib Ali, Krister Emanuelsen, Steven A. Hicks, VajiraThambawita, Enrique Garcia-Ceja, Michael A. Riegler, Thomas de Lange, Peter T. Schmidt, Håvard D. Johansen, Dag Johansen, Pål Halvorsen
Gastrointestinal (GI) pathologies are periodically screened, biopsied, and resected using surgical tools. Usually the procedures and the treated or resected areas are not specifically tracked or analysed during or after colonoscopies. Information regarding disease borders, development and amount and size of the resected area get lost. This can lead to poor follow-up and bothersome reassessment difficulties post-treatment. To improve the current standard and also to foster more research on the topic we have released the ``Kvasir-Instrument'' dataset which consists of $590$ annotated frames containing GI procedure tools such as snares, balloons and biopsy forceps, etc. Beside of the images, the dataset includes ground truth masks and bounding boxes and has been verified by two expert GI endoscopists. Additionally, we provide a baseline for the segmentation of the GI tools to promote research and algorithm development. We obtained a dice coefficient score of 0.9158 and a Jaccard index of 0.8578 using a classical U-Net architecture. A similar dice coefficient score was observed for DoubleUNet. The qualitative results showed that the model did not work for the images with specularity and the frames with multiple instruments, while the best result for both methods was observed on all other types of images. Both, qualitative and quantitative results show that the model performs reasonably good, but there is a large potential for further improvements. Benchmarking using the dataset provides an opportunity for researchers to contribute to the field of automatic endoscopic diagnostic and therapeutic tool segmentation for GI endoscopy.
Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy
Simula Metropolitan Center for Digital Engineering (SimulaMet)（Simula Metropolitan Center for Digital Engineering）；Oslo Metropolitan University (OsloMet)（Oslo Metropolitan University）；Simula Research Laboratory
GastroVision: A Multi-class Endoscopy Image Dataset for Computer Aided Gastrointestinal Disease Detection
Northwestern University（西北大学）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）；Jefferson Health NJ（新泽西杰斐逊医疗）；SimulaMet（西穆拉梅特研究所）；Sahlgrenska University Hospital（萨尔格伦斯卡大学医院）
The article presents a new multi-label comprehensive image dataset from flexible endoscopy, colonoscopy and capsule endoscopy, named ERS. The collection has been labeled according to the full medical specification of 'Minimum Standard Terminology 3.0' (MST 3.0), describing all possible findings in the gastrointestinal tract (104 possible labels), extended with an additional 19 labels useful in common machine learning applications. The dataset contains around 6000 precisely and 115,000 approximately labeled frames from endoscopy videos, 3600 precise and 22,600 approximate segmentation masks, an…
MultiAttenGastro: Multi-Dimensional Attention Augmentation for Gastrointestinal Endoscopy Classification
Sardar Vallabhbhai National Institute of Technology（萨达尔·瓦拉巴伊·国家理工学院）；Swasthyam Gastro and Liver Hospital（斯瓦斯蒂亚姆胃肠肝病医院）；Norwegian University of Science and Technology（挪威科技大学）
PDPM Indian Institute of Information Technology, Design and Manufacturing（PDPM印度信息技术设计与制造学院）
Wireless Capsule Endoscopy is one of the most advanced non-invasive methods for the examination of gastrointestinal tracts. An intelligent computer-aided diagnostic system for detecting gastrointestinal abnormalities like polyp, bleeding, inflammation, etc. is highly exigent in wireless capsule endoscopy image analysis. Abnormalities greatly differ in their shape, size, color, and texture, and some appear to be visually similar to normal regions. This poses a challenge in designing a binary classifier due to intra-class variations. In this study, a hybrid convolutional neural network is propos…
University of Oxford（牛津大学）；CRO Centro Riferimento Oncologico IRCCS Aviano（阿维亚诺肿瘤参考中心IRCCS）；University of Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；University of Lincoln（林肯大学）
Whilst many technologies are built around endoscopy, there is a need to have a comprehensive dataset collected from multiple centers to address the generalization issues with most deep learning frameworks. What could be more important than disease detection and localization? Through our extensive network of clinical and computational experts, we have collected, curated and annotated gastrointestinal endoscopy video frames. We have released this dataset and have launched disease detection and segmentation challenge EDD2020 https://edd2020.grand-challenge.org to address the limitations and explo…
This paper attempts to provide the reader a place to begin studying the application of computer vision and machine learning to gastrointestinal (GI) endoscopy. They have been classified into 18 categories. It should be be noted by the reader that this is a review from pre-deep learning era. A lot of deep learning based applications have not been covered in this thesis.
University of Oxford（牛津大学）；Big Data Institute（大数据研究所）；NIHR Oxford Biomedical Research Centre（英国国家健康研究所牛津生物医学研究中心）；John Radcliffe Hospital（约翰·拉德克利夫医院）
Gastrointestinal (GI) cancer precursors require frequent monitoring for risk stratification of patients. Automated segmentation methods can help to assess risk areas more accurately, and assist in therapeutic procedures or even removal. In clinical practice, addition to the conventional white-light imaging (WLI), complimentary modalities such as narrow-band imaging (NBI) and fluorescence imaging are used. While, today most segmentation approaches are supervised and only concentrated on a single modality dataset, this work exploits to use a target-independent unsupervised domain adaptation (UDA…
Open Set Recognition for Endoscopic Image Classification: A Deep Learning Approach on the Kvasir Dataset
Emory University（埃默里大学）；Boston University（波士顿大学）；Gretchen Whitney High（格蕾琴·惠特尼高中）；Auburn High School（奥本高中）；Hanyang University（汉阳大学）；Hanbat National University（韩巴国立大学）
Automated Bleeding Detection and Classification in Wireless Capsule Endoscopy with YOLOv8-X
Indian Institute of Technology, Indore (IITI)（印度理工学院印多尔分校）；Thapar Institute of Engineering & Technology (TIET)（塔帕尔工程技术学院）；Vellore Institute of Technology, Chennai (VIT)（韦洛尔理工学院 Chennai 校区）
Multi-contrast laser endoscopy for in vivo gastrointestinal imaging
Department of Biomedical Engineering（生物医学工程系）；Johns Hopkins University（约翰霍普金斯大学）；Division of Gastroenterology and Hepatology（消化内科与肝病科）；Johns Hopkins Hospital（约翰霍普金斯医院）
South Central University for Nationalities（中南民族大学）；Nanjing Eastern Theater General Hospital（南京东部战区总医院）
Wireless Capsule Endoscopy(WCE) has been widely used for the screening of gastrointestinal(GI) diseases, especially the small intestine, due to its advantages of non-invasive and painless imaging of the entire digestive tract.However, the huge amount of image data captured by WCE makes manual reading a process that requires a huge amount of tasks and can easily lead to missed detection and false detection of lesions.Therefore, In this paper, we propose a \textbf{T}wo-stage \textbf{M}ulti-scale \textbf{F}eature-fusion learning network(\textbf{TMFNet}) to automatically detect small intestinal po…
A Graph-Augmented knowledge Distillation based Dual-Stream Vision Transformer with Region-Aware Attention for Gastrointestinal Disease Classification with Explainable AI
Department of Computer Science and Engineering（计算机科学与工程系）
Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models
Rochester Institute of Technology, Rochester, NY, USA（罗切斯特技术学院）；Nepal Applied Mathematics and Informatics Institute for Research (NAAMII)（尼泊尔应用数学与信息技术研究所）；Kathmandu University（加德满都大学）；University of Lausanne（洛桑大学）；University of Aberdeen（阿伯丁大学）
Federated Foundation Model for GI Endoscopy Images
Lane Department of Computer Science and Electrical Engineering, West Virginia University（沃里奇大学计算机科学与电气工程系）；West Virginia University（沃里奇大学）；School of Medicine, West Virginia University（沃里奇大学医学院）；University of Memphis（密苏里州孟菲斯大学）；University of Aberdeen（阿伯丁大学）
There has been a longstanding expectation that the optical resolution embodiment of photoacoustic tomography could have a substantial impact on gastrointestinal endoscopy by enabling microscopic visualization of the vasculature based on the endogenous contrast mechanism. Although multiple studies have demonstrated the in vivo imaging capability of a developed imaging device over the last decade, the implementation of such an endoscopic system that can be applied immediately when necessary via the instrument channel of a video endoscope has been a challenge. In this study, we developed a 3.38 m…
Training-Time Optical Priors for Wireless Capsule Endoscopy Classification: Hemoglobin-Aware Input Fusion with Cross-Vendor Evaluation
Department of Biomedical Engineering, University of Texas Southwestern Medical Center（生物医学工程系，德克萨斯西南医学中心）；Department of Internal Medicine, University of Texas Southwestern Medical Center（内科系，德克萨斯西南医学中心）；Department of Radiation Oncology, Stanford University（放射肿瘤学系，斯坦福大学）；VELVETECH, LLC（VELVETECH公司）；Division of Digestive and Liver Diseases, Clements University Hospital（消化和肝病系，克莱门斯大学医院）；Internal Medicine, Division of Digestive and Liver Diseases, Parkland Hospital（内科，消化和肝病系，帕克兰医院）
Real-time monitoring of the gastrointestinal tract in a safe and comfortable manner is valuable for the diagnosis and therapy of many diseases. Within this realm, our review captures the trends in ingestible capsule systems with a focus on hardware and software technologies used for capsule endoscopy and remote patient monitoring. We introduce the structure and functions of the gastrointestinal tract, and the FDA guidelines for ingestible wireless telemetric medical devices. We survey the advanced features incorporated in ingestible capsule systems, such as microrobotics, closed-loop feedback,…
