For many real-world applications involving low-power sensor edge devices deep neural networks used for image classification might not be suitable. This is due to their typically large model size and require- ment of operations often exceeding the capabilities of such resource lim- ited devices. Furthermore, camera sensors usually capture images with a Bayer color filter applied, which are subsequently converted to RGB images that are commonly used for neural network training. However, on resource-constrained devices, such conversions demands their share of energy and optimally should be skipped if possible. This work ad- dresses the need for hardware-suitable AI targeting sensor edge devices by means of the Video Capsule Endoscopy, an important medical proce- dure for the investigation of the small intestine, which is strongly limited by its battery lifetime. Accurate organ classification is performed with a final accuracy of 93.06% evaluated directly on Bayer images involv- ing a CNN with only 63,000 parameters and time-series analysis in the form of Viterbi decoding. Finally, the process of capturing images with a camera and raw image processing is demonstrated with a customized PULPissimo System-on-Chip with a RISC-V core and an ultra-low power hardware accelerator providing an energy-efficient AI-based image clas- sification approach requiring just 5.31 μJ per image. As a result, it is possible to save an average of 89.9% of energy before entering the small intestine compared to classic video capsules.
University of Oxford（牛津大学）；Bogazici University（博阿齐奇大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Friedrich Alexander Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希·亚历山大大学）；Harvard Medical School（哈佛医学院）
Although wireless capsule endoscopy is the preferred modality for diagnosis and assessment of small bowel diseases, the poor camera resolution is a substantial limitation for both subjective and automated diagnostics. Enhanced-resolution endoscopy has shown to improve adenoma detection rate for conventional endoscopy and is likely to do the same for capsule endoscopy. In this work, we propose and quantitatively validate a novel framework to learn a mapping from low-to-high resolution endoscopic images. We combine conditional adversarial networks with a spatial attention block to improve the re…
Isfahan University of Technology（伊斯法罕理工大学）；University of Michigan（密歇根大学）
Capsule endoscopy is a novel and non-invasive method for diagnosis, which assists gastroenterologists to monitor the digestive track. Although this new technology has many advantages over the conventional endoscopy, there are weaknesses that limits the usage of this technology. Some weaknesses are due to using small-size batteries. Radio transmitter consumes the largest portion of energy; consequently, a simple way to reduce the power consumption is to reduce the data to be transmitted. Many works are proposed to reduce the amount of data to be transmitted consist of specific compression metho…
Indian Statistical Institute（印度统计研究所）；Indira Gandhi National Open University（英迪拉·甘地国立开放大学）；Feinberg School of Medicine, Northwestern University（西北大学范伯格医学院）
Video capsule endoscopy is a hot topic in computer vision and medicine. Deep learning can have a positive impact on the future of video capsule endoscopy technology. It can improve the anomaly detection rate, reduce physicians' time for screening, and aid in real-world clinical analysis. CADx classification system for video capsule endoscopy has shown a great promise for further improvement. For example, detection of cancerous polyp and bleeding can lead to swift medical response and improve the survival rate of the patients. To this end, an automated CADx system must have high throughput and …
SimulaMet；UiT The Arctic University of Norway（挪威北极大学）；Oslo Metropolitan University（奥斯陆城市大学）；Institute of Biomedical Engineering, University of Oxford（牛津大学生物医学工程研究所）；Bærum Hospital（贝鲁姆医院）；Augere Medical AS（Augere Medical 公司）；Sahlgrenska University Hospital-Mölndal Hospital（萨尔格伦斯卡大学医院-默恩达尔医院）；Sahlgrenska Academy, University of Gothenburg（哥德堡大学萨尔格伦斯卡学院）
Deep learning in gastrointestinal endoscopy can assist to improve clinical performance and be helpful to assess lesions more accurately. To this extent, semantic segmentation methods that can perform automated real-time delineation of a region-of-interest, e.g., boundary identification of cancer or precancerous lesions, can benefit both diagnosis and interventions. However, accurate and real-time segmentation of endoscopic images is extremely challenging due to its high operator dependence and high-definition image quality. To utilize automated methods in clinical settings, it is crucial to de…
Norwegian University of Science and Technology（挪威科技大学）；Innlandet Hospital Trust（内陆医院信托）
This paper proposes an advanced method for contrast enhancement of capsule endoscopic images, with the main objective to obtain sufficient information about the vessels and structures in more distant (or darker) parts of capsule endoscopic images. The proposed method (PM) combines two algorithms for the enhancement of darker and brighter areas of capsule endoscopic images, respectively. The half-unit weighted bilinear algorithm (HWB) proposed in our previous work is used to enhance darker areas according to the darker map content of its HSV's component V. Enhancement of brighter areas is achie…
Semantic Map Guided Synthesis of Wireless Capsule Endoscopy Images using Diffusion Models
Captos Co., Ltd.（Captos株式会社）；Seoreu Co., Ltd.（Seoreu株式会社）；Pusan National University School of Medicine（釜山大学医学院）；Pusan National University Yangsan Hospital（釜山大学梁山医院）；Kyungpook National University（庆北大学）
Wireless Capsule Endoscopy (WCE) presented in 2001 as one of the key approaches to observe the entire gastrointestinal (GI) tract, generally the small bowels. It has been used to detect diseases in the gastrointestinal tract. Endoscopic image analysis is still a required field with many open problems. The quality of many images it produced is rather unacceptable due to the nature of this imaging system, which causes some issues to prognosticate by physicians and computer-aided diagnosis. In this paper, a novel technique is proposed to improve the quality of images captured by the WCE. More spe…
Endoscopy provides a major contribution to the diagnosis of the Gastrointestinal Tract (GIT) diseases. With Colon Endoscopy having its certain limitations, Wireless Capsule Endoscopy is gradually taking over it in the terms of ease and efficiency. WCE is performed with a miniature optical endoscope which is swallowed by the patient and transmits colour images wirelessly during its journey through the GIT, inside the body of the patient. These images are used to implement an effective and computationally efficient approach which aims to detect the abnormal and normal tissues in the GIT automati…
LLCaps: Learning to Illuminate Low-Light Capsule Endoscopy with Curved Wavelet Attention and Reverse Diffusion
The University of Sydney（悉尼大学）；The Chinese University of Hong Kong (CUHK)（香港中文大学）；Northeastern University（东北大学）；University College London（伦敦大学学院）；Shun Hing Institute of Advanced Engineering, CUHK（香港中文大学信兴高等工程研究所）
EndoUIC: Promptable Diffusion Transformer for Unified Illumination Correction in Capsule Endoscopy
The Chinese University of Hong Kong (CUHK)（香港中文大学）；Shenzhen Research Institute, CUHK（香港中文大学深圳研究院）；City University of Hong Kong（香港城市大学）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（中国科学院香港智能科学与健康研究院人工智能与机器人中心）；University College London（伦敦大学学院）；Qilu Hospital of Shandong University（山东大学齐鲁医院）；The University of Sydney（悉尼大学）
针对胶囊内窥镜图像不同曝光水平校正难题，提出可提示扩散 Transformer 框架 EndoUIC 及 CEC 数据集，在多数据集上优于 SOTA 并具临床应用价值。
Wireless capsule endoscopy (WCE) is a process in which a patient swallows a camera-embedded pill-shaped device that passes through the gastrointestinal (GI) tract, captures and transmits images to an external receiver. WCE devices are considered as a replacement of conventional endoscopy methods which are usually painful and distressful for the patients. WCE devices produce over 60,000 images typically during their course of operation inside the GI tract. These images need to be examined by expert physicians who attempt to identify frames that contain inflammation / disease. It can be hectic f…
Clinical Evaluation of Medical Image Synthesis: A Case Study in Wireless Capsule Endoscopy
University of Thessaly（塞萨利大学）；University of Southern Denmark（南丹麦大学）；Pomeranian Medical University（波美拉尼亚医科大学）；Clínica Universidad de Navarra（纳瓦拉大学临床医院）；Mater Dei Hospital（马特尔迪医院）；University of Parma（帕尔马大学）；Hospital Clínic i Provincial de Barcelona（巴塞罗那临床医院）；Centro de Investigación Biomédica en Red de Enfermedades Hepáticas y Digestivas（肝病和消化系统疾病生物医学研究网络中心）；Royal Infirmary of Edinburgh（爱丁堡皇家医院）；University Hospitals of Leicester NHS Trust（莱斯特大学医院NHS信托基金）；Hospital da Senhora da Oliveira - Guimarães（吉马良斯圣母奥利维拉医院）；Lund University（隆德大学）；Skåne University Hospital（斯科讷大学医院）
Small intestinal capsule endoscopy is the mainstream method for inspecting small intestinal lesions,but a single small intestinal capsule endoscopy will produce 60,000 - 120,000 images, the majority of which are similar and have no diagnostic value. It takes 2 - 3 hours for doctors to identify lesions from these images. This is time-consuming and increase the probability of misdiagnosis and missed diagnosis since doctors are likely to experience visual fatigue while focusing on a large number of similar images for an extended period of time.In order to solve these problems, we proposed a simil…
University of Barcelona（巴塞罗那大学）；University Hospital Vall d’Hebron（希伯伦谷大学医院）
Wireless capsule endoscopy is a medical procedure used to visualize the entire gastrointestinal tract and to diagnose intestinal conditions, such as polyps or bleeding. Current analyses are performed by manually inspecting nearly each one of the frames of the video, a tedious and error-prone task. Automatic image analysis methods can be used to reduce the time needed for physicians to evaluate a capsule endoscopy video, however these methods are still in a research phase. In this paper we focus on computer-aided polyp detection in capsule endoscopy images. This is a challenging problem because…
Medical endoscopy remains a challenging application for simultaneous localization and mapping (SLAM) due to the sparsity of image features and size constraints that prevent direct depth-sensing. We present a SLAM approach that incorporates depth predictions made by an adversarially-trained convolutional neural network (CNN) applied to monocular endoscopy images. The depth network is trained with synthetic images of a simple colon model, and then fine-tuned with domain-randomized, photorealistic images rendered from computed tomography measurements of human colons. Each image is paired with an …
University of Oxford（牛津大学）；Imperial College London（帝国理工学院）
Endoscopy is a widely used imaging modality to diagnose and treat diseases in hollow organs as for example the gastrointestinal tract, the kidney and the liver. However, due to varied modalities and use of different imaging protocols at various clinical centers impose significant challenges when generalising deep learning models. Moreover, the assembly of large datasets from different clinical centers can introduce a huge label bias that renders any learnt model unusable. Also, when using new modality or presence of images with rare patterns, a bulk amount of similar image data and their corre…
Endoscopy is a routine imaging technique used for both diagnosis and minimally invasive surgical treatment. While the endoscopy video contains a wealth of information, tools to capture this information for the purpose of clinical reporting are rather poor. In date, endoscopists do not have any access to tools that enable them to browse the video data in an efficient and user friendly manner. Fast and reliable video retrieval methods could for example, allow them to review data from previous exams and therefore improve their ability to monitor disease progression. Deep learning provides new ave…
GastroVision: A Multi-class Endoscopy Image Dataset for Computer Aided Gastrointestinal Disease Detection
Northwestern University（西北大学）；Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）；Jefferson Health NJ（新泽西杰斐逊医疗）；SimulaMet（西穆拉梅特研究所）；Sahlgrenska University Hospital（萨尔格伦斯卡大学医院）
Institute of Microelectronics, Tsinghua University（清华大学微电子学研究所）；Graduate School at Shenzhen, Tsinghua University（清华大学深圳国际研究生院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
The quality of images captured by wireless capsule endoscopy (WCE) is key for doctors to diagnose diseases of gastrointestinal (GI) tract. However, there exist many low-quality endoscopic images due to the limited illumination and complex environment in GI tract. After an enhancement process, the severe noise become an unacceptable problem. The noise varies with different cameras, GI tract environments and image enhancement. And the noise model is hard to be obtained. This paper proposes a convolutional blind denoising network for endoscopic images. We apply Deep Image Prior (DIP) method to re…
University of Sheffield（谢菲尔德大学）；Tsinghua University（清华大学）；University of Lincoln（林肯大学）
Examining and interpreting of a large number of wireless endoscopic images from the gastrointestinal tract is a tiresome task for physicians. A practical solution is to automatically construct a two dimensional representation of the gastrointestinal tract for easy inspection. However, little has been done on wireless endoscopic image stitching, let alone systematic investigation. The proposed new wireless endoscopic image stitching method consists of two main steps to improve the accuracy and efficiency of image registration. First, the keypoints are extracted by Principle Component Analysis a…
The use of wireless implanted medical devices (IMDs) is growing because they facilitate continuous monitoring of patients during normal activities, simplify medical procedures required for data retrieval and reduce the likelihood of infection associated with trailing wires. However, most of the state-of-the-art IMDs are passive and offline devices. One of the key obstacles to an active and online IMD is the infeasibility of real-time, high-quality video broadcast from the IMD. Such broadcast would help develop innovative devices such as a video-streaming capsule endoscopy (CE) pill with therap…
