The size and geometry of the prostate are known to be pivotal quantities used by clinicians to assess the condition of the gland during prostate cancer screening. As an alternative to palpation, an increasing number of methods for estimation of the above-mentioned quantities are based on using imagery data of prostate. The necessity to process large volumes of such data creates a need for automatic segmentation tools which would allow the estimation to be carried out with maximum accuracy and efficiency. In particular, the use of transrectal ultrasound (TRUS) imaging in prostate cancer screening seems to be becoming a standard clinical practice due to the high benefit-to-cost ratio of this imaging modality. Unfortunately, the segmentation of TRUS images is still hampered by relatively low contrast and reduced SNR of the images, thereby requiring the segmentation algorithms to incorporate prior knowledge about the geometry of the gland. In this paper, a novel approach to the problem of segmenting the TRUS images is described. The proposed approach is based on the concept of distribution tracking, which provides a unified framework for modeling and fusing image-related and morphological features of the prostate. Moreover, the same framework allows the segmentation to be regularized via using a new type of "weak" shape priors, which minimally bias the estimation procedure, while rendering the latter stable and robust.
Medical imaging based prostate cancer diagnosis procedure uses intra-operative transrectal ultrasound (TRUS) imaging to visualize the prostate shape and location to collect tissue samples. Correct tissue sampling from prostate requires accurate prostate segmentation in TRUS images. To achieve this, this study uses a novel residual connection based fully convolutional network. The advantage of this segmentation technique is that it requires no pre-processing of TRUS images to perform the segmentation. Thus, it offers a faster and straightforward prostate segmentation from TRUS images. Results s…
The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；The Hong Kong Polytechnic University（香港理工大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Boundary incompleteness raises great challenges to automatic prostate segmentation in ultrasound images. Shape prior can provide strong guidance in estimating the missing boundary, but traditional shape models often suffer from hand-crafted descriptors and local information loss in the fitting procedure. In this paper, we attempt to address those issues with a novel framework. The proposed framework can seamlessly integrate feature extraction and shape prior exploring, and estimate the complete boundary with a sequential manner. Our framework is composed of three key modules. Firstly, we seria…
Queen’s University（女王大学）；University of British Columbia（不列颠哥伦比亚大学）；Exact Imaging（Exact Imaging公司）
Deep learning-based analysis of high-frequency, high-resolution micro-ultrasound data shows great promise for prostate cancer detection. Previous approaches to analysis of ultrasound data largely follow a supervised learning paradigm. Ground truth labels for ultrasound images used for training deep networks often include coarse annotations generated from the histopathological analysis of tissue samples obtained via biopsy. This creates inherent limitations on the availability and quality of labeled data, posing major challenges to the success of supervised learning methods. On the other hand, …
Queen’s University（女王大学）；University of British Columbia（不列颠哥伦比亚大学）；Exact Imaging
MOTIVATION: Detection of prostate cancer during transrectal ultrasound-guided biopsy is challenging. The highly heterogeneous appearance of cancer, presence of ultrasound artefacts, and noise all contribute to these difficulties. Recent advancements in high-frequency ultrasound imaging - micro-ultrasound - have drastically increased the capability of tissue imaging at high resolution. Our aim is to investigate the development of a robust deep learning model specifically for micro-ultrasound-guided prostate cancer biopsy. For the model to be clinically adopted, a key challenge is to design a so…
In this work we propose to segment the prostate on a challenging dataset of trans-rectal ultrasound (TRUS) images using convolutional neural networks (CNNs) and statistical shape models (SSMs). TRUS is commonly used for a number of image-guided interventions on the prostate. Fast and accurate segmentation on the organ in these images is crucial to planning and fusion with other modalities such as magnetic resonance images (MRIs) . However, TRUS has limited soft tissue contrast and signal to noise ratio which makes the task of segmenting the prostate challenging and subject to inter-observer an…
Siemens Healthineers（西门子医疗）；Imperial College London（帝国理工学院）
Real-time localization of prostate gland in trans-rectal ultrasound images is a key technology that is required to automate the ultrasound guided prostate biopsy procedures. In this paper, we propose a new deep learning based approach which is aimed at localizing several prostate landmarks efficiently and robustly. We propose a multitask learning approach primarily to make the overall algorithm more contextually aware. In this approach, we not only consider the explicit learning of landmark locations, but also build-in a mechanism to learn the contour of the prostate. This multitask learning i…
Stanford University（斯坦福大学）；University College London（伦敦大学学院）；University of California Los Angeles（加利福尼亚大学洛杉矶分校）
Prostate biopsy and image-guided treatment procedures are often performed under the guidance of ultrasound fused with magnetic resonance images (MRI). Accurate image fusion relies on accurate segmentation of the prostate on ultrasound images. Yet, the reduced signal-to-noise ratio and artifacts (e.g., speckle and shadowing) in ultrasound images limit the performance of automated prostate segmentation techniques and generalizing these methods to new image domains is inherently difficult. In this study, we address these challenges by introducing a novel 2.5D deep neural network for prostate segm…
Institute of Cancer Research（癌症研究所）；University College London（伦敦大学学院）
Effective transperineal ultrasound image guidance in prostate external beam radiotherapy requires consistent alignment between probe and prostate at each session during patient set-up. Probe placement and ultrasound image inter-pretation are manual tasks contingent upon operator skill, leading to interoperator uncertainties that degrade radiotherapy precision. We demonstrate a method for ensuring accurate probe placement through joint classification of images and probe position data. Using a multi-input multi-task algorithm, spatial coordinate data from an optically tracked ultrasound probe is…
A Narrative Review of Image Processing Techniques Related to Prostate Ultrasound
School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University（深圳大学医学部生物医学工程学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学系）
Queen’s University（女王大学）；University of British Columbia（不列颠哥伦比亚大学）；Exact Imaging（Exact Imaging公司）
A large body of previous machine learning methods for ultrasound-based prostate cancer detection classify small regions of interest (ROIs) of ultrasound signals that lie within a larger needle trace corresponding to a prostate tissue biopsy (called biopsy core). These ROI-scale models suffer from weak labeling as histopathology results available for biopsy cores only approximate the distribution of cancer in the ROIs. ROI-scale models do not take advantage of contextual information that are normally considered by pathologists, i.e. they do not consider information about surrounding tissue and …
3D EAGAN: 3D edge-aware attention generative adversarial network for prostate segmentation in transrectal ultrasound images
Nanchang Hangkong University（南昌航空大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；The First Affiliated Hospital of Nanchang University（南昌大学第一附属医院）
OpenPros: A Large-Scale Dataset for Limited View Prostate Ultrasound Computed Tomography
The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Johns Hopkins University（约翰霍普金斯大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；QT Imaging, Inc.（QT影像公司）；National Institutes of Health（美国国立卫生研究院）；Iowa State University（爱荷华州立大学）；Google DeepMind（谷歌DeepMind）
Mask Enhanced Deeply Supervised Prostate Cancer Detection on B-mode Micro-Ultrasound
Stanford University（斯坦福大学）；The Catholic University of Korea（韩国天主教大学）；University of Alberta（阿尔伯塔大学）；University of California Los Angeles（加州大学洛杉矶分校）；Humanitas University（乌尼塔斯大学）；IRCCS Humanitas Research Hospital（IRCCS乌尼塔斯研究医院）；University of Florida（佛罗里达大学）；University of California San Diego（加州大学圣地亚哥分校）
Deep Attentive Features for Prostate Segmentation in 3D Transrectal Ultrasound
Shenzhen University（深圳大学）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Sun Yat-Sen University（中山大学）；the First Affiliated Hospital（第一附属医院）
Weakly Supervised Spatial Grounding for Discriminative Attention-Based Ultrasound-Histopathology Alignment in Prostate Cancer Grading
University of British Columbia（不列颠哥伦比亚大学）；Vector Institute（向量研究所）；Ordensklinikum Linz（林茨教团医院）；University of Alberta（阿尔伯塔大学）；Exact Imaging（Exact Imaging公司）
Learning Prostate Anatomy at Test Time for Cancer Detection in Micro-Ultrasound
University of British Columbia（不列颠哥伦比亚大学）；Vector Institute（向量研究所）；Ordensklinikum Linz（林茨 Ordenklinikum 医院）；University of Alberta（阿尔伯塔大学）；Exact Imaging（Exact Imaging 公司）
Towards Multi-modality Fusion and Prototype-based Feature Refinement for Clinically Significant Prostate Cancer Classification in Transrectal Ultrasound
School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University（深圳大学医学部生物医学工程学院）；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）
Hybrid-View Attention Network for Clinically Significant Prostate Cancer Classification in Transrectal Ultrasound
Smart Medical Imaging, Learning and Engineering (SMILE) Lab（智能医学影像、学习与工程实验室）；Medical UltraSound Image Computing (MUSIC) Lab（超声影像计算（MUSIC）实验室）；School of Biomedical Engineering（生物医学工程学院）；Shenzhen University Medical School（深圳大学医学院）；Shenzhen University（深圳大学）；The Department of Ultrasound（超声科）；State Key Laboratory of Oncology in South China（南方肿瘤学关键实验室）；Guangdong Provincial Clinical Research Center for Cancer（广东省癌症临床研究中心）；Sun Yat-sen University Cancer Center（中山大学肿瘤中心）
Distilling Temporal Coherence into 2D Networks for Transrectal Ultrasound Prostate Video Segmentation
Interdisciplinary Program in Bioengineering, Seoul National University（首尔大学生物工程跨学科项目）；Department of Transdisciplinary Medicine, Seoul National University Hospital（首尔大学医院跨学科医学系）；Department of Artificial Intelligence, Yonsei University（延世大学人工智能系）；Department of Radiology, Seoul National University Hospital（首尔大学医院放射科）
Northumbria University at Newcastle（纽卡斯尔诺森比亚大学）；College of Computer and Information Sciences, King Saud University（沙特国王大学计算机与信息科学学院）
Colorectal and prostate cancers are the most common types of cancer in men worldwide. To diagnose colorectal and prostate cancer, a pathologist performs a histological analysis on needle biopsy samples. This manual process is time-consuming and error-prone, resulting in high intra and interobserver variability, which affects diagnosis reliability. This study aims to develop an automatic computerized system for diagnosing colorectal and prostate tumors by using images of biopsy samples to reduce time and diagnosis error rates associated with human analysis. We propose a CNN model for classifyin…
Université J. Fourier（傅里叶大学）；CNRS（法国国家科学研究中心）；Koelis SAS（Koelis SAS公司）；La Pitié-Salpêtrière Hospital（皮蒂耶-萨尔佩特里埃医院）
Transrectal biopsies under 2D ultrasound (US) control are the current clinical standard for prostate cancer diagnosis. The isoechogenic nature of prostate carcinoma makes it necessary to sample the gland systematically, resulting in a low sensitivity. Also, it is difficult for the clinician to follow the sampling protocol accurately under 2D US control and the exact anatomical location of the biopsy cores is unknown after the intervention. Tracking systems for prostate biopsies make it possible to generate biopsy distribution maps for intra- and post-interventional quality control and 3D visua…
Influence of High-Performance Image-to-Image Translation Networks on Clinical Visual Assessment and Outcome Prediction: Utilizing Ultrasound to MRI Translation in Prostate Cancer
Samrat Ashok Technological Institute（萨姆拉特·阿肖克技术学院）；Department of Trg. & Technical Education, Govt. of NCT Delhi（德里国家首都辖区政府培训与技术教育局）
Digital image plays a vital role in the early detection of cancers, such as prostate cancer, breast cancer, lungs cancer, cervical cancer. Ultrasound imaging method is also suitable for early detection of the abnormality of fetus. The accurate detection of region of interest in ultrasound image is crucial. Since the result of reflection, refraction and deflection of ultrasound waves from different types of tissues with different acoustic impedance. Usually, the contrast in ultrasound image is very low and weak edges make the image difficult to identify the fetus region in the ultrasound image.…
Clinically significant prostate cancer has a better chance to be sampled during ultrasound-guided biopsy procedures, if suspected lesions found in pre-operative magnetic resonance (MR) images are used as targets. However, the diagnostic accuracy of the biopsy procedure is limited by the operator-dependent skills and experience in sampling the targets, a sequential decision making process that involves navigating an ultrasound probe and placing a series of sampling needles for potentially multiple targets. This work aims to learn a reinforcement learning (RL) policy that optimises the actions o…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National Institutes of Health（美国国立卫生研究院）；National Cancer Institute（美国国家癌症研究所）
Prostate cancer biopsy benefits from accurate fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images. In the past few years, convolutional neural networks (CNNs) have been proved powerful in extracting image features crucial for image registration. However, challenging applications and recent advances in computer vision suggest that CNNs are quite limited in its ability to understand spatial correspondence between features, a task in which the self-attention mechanism excels. This paper aims to develop a self-attention mechanism specifically for cross-modal image registrati…
Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate Segmentation in TRUS Images
University College London（伦敦大学学院）；UCL Hospital NHS Foundation Trust（伦敦大学学院医院NHS基金会信托）；Wellcome EPSRC Centre for Interventional and Surgical Sciences（惠康EPSRC介入与外科科学中心）；Centre for Medical Image Computing（医学影像计算中心）；Department of Medical Physics and Biomedical Engineering（医学物理与生物医学工程系）；Division of Surgery & Interventional Science（外科与介入科学部）；Department of Urology（泌尿外科）
The Chinese University of Hong Kong（香港中文大学）；University of Cambridge（剑桥大学）；Inception Institute of Artificial Intelligence（启迪人工智能研究院）；The Hong Kong University of Science and Technology（香港科技大学）；Jinan University（暨南大学）；Southern University of Science and Technology（南方科技大学）；Chinese Academy of Sciences（中国科学院）
Automatic breast lesion segmentation in ultrasound helps to diagnose breast cancer, which is one of the dreadful diseases that affect women globally. Segmenting breast regions accurately from ultrasound image is a challenging task due to the inherent speckle artifacts, blurry breast lesion boundaries, and inhomogeneous intensity distributions inside the breast lesion regions. Recently, convolutional neural networks (CNNs) have demonstrated remarkable results in medical image segmentation tasks. However, the convolutional operations in a CNN often focus on local regions, which suffer from limit…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National Institutes of Health（美国国立卫生研究院）
Fusing intra-operative 2D transrectal ultrasound (TRUS) image with pre-operative 3D magnetic resonance (MR) volume to guide prostate biopsy can significantly increase the yield. However, such a multimodal 2D/3D registration problem is a very challenging task. In this paper, we propose an end-to-end frame-to-volume registration network (FVR-Net), which can efficiently bridge the previous research gaps by aligning a 2D TRUS frame with a 3D TRUS volume without requiring hardware tracking. The proposed FVR-Net utilizes a dual-branch feature extraction module to extract the information from TRUS fr…
HFS-TriNet: A Three-Branch Collaborative Feature Learning Network for Prostate Cancer Classification from TRUS Videos
Guangdong Polytechnic Normal University（广东 polytechnic 正规大学）；Guangdong Provincial Key Laboratory of Intellectual Property & Big Data（广东省知识产权与大数据重点实验室）；The First Affiliated Hospital of Jinan University（暨南大学第一附属医院）；Shenzhen People’s Hospital（深圳人民医院）
Eindhoven University of Technology（埃因霍温理工大学）；Shandong University of Science and Technology（山东科技大学）；Philips Research（飞利浦研究院）
Medical instrument detection is essential for computer-assisted interventions since it would facilitate the surgeons to find the instrument efficiently with a better interpretation, which leads to a better outcome. This article reviews medical instrument detection methods in the ultrasound-guided intervention. First, we present a comprehensive review of instrument detection methodologies, which include traditional non-data-driven methods and data-driven methods. The non-data-driven methods were extensively studied prior to the era of machine learning, i.e. data-driven approaches. We discuss th…
Adapting Vision-Language Foundation Model for Next Generation Medical Ultrasound Image Analysis
Department of Health Technology and Informatics, The Hong Kong Polytechnic University（健康科技与信息学系，香港理工大学）；Centre for Smart Health and School of Nursing, The Hong Kong Polytechnic University（智能健康中心及护理学院，香港理工大学）；Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong（影像与介入放射学系，香港中文大学）；Suzhou Hospital of Traditional Chinese Medicine Affiliated to Nanjing University of Chinese Medicine（南京中医药大学附属苏州中医医院）；Department of Ultrasound, The Affiliated Changzhou No. 2 People's Hospital of Nanjing Medical University（南京医科大学附属常州第二人民医院超声科）
Multiscale Switch for Semi-Supervised and Contrastive Learning in Medical Ultrasound Image Segmentation
Department of Health Technology and Informatics, The Hong Kong Polytechnic University（健康科技与信息学系，香港理工大学）；Centre for Smart Health and School of Nursing, The Hong Kong Polytechnic University（智能健康中心及护理学院，香港理工大学）；Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong（影像与介入放射科，香港中文大学）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National Institutes of Health（美国国立卫生研究院）；Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）
Robust and accurate alignment of multimodal medical images is a very challenging task, which however is very useful for many clinical applications. For example, magnetic resonance (MR) and transrectal ultrasound (TRUS) image registration is a critical component in MR-TRUS fusion guided prostate interventions. However, due to the huge difference between the image appearances and the large variation in image correspondence, MR-TRUS image registration is a very challenging problem. In this paper, an adversarial image registration (AIR) framework is proposed. By training two deep neural networks s…
Rensselaer Polytechnic Institute（伦斯勒理工学院）；National Institutes of Health（美国国立卫生研究院）
Transrectal ultrasound (US) is the most commonly used imaging modality to guide prostate biopsy and its 3D volume provides even richer context information. Current methods for 3D volume reconstruction from freehand US scans require external tracking devices to provide spatial position for every frame. In this paper, we propose a deep contextual learning network (DCL-Net), which can efficiently exploit the image feature relationship between US frames and reconstruct 3D US volumes without any tracking device. The proposed DCL-Net utilizes 3D convolutions over a US video segment for feature extra…
University College London（伦敦大学学院）；University of Leeds（利兹大学）
Biomechanical modelling of soft tissue provides a non-data-driven method for constraining medical image registration, such that the estimated spatial transformation is considered biophysically plausible. This has not only been adopted in real-world clinical applications, such as the MR-to-ultrasound registration for prostate intervention of interest in this work, but also provides an explainable means of understanding the organ motion and spatial correspondence establishment. This work instantiates the recently-proposed physics-informed neural networks (PINNs) to a 3D linear elastic model for …
Salient Region Matching for Fully Automated MR-TRUS Registration
School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；Medical UltraSound Image Computing (MUSIC) Lab（医学超声影像计算实验室）；Smart Medical Imaging, Learning and Engineering (SMILE) Lab（智能医学影像、学习与工程实验室）
University College London（伦敦大学学院）；University of Oxford（牛津大学）
In this paper, we describe how a patient-specific, ultrasound-probe-induced prostate motion model can be directly generated from a single preoperative MR image. Our motion model allows for sampling from the conditional distribution of dense displacement fields, is encoded by a generative neural network conditioned on a medical image, and accepts random noise as additional input. The generative network is trained by a minimax optimisation with a second discriminative neural network, tasked to distinguish generated samples from training motion data. In this work, we propose that 1) jointly optim…
FRAMOS GmbH（FRAMOS有限公司）；Johns Hopkins University（约翰斯·霍普金斯大学）；Technische Universität München（慕尼黑工业大学）
Tracking of rotation and translation of medical instruments plays a substantial role in many modern interventions. Traditional external optical tracking systems are often subject to line-of-sight issues, in particular when the region of interest is difficult to access or the procedure allows only for limited rigid body markers. The introduction of inside-out tracking systems aims to overcome these issues. We propose a marker-less tracking system based on visual SLAM to enable tracking of instruments in an interventional scenario. To achieve this goal, we mount a miniature multi-modal (monocula…
University of Waterloo（滑铁卢大学）；Suez Canal University（苏伊士运河大学）
Quantifying the accuracy of segmentation and manual delineation of organs, tissue types and tumors in medical images is a necessary measurement that suffers from multiple problems. One major shortcoming of all accuracy measures is that they neglect the anatomical significance or relevance of different zones within a given segment. Hence, existing accuracy metrics measure the overlap of a given segment with a ground-truth without any anatomical discrimination inside the segment. For instance, if we understand the rectal wall or urethral sphincter as anatomical zones, then current accuracy measu…
