Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks
Wellcome / EPSRC Centre for Interventional and Surgical Sciences, University College London（伦敦大学学院威康/EPSRC介入与外科学中心）；UCL Centre for Medical Image Computing, University College London（伦敦大学学院医学图像计算中心）；Department of Gastroenterology, University College London Hospital（伦敦大学学院医院消化内科）；Division of Surgery and Interventional Science, University College London（伦敦大学学院外科与介入科学部）；Institute for Liver and Digestive Health, University College London（伦敦大学学院肝脏与消化健康研究所）
Ester Bonmati, Yipeng Hu, Alexander Grimwood, Gavin J. Johnson, George Goodchild, Margaret G. Keane, Kurinchi Gurusamy, Brian Davidson, Matthew J. Clarkson, Ste… 展开作者
Ester Bonmati, Yipeng Hu, Alexander Grimwood, Gavin J. Johnson, George Goodchild, Margaret G. Keane, Kurinchi Gurusamy, Brian Davidson, Matthew J. Clarkson, Stephen P. Pereira, Dean C. Barratt
Ultrasound imaging is a commonly used technology for visualising patient anatomy in real-time during diagnostic and therapeutic procedures. High operator dependency and low reproducibility make ultrasound imaging and interpretation challenging with a steep learning curve. Automatic image classification using deep learning has the potential to overcome some of these challenges by supporting ultrasound training in novices, as well as aiding ultrasound image interpretation in patient with complex pathology for more experienced practitioners. However, the use of deep learning methods requires a large amount of data in order to provide accurate results. Labelling large ultrasound datasets is a challenging task because labels are retrospectively assigned to 2D images without the 3D spatial context available in vivo or that would be inferred while visually tracking structures between frames during the procedure. In this work, we propose a multi-modal convolutional neural network (CNN) architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. We use a CNN composed of two branches, one for voice data and another for image data, which are joined to predict image labels from the spoken names of anatomical landmarks. The network was trained using recorded verbal comments from expert operators. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels. We conclude that the addition of spoken commentaries can increase the performance of ultrasound image classification, and eliminate the burden of manually labelling large EUS datasets necessary for deep learning applications.
School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Pancreas Centre, First Affiliated Hospital, Nanjing Medical University（南京医科大学第一附属医院胰腺中心）；Department of General Surgery, First Affiliated Hospital, Nanjing Medical University（南京医科大学第一附属医院普外科）；Big Data Computing Center of Southeast University（东南大学大数据计算中心）
Université Pierre et Marie Curie（皮埃尔和玛丽·居里大学）；SIGnal processing and MAchine learning Lab, ESPCI ParisTech, PSL Research University（巴黎高等物理化工学院信号处理与机器学习实验室，巴黎文理研究大学）；Vocal Tract Visualization Lab, University of Maryland Dental School（马里兰大学牙科学院声道可视化实验室）
Studying tongue motion during speech using ultrasound is a standard procedure, but automatic ultrasound image labelling remains a challenge, as standard tongue shape extraction methods typically require human intervention. This article presents a method based on deep neural networks to automatically extract tongue contour from ultrasound images on a speech dataset. We use a deep autoencoder trained to learn the relationship between an image and its related contour, so that the model is able to automatically reconstruct contours from the ultrasound image alone. In this paper, we use an automati…
TextSAM-EUS: Text Prompt Learning for SAM to Accurately Segment Pancreatic Tumor in Endoscopic Ultrasound
Concordia University（康科迪亚大学）；Jewish General Hospital（犹太总医院）；McGill University Faculty of Medicine（麦吉尔大学医学院）；Lady Davis Institute for Medical Research（拉德克利夫医学研究 institute）
Automatic Endoscopic Ultrasound Station Recognition with Limited Data
Amrita Center for Cybersecurity Systems & Networks, Amrita Vishwa Vidyapeetham（阿姆里塔网络安全系统与网络中心，阿姆里塔大学）；Amrita Institute of Medical Sciences（阿姆里塔医学科学研究所）
Multistep Automated Data Labelling Procedure (MADLaP) for Thyroid Nodules on Ultrasound: An Artificial Intelligence Approach for Automating Image Annotation
Duke University（杜克大学）；Duke University Medical Center（杜克大学医学中心）
Deep learning models with large learning capacities often overfit to medical imaging datasets. This is because training sets are often relatively small due to the significant time and financial costs incurred in medical data acquisition and labelling. Data augmentation is therefore often used to expand the availability of training data and to increase generalization. However, augmentation strategies are often chosen on an ad-hoc basis without justification. In this paper, we present an augmentation policy search method with the goal of improving model classification performance. We include in …
Ultromics Ltd.（奥特罗米克斯有限公司）；Metalynx Ltd.（梅塔林克斯有限公司）；Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）
Analysis of cardiac ultrasound images is commonly performed in routine clinical practice for quantification of cardiac function. Its increasing automation frequently employs deep learning networks that are trained to predict disease or detect image features. However, such models are extremely data-hungry and training requires labelling of many thousands of images by experienced clinicians. Here we propose the use of contrastive learning to mitigate the labelling bottleneck. We train view classification models for imbalanced cardiac ultrasound datasets and show improved performance for views/cl…
Government Medical College, Thiruvananthapuram（特里凡得琅政府医学院）；Hospital Universitario Puerta de Hierro（普埃尔塔德耶罗大学医院）
Ultrasound is fast becoming an inevitable diagnostic tool for regular and continuous monitoring of the lung with the recent outbreak of COVID-19. In this work, a novel approach is presented to extract acoustic propagation-based features to automatically highlight the region below pleura, which is an important landmark in lung ultrasound (LUS). Subsequently, a multichannel input formed by using the acoustic physics-based feature maps is fused to train a neural network, referred to as LUSNet, to classify the LUS images into five classes of varying severity of lung infection to track the progress…
University College London（伦敦大学学院）；InstaDeep（InstaDeep公司）；Stanford School of Medicine（斯坦福医学院）
In this paper, we consider a type of image quality assessment as a task-specific measurement, which can be used to select images that are more amenable to a given target task, such as image classification or segmentation. We propose to train simultaneously two neural networks for image selection and a target task using reinforcement learning. A controller network learns an image selection policy by maximising an accumulated reward based on the target task performance on the controller-selected validation set, whilst the target task predictor is optimised using the training set. The trained con…
Compute-Optimal Network Design for Echocardiography Myocardial Segmentation and Perfusion Quantification using Neural Scaling Laws
Department of Bioengineering, Imperial College London（生物工程系，帝国理工学院伦敦分校）；National Heart and Lung Institute, Imperial College London（国家心脏和肺 institute，帝国理工学院伦敦分校）；Guy’s and St. Thomas’ NHS Foundation Trust（圣泰莫斯国家健康服务信托基金）
Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健服务基金会信托）；City, University of London（伦敦城市大学）；Friedrich–Alexander University Erlangen–Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
Probably yes. -- Supervised Deep Learning dominates performance scores for many computer vision tasks and defines the state-of-the-art. However, medical image analysis lags behind natural image applications. One of the many reasons is the lack of well annotated medical image data available to researchers. One of the first things researchers are told is that we require significant expertise to reliably and accurately interpret and label such data. We see significant inter- and intra-observer variability between expert annotations of medical images. Still, it is a widely held assumption that nov…
Three-Dimensional Anatomical Data Generation Based on Artificial Neural Networks
Division of Smart Technologies for Tumor Therapy, German Cancer Research Center (DKFZ) Site Dresden（智能肿瘤治疗技术部，德国癌症研究中心（DKFZ）德累斯顿分部）；Faculty of Computer Science, Dresden University of Technology（计算机科学系，德累斯顿技术大学）；Department of Urology, Faculty of Medicine, University of Freiburg - Medical Centre（泌尿科，弗赖堡大学医学中心医学院）；Division of Translational Surgical Oncology, National Center for Tumor Diseases (NCT/UCC) Dresden（转化外科肿瘤学部，国家肿瘤疾病中心（NCT/UCC）德累斯顿分部）；Faculty of Medicine Carl Gustav Carus, Dresden University of Technology（卡尔·古斯塔夫·卡鲁斯医学院，德累斯顿技术大学）；Faculty of Electrical and Computer Engineering, Dresden University of Technology（电气与计算机工程系，德累斯顿技术大学）
Deep learning models, e.g. supervised Encoder-Decoder style networks, exhibit promising performance in medical image segmentation, but come with a high labelling cost. We propose TriSegNet, a semi-supervised semantic segmentation framework. It uses triple-view feature learning on a limited amount of labelled data and a large amount of unlabeled data. The triple-view architecture consists of three pixel-level classifiers and a low-level shared-weight learning module. The model is first initialized with labelled data. Label processing, including data perturbation, confidence label voting and unc…
Queensland University of Technology（昆士兰科技大学）；University of Jaen（哈恩大学）；The University of Queensland（昆士兰大学）；University of Sumer（苏美尔大学）；University of Baghdad（巴格达大学）；The University of Information Technology & Communications（信息与通信技术大学）；University of Missouri（密苏里大学）
Deep Learning (DL) requires a large amount of training data to provide quality outcomes. However, the field of medical imaging suffers from the lack of sufficient data for properly training DL models because medical images require manual labelling carried out by clinical experts thus the process is time-consuming, expensive, and error-prone. Recently, transfer learning (TL) was introduced to reduce the need for the annotation procedure by means of transferring the knowledge performed by a previous task and then fine-tuning the result using a relatively small dataset. Nowadays, multiple classif…
A Point-Neighborhood Learning Framework for Nasal Endoscope Image Segmentation
School of Intelligent Engineering, Sun Yat-Sen University-Shenzhen Campus（中山大学深圳校区智能工程学院）；Department of Otolaryngology, The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院耳鼻喉科）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）
