We design a framework based on Mask Region-based Convolutional Neural Network (Mask R-CNN) to automatically detect and separately extract anatomical components of mosquitoes - thorax, wings, abdomen and legs from images. Our training dataset consisted of 1500 smartphone images of nine mosquito species trapped in Florida. In the proposed technique, the first step is to detect anatomical components within a mosquito image. Then, we localize and classify the extracted anatomical components, while simultaneously adding a branch in the neural network architecture to segment pixels containing only the anatomical components. Evaluation results are favorable. To evaluate generality, we test our architecture trained only with mosquito images on bumblebee images. We again reveal favorable results, particularly in extracting wings. Our techniques in this paper have practical applications in public health, taxonomy and citizen-science efforts.
The Education University of Hong Kong（香港教育大学）；Shanghai University of Engineering Science（上海工程技术大学）
Accurate hand joints detection from images is a fundamental topic which is essential for many applications in computer vision and human computer interaction. This paper presents a two stage network for hand joints detection from single unmarked image by using serial-parallel multi-scale feature fusion. In stage I, the hand regions are located by a pre-trained network, and the features of each detected hand region are extracted by a shallow spatial hand features representation module. The extracted hand features are then fed into stage II, which consists of serially connected feature extraction…
University of Oxford（牛津大学）；SRI International（SRI国际）；Mind Foundry Ltd（Mind Foundry有限公司）；University of Surrey（萨里大学）；IHI Tanzania（坦桑尼亚IHI）；UCC, BEES（科克大学学院BEES）
This paper presents the first large-scale multi-species dataset of acoustic recordings of mosquitoes tracked continuously in free flight. We present 20 hours of audio recordings that we have expertly labelled and tagged precisely in time. Significantly, 18 hours of recordings contain annotations from 36 different species. Mosquitoes are well-known carriers of diseases such as malaria, dengue and yellow fever. Collecting this dataset is motivated by the need to assist applications which utilise mosquito acoustics to conduct surveys to help predict outbreaks and inform intervention policy. The t…
Convolutional neural networks demonstrated outstanding empirical results in computer vision and speech recognition tasks where labeled training data is abundant. In medical imaging, there is a huge variety of possible imaging modalities and contrasts, where annotated data is usually very scarce. We present two approaches to deal with this challenge. A network pretrained in a different domain with abundant data is used as a feature extractor, while a subsequent classifier is trained on a small target dataset; and a deep architecture trained with heavy augmentation and equipped with sophisticate…
Interpretable Rheumatoid Arthritis Scoring via Anatomy-aware Multiple Instance Learning
Big Data Institute, Nuffield Department of Population Health, University of Oxford, Oxford, UK（大数据研究所，人口健康系，牛津大学，牛津，英国）；Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK（骨科、风湿病学和肌肉骨骼科学系，牛津大学，牛津，英国）
Hangzhou Dianzi University（杭州电子科技大学）；University of Bern（伯尔尼大学）；Sichuan University（四川大学）；Shanghai Jiao Tong University（上海交通大学）；Waseda University（早稻田大学）
Accurate evaluation of the treatment result on X-ray images is a significant and challenging step in root canal therapy since the incorrect interpretation of the therapy results will hamper timely follow-up which is crucial to the patients' treatment outcome. Nowadays, the evaluation is performed in a manual manner, which is time-consuming, subjective, and error-prone. In this paper, we aim to automate this process by leveraging the advances in computer vision and artificial intelligence, to provide an objective and accurate method for root canal therapy result assessment. A novel anatomy-guid…
Yale University（耶鲁大学）；Yale School of Medicine（耶鲁大学医学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；Chinese Academy of Sciences（中国科学院）
Multimodal image registration has many applications in diagnostic medical imaging and image-guided interventions, such as Transcatheter Arterial Chemoembolization (TACE) of liver cancer guided by intraprocedural CBCT and pre-operative MR. The ability to register peri-procedurally acquired diagnostic images into the intraprocedural environment can potentially improve the intra-procedural tumor targeting, which will significantly improve therapeutic outcomes. However, the intra-procedural CBCT often suffers from suboptimal image quality due to lack of signal calibration for Hounsfield unit, limi…
University of Pittsburgh（匹兹堡大学）；Boston University（波士顿大学）
Large-scale volumetric medical images with annotation are rare, costly, and time prohibitive to acquire. Self-supervised learning (SSL) offers a promising pre-training and feature extraction solution for many downstream tasks, as it only uses unlabeled data. Recently, SSL methods based on instance discrimination have gained popularity in the medical imaging domain. However, SSL pre-trained encoders may use many clues in the image to discriminate an instance that are not necessarily disease-related. Moreover, pathological patterns are often subtle and heterogeneous, requiring the ability of the…
F2PASeg: Feature Fusion for Pituitary Anatomy Segmentation in Endoscopic Surgery
1Centre for Artificial Intelligence；Robotics, Hong Kong Institute of Science \& Innovation, Chinese Academy of Sciences 2The University of Hong Kong 3Peking Union Medical College Hospital, Chinese Academy of Medical Science
Segmentation has been a major task in neuroimaging. A large number of automated methods have been developed for segmenting healthy and diseased brain tissues. In recent years, deep learning techniques have attracted a lot of attention as a result of their high accuracy in different segmentation problems. We present a new deep learning based segmentation method, DeepMRSeg, that can be applied in a generic way to a variety of segmentation tasks. The proposed architecture combines recent advances in the field of biomedical image segmentation and computer vision. We use a modified UNet architectur…
Anatomy-Informed Deep Learning and Radiomics for Automated Neurofibroma Segmentation in Whole-Body MRI
Institute for Applied Medical Informatics（应用医学信息学研究所）；Institute of Computational Neuroscience（计算神经科学研究所）；Center for Biomedical Artificial Intelligence (bAIome)（生物医学人工智能中心）；University Medical Center Hamburg-Eppendorf（汉堡埃彭多夫大学医学中心）；Department of Diagnostic and Interventional Radiology and Nuclear Medicine（诊断与介入放射学及核医学部门）；Department of Neurology（神经内科部门）
Anatomy-Guided Representation Learning Using a Transformer-Based Network for Thyroid Nodule Segmentation in Ultrasound Images
1 Department of Computer Science, Hanyang University, Seoul, 04762, South Korea；2 Department of Computer Science, The University of Alabama, Seoul, 04762, South Korea；3 Department of Biomedical Sciences, Seoul National University, Seoul, 08826, South Korea ( )；4 Department of Anesthesiology, Perioperative；Pain Medicine, Stanford University, CA 94305, USA ( )；5 Department of Computer Science, Hanyang University, Seoul, 04762, South Korea ( )；6 Department of Computer Engineering, Qatar University, Doha, Qatar ( )
University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Lab（上海人工智能实验室）；SenseTime Research（商汤科技研究院）；Shanghai Jiao Tong University（上海交通大学）；ShanghaiTech University（上海科技大学）；West China Hospital, Sichuan University（四川大学华西医院）
Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance for medical image segmentation, yet need plenty of manual annotations for training. Semi-Supervised Learning (SSL) methods are promising to reduce the requirement of annotations, but their performance is still limited when the dataset size and the number of annotated images are small. Leveraging existing annotated datasets with similar anatomical structures to assist training has a potential for improving the model's performance. However, it is further challenged by the cross-anatomy domain shift due to the differe…
Beijing University of Posts and Telecommunications（北京邮电大学）；Peking University（北京大学）；Tsinghua University（清华大学）
Automated radiology report generation aims at automatically generating a detailed description of medical images, which can greatly alleviate the workload of radiologists and provide better medical services to remote areas. Most existing works pay attention to the holistic impression of medical images, failing to utilize important anatomy information. However, in actual clinical practice, radiologists usually locate important anatomical structures, and then look for signs of abnormalities in certain structures and reason the underlying disease. In this paper, we propose a novel framework AGFNet…
Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for anomaly screening. For this ultrasound (US) is employed. While expert sonographers are adept at reading US images, MR images are much easier for non-experts to interpret. Hence in this paper we seek to produce images with MRI-like appearance directly from clinical US images. Our own clinical motivation is to seek a way to communicate US findings to patients or clinical professionals unfamiliar with US, but in medical image analysis such a capability is potentially useful, for in…
Higher Institute of Computer Science and Multimedia（计算机科学与多媒体高等学院）；National Engineering School Tunis（突尼斯国立工程学院）
We present in this paper a new approach for hand gesture analysis that allows digit recognition. The analysis is based on extracting a set of features from a hand image and then combining them by using an induction graph. The most important features we extract from each image are the fingers locations, their heights and the distance between each pair of fingers. Our approach consists of three steps: (i) Hand detection and localization, (ii) fingers extraction and (iii) features identification and combination to digit recognition. Each input image is assumed to contain only one person, thus we …
PAII Inc.（PAII公司）；Stony Brook University（石溪大学）；Harvard University（哈佛大学）；Ping An Technology（平安科技）；Changhai Hospital（长海医院）
The pancreatic disease taxonomy includes ten types of masses (tumors or cysts)[20,8]. Previous work focuses on developing segmentation or classification methods only for certain mass types. Differential diagnosis of all mass types is clinically highly desirable [20] but has not been investigated using an automated image understanding approach. We exploit the feasibility to distinguish pancreatic ductal adenocarcinoma (PDAC) from the nine other nonPDAC masses using multi-phase CT imaging. Both image appearance and the 3D organ-mass geometry relationship are critical. We propose a holistic segme…
South China University of Technology（华南理工大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）
Ultrasonography is an important routine examination for breast cancer diagnosis, due to its non-invasive, radiation-free and low-cost properties. However, the diagnostic accuracy of breast cancer is still limited due to its inherent limitations. It would be a tremendous success if we can precisely diagnose breast cancer by breast ultrasound images (BUS). Many learning-based computer-aided diagnostic methods have been proposed to achieve breast cancer diagnosis/lesion classification. However, most of them require a pre-define ROI and then classify the lesion inside the ROI. Conventional classif…
NeuroABench: A Multimodal Evaluation Benchmark for Neurosurgical Anatomy Identification
Hong Kong Institute of Science and Innovation（香港科学与创新研究院）；The University of Hong Kong-Shenzhen Hospital（香港大学深圳医院）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）
OSCAR: Occupancy-based Shape Completion via Acoustic Neural Implicit Representations
Chair for Computer Aided Medical Procedures (CAMP) Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序教席（CAMP））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心（MCML））；Konrad Zuse School of Excellence in Reliable AI (relAI)（康拉德·楚泽可靠人工智能卓越学校（relAI））
ViT-NeBLa: A Hybrid Vision Transformer and Neural Beer-Lambert Framework for Single-View 3D Reconstruction of Oral Anatomy from Panoramic Radiographs
Artificial Intelligence and Image Processing Lab., Department of Information and Communication Engineering, Sun Moon University, South Korea（人工智能与图像处理实验室，信息与通信工程系，顺 Moon 大学，韩国）；Department of Physics and Engineering Physics, Tulane University, New Orleans, LA 70118, USA（物理系与工程物理系，路易斯安那大学，新奥尔良，路易斯安那州 70118，美国）
Three-dimensional cine-MRI is of crucial importance for assessing the cardiac function. Features that describe the anatomy and function of cardiac structures (e.g. Left Ventricle (LV), Right Ventricle (RV), and Myocardium(MC)) are known to have significant diagnostic value and can be computed from 3D cine-MR images. However, these features require precise segmentation of cardiac structures. Among the fully automated segmentation methods, Fully Convolutional Networks (FCN) with Skip Connections have shown robustness in medical segmentation problems. In this study, we develop a complete pipeline…
Tracing 3D Anatomy in 2D Strokes: A Multi-Stage Projection Driven Approach to Cervical Spine Fracture Identification
Department of Computer Science and Engineering, BRAC University（布拉格大学计算机科学与工程系）；Department of Electrical and Electronic Engineering, University of Dhaka（达卡大学电子与电气工程系）；Department of Electrical Engineering, Qatar University（卡塔尔大学电子工程系）；Department of Radiology, Hamad Medical Corporation（哈马德医疗中心放射科）
Statistical Hand Shape Modeling from Clinical CT Scans Using Deep Learning and Implicit Skinning
Dept. of Computer Science and Engineering（计算机科学与工程系）；Özyeğin University（奥兹耶尼大学）；Dept. of Artificial Intelligence and Data Engineering（人工智能与数据工程系）；Osteoid Inc.（Osteoid公司）
Oslo Metropolitan University（奥斯陆城市大学）；Norwegian University of Science and Technology（挪威科技大学）
In the brain, the structure of a network of neurons defines how these neurons implement the computations that underlie the mind and the behavior of animals and humans. Provided that we can describe the network of neurons as a graph, we can employ methods from graph theory to investigate its structure or use cellular automata to mathematically assess its function. Although, software for the analysis of graphs and cellular automata are widely available. Graph extraction from the image of networks of brain cells remains difficult. Nervous tissue is heterogeneous, and differences in anatomy may re…
Correspondence-based shape models are key to various medical imaging applications that rely on a statistical analysis of anatomies. Such shape models are expected to represent consistent anatomical features across the population for population-specific shape statistics. Early approaches for correspondence placement rely on nearest neighbor search for simpler anatomies. Coordinate transformations for shape correspondence hold promise to address the increasing anatomical complexities. Nonetheless, due to the inherent shape-level geometric complexity and population-level shape variation, the coor…
Deep Learning-Based Cross-Anatomy CT Synthesis Using Adapted nnResU-Net with Anatomical Feature Prioritized Loss
Univ. Bordeaux, CNRS, LaBRI, UMR 5800, F-33400 Talence, France（波尔多大学，法国国家科学研究中心，LaBRI研究院，UMR 5800）；Universidad Autónoma de Madrid, Madrid, 28049, Spain（马德里自治大学，西班牙）；Pazmany Peter Catholic University, Budapest, 1083, Hungary（佩斯大学，匈牙利）；RIKEN Center for Integrative Medical Sciences, Medical Data Deep Learning Team, Tokyo, Japan（日本再生医学科学中心，医学数据深度学习团队）
Statistical shape modeling (SSM) characterizes anatomical variations in a population of shapes generated from medical images. SSM requires consistent shape representation across samples in shape cohort. Establishing this representation entails a processing pipeline that includes anatomy segmentation, re-sampling, registration, and non-linear optimization. These shape representations are then used to extract low-dimensional shape descriptors that facilitate subsequent analyses in different applications. However, the current process of obtaining these shape descriptors from imaging data relies o…
Statistical shape modeling (SSM) has recently taken advantage of advances in deep learning to alleviate the need for a time-consuming and expert-driven workflow of anatomy segmentation, shape registration, and the optimization of population-level shape representations. DeepSSM is an end-to-end deep learning approach that extracts statistical shape representation directly from unsegmented images with little manual overhead. It performs comparably with state-of-the-art shape modeling methods for estimating morphologies that are viable for subsequent downstream tasks. Nonetheless, DeepSSM produce…
Trusted Multi-View Deep Learning Classification of Fetal Congenital Heart Disease with Feature-level and Decision-level Fusion
School of Biomedical Engineering, Shanghai Jiao Tong University（生物医学工程学院，上海交通大学）；Shanghai Key Laboratory of Embryo Original Diseases（上海胚胎原发疾病重点实验室）；Shanghai Municipal Key Clinical Specialty（上海市级重点临床专科）
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；University of Wisconsin Madison（威斯康星大学麦迪逊分校）
Automated anatomical labeling plays a vital role in coronary artery disease diagnosing procedure. The main challenge in this problem is the large individual variability inherited in human anatomy. Existing methods usually rely on the position information and the prior knowledge of the topology of the coronary artery tree, which may lead to unsatisfactory performance when the main branches are confusing. Motivated by the wide application of the graph neural network in structured data, in this paper, we propose a conditional partial-residual graph convolutional network (CPR-GCN), which takes bot…
Southeast University（东南大学）；Centre de Recherche en Information Biomédicale Sino-Français (CRIBs)（中法生物医学信息研究中心）；The First Affiliated Hospital of Nanjing Medical University（南京医科大学第一附属医院）；Univ Rennes, Inserm, LTSI - UMR1099（雷恩大学，法国国家健康与医学研究院，信号与图像处理实验室）；University of Western Ontario（西安大略大学）
Renal compartment segmentation on CT images targets on extracting the 3D structure of renal compartments from abdominal CTA images and is of great significance to the diagnosis and treatment for kidney diseases. However, due to the unclear compartment boundary, thin compartment structure and large anatomy variation of 3D kidney CT images, deep-learning based renal compartment segmentation is a challenging task. We propose a novel weakly supervised learning framework, Cycle Prototype Network, for 3D renal compartment segmentation. It has three innovations: 1) A Cycle Prototype Learning (CPL) is…
IBM Research – Almaden Research Center（IBM研究院阿尔马登研究中心）
Although deep learning can provide promising results in medical image analysis, the lack of very large annotated datasets confines its full potential. Furthermore, limited positive samples also create unbalanced datasets which limit the true positive rates of trained models. As unbalanced datasets are mostly unavoidable, it is greatly beneficial if we can extract useful knowledge from negative samples to improve classification accuracy on limited positive samples. To this end, we propose a new strategy for building medical image analysis pipelines that target disease detection. We train a disc…
University of Idaho（爱达荷大学）；University of Utah School of Medicine（犹他大学医学院）
Breast tumor segmentation is a critical task in computer-aided diagnosis (CAD) systems for breast cancer detection because accurate tumor size, shape and location are important for further tumor quantification and classification. However, segmenting small tumors in ultrasound images is challenging, due to the speckle noise, varying tumor shapes and sizes among patients, and the existence of tumor-like image regions. Recently, deep learning-based approaches have achieved great success for biomedical image analysis, but current state-of-the-art approaches achieve poor performance for segmenting …
In accelerated MRI reconstruction, the anatomy of a patient is recovered from a set of under-sampled and noisy measurements. Deep learning approaches have been proven to be successful in solving this ill-posed inverse problem and are capable of producing very high quality reconstructions. However, current architectures heavily rely on convolutions, that are content-independent and have difficulties modeling long-range dependencies in images. Recently, Transformers, the workhorse of contemporary natural language processing, have emerged as powerful building blocks for a multitude of vision task…
