Deep Learning-Based Regression and Classification for Automatic Landmark Localization in Medical Images
University Medical Center Utrecht（乌得勒支大学医学中心）；Amsterdam UMC, University of Amsterdam（阿姆斯特丹大学阿姆斯特丹医学中心）；University of Twente（特文特大学）；Wageningen University（瓦赫宁根大学）
Julia M. H. Noothout, Bob D. de Vos, Jelmer M. Wolterink, Elbrich M. Postma, Paul A. M. Smeets, Richard A. P. Takx, Tim Leiner, Max A. Viergever, Ivana Išgum
备注：
12 pages, accepted at IEEE transactions in Medical Imaging
In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage.
Shanghai Jiao Tong University（上海交通大学）；University of Sydney（悉尼大学）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院瑞金医院）
Motion estimation is a fundamental step in dynamic medical image processing for the assessment of target organ anatomy and function. However, existing image-based motion estimation methods, which optimize the motion field by evaluating the local image similarity, are prone to produce implausible estimation, especially in the presence of large motion. In this study, we provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which comprises two stages. In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ anatomical topology…
Landmarks Are Alike Yet Distinct: Harnessing Similarity and Individuality for One-Shot Medical Landmark Detection
University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）；School of Information Science and Technology, Eastern Institute of Technology (EIT)（东方理工大学信息科学与技术学院）；Stanford University（斯坦福大学）
The success of deep learning methods relies on the availability of a large number of datasets with annotations; however, curating such datasets is burdensome, especially for medical images. To relieve such a burden for a landmark detection task, we explore the feasibility of using only a single annotated image and propose a novel framework named Cascade Comparing to Detect (CC2D) for one-shot landmark detection. CC2D consists of two stages: 1) Self-supervised learning (CC2D-SSL) and 2) Training with pseudo-labels (CC2D-TPL). CC2D-SSL captures the consistent anatomical information in a coarse-t…
Centrum Wiskunde & Informatica（数学与计算机科学国家研究所）；Amsterdam UMC, University of Amsterdam（阿姆斯特丹大学阿姆斯特丹大学医学中心）；Delft University of Technology（代尔夫特理工大学）
Anatomical landmark correspondences in medical images can provide additional guidance information for the alignment of two images, which, in turn, is crucial for many medical applications. However, manual landmark annotation is labor-intensive. Therefore, we propose an end-to-end deep learning approach to automatically detect landmark correspondences in pairs of two-dimensional (2D) images. Our approach consists of a Siamese neural network, which is trained to identify salient locations in images as landmarks and predict matching probabilities for landmark pairs from two different images. We t…
Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究所）；Amsterdam University Medical Centers（阿姆斯特丹大学医学中心）；University of Amsterdam（阿姆斯特丹大学）；Delft University of Technology（代尔夫特理工大学）；Leiden University Medical Center（莱顿大学医学中心）
Purpose: Deformable Image Registration (DIR) can benefit from additional guidance using corresponding landmarks in the images. However, the benefits thereof are largely understudied, especially due to the lack of automatic landmark detection methods for three-dimensional (3D) medical images. Approach: We present a Deep Convolutional Neural Network (DCNN), called DCNN-Match, that learns to predict landmark correspondences in 3D images in a self-supervised manner. We trained DCNN-Match on pairs of Computed Tomography (CT) scans containing simulated deformations. We explored five variants of DCNN…
H3DE-Net: Efficient and Accurate 3D Landmark Detection in Medical Imaging
School of Computer Science USTC（计算机科学学院 中国科学技术大学）；School of Computer Science（计算机科学学院）；Shanghai Jiao Tong University（上海交通大学）；ShanghaiTech University（上海科技大学）；Sciences College（科学学院）；School of Information Science, NENU（信息科学学院 南京师范大学）；Technical University of Munich (TUM)（慕尼黑技术大学）；Stanford University（斯坦福大学）
Deploying the idea of long-term cumulative return, reinforcement learning has shown remarkable performance in various fields. We propose a formulation of the landmark localization in 3D medical images as a reinforcement learning problem. Whereas value-based methods have been widely used to solve similar problems, we adopt an actor-critic based direct policy search method framed in a temporal difference learning approach. Successful behavior learning is challenging in large state and/or action spaces, requiring many trials. We introduce a partial policy-based reinforcement learning to enable so…
Automatic and accurate detection of anatomical landmarks is an essential operation in medical image analysis with a multitude of applications. Recent deep learning methods have improved results by directly encoding the appearance of the captured anatomy with the likelihood maps (i.e., heatmaps). However, most current solutions overlook another essence of heatmap regression, the objective metric for regressing target heatmaps and rely on hand-crafted heuristics to set the target precision, thus being usually cumbersome and task-specific. In this paper, we propose a novel learning-to-learn frame…
MedSapiens: Taking a Pose to Rethink Medical Imaging Landmark Detection
The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）；Erasmus MC（埃因霍温麦斯学院）；Medical University of Innsbruck（因斯布鲁克医科大学）；University of Barcelona（巴塞罗那大学）
Beijing Institute of Technology（北京理工大学）；The First Affiliated Hospital of Jinzhou Medical University（锦州医科大学附属第一医院）；Zhejiang University of Science & Technology（浙江科技学院）；Peking University People’s Hospital（北京大学人民医院）
Calcaneus is the largest tarsal bone to withstand the daily stresses of weight-bearing. The calcaneal fracture is the most common type in the tarsal bone fractures. After a fracture is suspected, plain radiographs should be taken first. Bohler's Angle (BA) and Critical Angle of Gissane (CAG), measured by four anatomic landmarks in lateral foot radiograph, can guide fracture diagnosis and facilitate operative recovery of the fractured calcaneus. This study aims to develop an analysis system that can automatically locate four anatomic landmarks, measure BA and CAG for fracture assessment, identi…
The Chinese University of Hong Kong（香港中文大学）；Massachusetts Institute of Technology（麻省理工学院）
In medical imaging, surface registration is extensively used for performing systematic comparisons between anatomical structures, with a prime example being the highly convoluted brain cortical surfaces. To obtain a meaningful registration, a common approach is to identify prominent features on the surfaces and establish a low-distortion mapping between them with the feature correspondence encoded as landmark constraints. Prior registration works have primarily focused on using manually labeled landmarks and solving highly nonlinear optimization problems, which are time-consuming and hence hin…
UAE: Universal Anatomical Embedding on Multi-modality Medical Images
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University（西北工业大学计算机科学与工程学院，空天地海一体化大数据应用技术国家工程实验室）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）；Beijing United Family Hospital（北京和睦家医院）；The First Affiliated Hospital, Zhejiang University（浙江大学医学院附属第一医院）；Peking Union Medical College Hospital（北京协和医院）
Image Sciences Institute（影像科学研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）；Department of Radiology（放射科）
Fast and accurate anatomical landmark detection can benefit many medical image analysis methods. Here, we propose a method to automatically detect anatomical landmarks in medical images. Automatic landmark detection is performed with a patch-based fully convolutional neural network (FCNN) that combines regression and classification. For any given image patch, regression is used to predict the 3D displacement vector from the image patch to the landmark. Simultaneously, classification is used to identify patches that contain the landmark. Under the assumption that patches close to a landmark can…
Adaptive Query Prompting for Multi-Domain Landmark Detection
School of Computer Science and Technology, Guangdong University of Technology（广东技术大学计算机科学与技术学院）；School of Automation, Guangdong University of Technology（广东技术大学自动化学院）；Faculty of Applied Sciences, Macao Polytechnic University（澳门 polytechnic university 应用科学学院）；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外语外贸大学信息科学与技术学院）；School of Computer Science and Engineering, Huizhou University（惠州市大学计算机科学与工程学院）
The various speech sounds of a language are obtained by varying the shape and position of the articulators surrounding the vocal tract. Analyzing their variations is crucial for understanding speech production, diagnosing speech disorders and planning therapy. Identifying key anatomical landmarks of these structures on medical images is a pre-requisite for any quantitative analysis and the rising amount of data generated in the field calls for an automatic solution. The challenge lies in the high inter- and intra-speaker variability, the mutual interaction between the articulators and the mode…
The University of Hong Kong（香港大学）；ShanghaiTech University（上海科技大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；South China University of Technology（华南理工大学）
Well-annotated medical images are costly and sometimes even impossible to acquire, hindering landmark detection accuracy to some extent. Semi-supervised learning alleviates the reliance on large-scale annotated data by exploiting the unlabeled data to understand the population structure of anatomical landmarks. The global shape constraint is the inherent property of anatomical landmarks that provides valuable guidance for more consistent pseudo labelling of the unlabeled data, which is ignored in the previously semi-supervised methods. In this paper, we propose a model-agnostic shape-regulated…
Accurate detection of anatomical landmarks is an essential step in several medical imaging tasks. We propose a novel communicative multi-agent reinforcement learning (C-MARL) system to automatically detect landmarks in 3D brain images. C-MARL enables the agents to learn explicit communication channels, as well as implicit communication signals by sharing certain weights of the architecture among all the agents. The proposed approach is evaluated on two brain imaging datasets from adult magnetic resonance imaging (MRI) and fetal ultrasound scans. Our experiments show that involving multiple coo…
Asynchronous Decentralized Federated Lifelong Learning for Landmark Localization in Medical Imaging
Rice University（莱斯大学）；McGovern Medical School, UTHealth（德克萨斯大学健康科学中心麦戈文医学院）；The University of Maryland Medical Intelligent Imaging (UM2ii) Center（马里兰大学医学智能成像中心）
Is in-domain data beneficial in transfer learning for landmarks detection in x-ray images?
MaLGa Center, DIBRIS, Università degli studi di Genova, Genoa, Italy（MaLGa中心，DIBRIS，意大利热那亚大学，热那亚，意大利）；MaLGa Center, DIMA, Università degli studi di Genova, Genoa, Italy（MaLGa中心，DIMA，意大利热那亚大学，热那亚，意大利）
University of Electronic Science and Technology of China（电子科技大学）；SenseTime Research（商汤科技研究院）
Deep learning networks have shown promising performance for accurate object localization in medial images, but require large amount of annotated data for supervised training, which is expensive and expertise burdensome. To address this problem, we present a one-shot framework for organ and landmark localization in volumetric medical images, which does not need any annotation during the training stage and could be employed to locate any landmarks or organs in test images given a support (reference) image during the inference stage. Our main idea comes from that tissues and organs from different…
The detection of anatomical landmarks is a vital step for medical image analysis and applications for diagnosis, interpretation and guidance. Manual annotation of landmarks is a tedious process that requires domain-specific expertise and introduces inter-observer variability. This paper proposes a new detection approach for multiple landmarks based on multi-agent reinforcement learning. Our hypothesis is that the position of all anatomical landmarks is interdependent and non-random within the human anatomy, thus finding one landmark can help to deduce the location of others. Using a Deep Q-Net…
CABLD: Contrast-Agnostic Brain Landmark Detection with Consistency-Based Regularization
Department of Computer Science and Software Engineering, Concordia University（计算机科学与软件工程系，康科迪亚大学）；Department of Electrical and Computer Engineering, Concordia University（电气与计算机工程系，康科迪亚大学）
Emory University（埃默里大学）；University of Chicago（芝加哥大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University School of Medicine（埃默里大学医学院）
This study proposed a deep learning-based tracking method for ultrasound (US) image-guided radiation therapy. The proposed cascade deep learning model is composed of an attention network, a mask region-based convolutional neural network (mask R-CNN), and a long short-term memory (LSTM) network. The attention network learns a mapping from a US image to a suspected area of landmark motion in order to reduce the search region. The mask R-CNN then produces multiple region-of-interest (ROI) proposals in the reduced region and identifies the proposed landmark via three network heads: bounding box re…
IBM Research - Australia（IBM澳大利亚研究院）；EPF Laussane（洛桑联邦理工学院）
Anatomical landmark segmentation and pathology localization are important steps in automated analysis of medical images. They are particularly challenging when the anatomy or pathology is small, as in retinal images and cardiac MRI, or when the image is of low quality due to device acquisition parameters as in magnetic resonance (MR) scanners. We propose an image super-resolution method using progressive generative adversarial networks (P-GAN) that can take as input a low-resolution image and generate a high resolution image of desired scaling factor. The super resolved images can be used for …
University of Science and Technology of China（中国科学技术大学）
Accurate anatomical landmark detection plays an increasingly vital role in medical image analysis. Although existing methods achieve satisfying performance, they are mostly based on CNN and specialized for a single domain say associated with a particular anatomical region. In this work, we propose a universal model for multi-domain landmark detection by taking advantage of transformer for modeling long dependencies and develop a domain-adaptive transformer model, named as DATR, which is trained on multiple mixed datasets from different anatomies and capable of detecting landmarks of any image …
Multi-environment lifelong deep reinforcement learning for medical imaging
Rice University（莱斯大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）；McGovern Medical School, UTHealth Houston（休斯顿UTHealth麦戈文医学院）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；University of Maryland Medical Intelligent Imaging (UM2ii)（马里兰大学医学智能成像中心）；University of Maryland School of Medicine（马里兰大学医学院）
Landmark matching via geodesic shooting is a prerequisite task for numerous registration based applications in biomedicine. Geodesic shooting has been developed as one solution approach and formulates the diffeomorphic registration as an optimal control problem under the Hamiltonian framework. In this framework, with landmark positions q0 fixed, the problem solely depends on the initial momentum p0 and evolves through time steps according to a set of constraint equations. Given an initial p0, the algorithm flows q and p forward through time steps, calculates a loss based on point-set mismatch …
Newton-Wellesley Hospital and Harvard Medical School（牛顿-韦尔斯利医院与哈佛医学院）；Massachusetts General Hospital and Harvard Medical School（麻省总医院与哈佛医学院）
Registration of 3D anatomic structures to their 2D dual fluoroscopic X-ray images is a widely used motion tracking technique. However, deep learning implementation is often impeded by a paucity of medical images and ground truths. In this study, we proposed a transfer learning strategy for 3D-to-2D registration using deep neural networks trained from an artificial dataset. Digitally reconstructed radiographs (DRRs) and radiographic skull landmarks were automatically created from craniocervical CT data of a female subject. They were used to train a residual network (ResNet) for landmark detecti…
Fudan University（复旦大学）；University of Oxford（牛津大学）；Shanghai Jiao Tong University（上海交通大学）；University of Chinese Academy of Sciences（中国科学院大学）；Huashan Hospital, Fudan University（复旦大学附属华山医院）；Zhong Shan Hospital, Fudan University（复旦大学附属中山医院）
Medical images are generally acquired with limited field-of-view (FOV), which could lead to incomplete regions of interest (ROI), and thus impose a great challenge on medical image analysis. This is particularly evident for the learning-based multi-target landmark detection, where algorithms could be misleading to learn primarily the variation of background due to the varying FOV, failing the detection of targets. Based on learning a navigation policy, instead of predicting targets directly, reinforcement learning (RL)-based methods have the potential totackle this challenge in an efficient ma…
University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州高等研究院）
Detecting anatomical landmarks in medical images plays an essential role in understanding the anatomy and planning automated processing. In recent years, a variety of deep neural network methods have been developed to detect landmarks automatically. However, all of those methods are unary in the sense that a highly specialized network is trained for a single task say associated with a particular anatomical region. In this work, for the first time, we investigate the idea of "You Only Learn Once (YOLO)" and develop a universal anatomical landmark detection model to realize multiple landmark det…
Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）；Singapore Eye Research Institute（新加坡眼科研究所）；Singapore National Eye Centre（新加坡全国眼科中心）；Yong Loo Lin School of Medicine, NUS（新加坡国立大学杨潞龄医学院）
Localizing anatomical landmarks are important tasks in medical image analysis. However, the landmarks to be localized often lack prominent visual features. Their locations are elusive and easily confused with the background, and thus precise localization highly depends on the context formed by their surrounding areas. In addition, the required precision is usually higher than segmentation and object detection tasks. Therefore, localization has its unique challenges different from segmentation or detection. In this paper, we propose a zoom-in attentive network (ZIAN) for anatomical landmark loc…
University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）
Fundus photography has routinely been used to document the presence and severity of retinal degenerative diseases such as age-related macular degeneration (AMD), glaucoma, and diabetic retinopathy (DR) in clinical practice, for which the fovea and optic disc (OD) are important retinal landmarks. However, the occurrence of lesions, drusen, and other retinal abnormalities during retinal degeneration severely complicates automatic landmark detection and segmentation. Here we propose HBA-U-Net: a U-Net backbone enriched with hierarchical bottleneck attention. The network consists of a novel bottle…
HYATT-Net is Grand: A Hybrid Attention Network for Performant Anatomical Landmark Detection
Suzhou Institute for Advanced Research, University of Science and Technology of China (USTC)（中国科学技术大学苏州高等研究院）；School of Computer Science and Technology, University of Science and Technology of China (USTC)（中国科学技术大学计算机科学与技术学院）；School of Information Science and Technology, Eastern Institute of Technology (EIT)（宁波东方理工大学信息科学与技术学院）；Stanford University（斯坦福大学）
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Arkansas（阿肯色大学）；University of Houston（休斯顿大学）；INRIA, Sophia Antipolis-Mediterranean Centre（法国国家信息与自动化研究所索菲亚·安蒂波利斯地中海中心）
Deep reinforcement learning (DRL) augments the reinforcement learning framework, which learns a sequence of actions that maximizes the expected reward, with the representative power of deep neural networks. Recent works have demonstrated the great potential of DRL in medicine and healthcare. This paper presents a literature review of DRL in medical imaging. We start with a comprehensive tutorial of DRL, including the latest model-free and model-based algorithms. We then cover existing DRL applications for medical imaging, which are roughly divided into three main categories: (I) parametric med…
FM-OSD: Foundation Model-Enabled One-Shot Detection of Anatomical Landmarks
The Chinese University of Hong Kong（香港中文大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Massachusetts General Brigham（麻省总医院布里格姆医疗系统）；Institute of Medical Intelligence and XR, The Chinese University of Hong Kong（香港中文大学医学智能与XR研究所）
UOD: Universal One-shot Detection of Anatomical Landmarks
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（生物医学工程学院，生命科学与医学系，科学技术大学）；Suzhou Institute for Advanced Research, University of Science and Technology of China（苏州先进研究院，科学技术大学）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS), Institute of Computing Technology, CAS（中国科学院智能信息处理重点实验室，计算技术研究所，中国科学院）；Department of Radiology, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences（放射科，广东省人民医院，广东省医学科学院）；Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application, Guangdong Provincial People's Hospital, Guangdong Academy of Medical Sciences（广东省人工智能在医学影像分析与应用重点实验室，广东省人民医院，广东省医学科学院）
本文提出 UOD 框架，通过领域自适应卷积与 transformer 与领域共享模块，在每域仅一个标注样本的条件下实现跨头、手、胸多领域 X 射线标志点检测，并达到最先进性能。
University of Electronic Science and Technology of China（电子科技大学）
Localization of anatomical landmarks is essential for clinical diagnosis, treatment planning, and research. In this paper, we propose a novel deep network, named feature aggregation and refinement network (FARNet), for the automatic detection of anatomical landmarks. To alleviate the problem of limited training data in the medical domain, our network adopts a deep network pre-trained on natural images as the backbone network and several popular networks have been compared. Our FARNet also includes a multi-scale feature aggregation module for multi-scale feature fusion and a feature refinement …
Institute of Information Technologies, Mathematics and Mechanics, Lobachevsky State University（罗巴切夫斯基州立大学信息技术、数学与力学研究所）；Medical University of Vienna（维也纳医科大学）
In this paper, we address the problem of automatic three-dimensional cephalometric analysis. Cephalometric analysis performed on lateral radiographs doesn't fully exploit the structure of 3D objects due to projection onto the lateral plane. With the development of three-dimensional imaging techniques such as CT, several analysis methods have been proposed that extend to the 3D case. The analysis based on these methods is invariant to rotations and translations and can describe difficult skull deformation, where 2D cephalometry has no use. In this paper, we provide a wide overview of existing a…
BioFace3D: A fully automatic pipeline for facial biomarkers extraction of 3D face reconstructions segmented from MRI
La Salle - Universitat Ramon Llull（拉蒙·柳利大学拉萨尔学院）；Universitat de Barcelona（巴塞罗那大学）；FIDMAG, Sisters Hospitallers Research Foundation（仁爱修女会研究基金会FIDMAG研究所）；CIBERSAM（精神健康生物医学研究网络）；Hospital de Sant Pau i la Santa Creu（圣保禄圣十字医院）
While facial biometrics has been widely used for identification purpose, it has recently been researched as medical biometrics for a range of diseases. In this chapter, we investigate the facial landmark detection for atypical 3D facial modelling in facial palsy cases, while potentially such modelling can assist the medical diagnosis using atypical facial features. In our work, a study of landmarks localisation methods such as stacked hourglass networks is conducted and evaluated to ascertain their accuracy when presented with unseen atypical faces. The evaluation highlights that the state-of-…
Petuum Inc, USA（Petuum公司（美国））；School of Computer Science, Carnegie Mellon University, USA（卡内基梅隆大学计算机科学学院（美国））
Medical imaging is widely used in clinical practice for diagnosis and treatment. Report-writing can be error-prone for unexperienced physicians, and time- consuming and tedious for experienced physicians. To address these issues, we study the automatic generation of medical imaging reports. This task presents several challenges. First, a complete report contains multiple heterogeneous forms of information, including findings and tags. Second, abnormal regions in medical images are difficult to identify. Third, the re- ports are typically long, containing multiple sentences. To cope with these …
Every year physicians face an increasing demand of image-based diagnosis from patients, a problem that can be addressed with recent artificial intelligence methods. In this context, we survey works in the area of automatic report generation from medical images, with emphasis on methods using deep neural networks, with respect to: (1) Datasets, (2) Architecture Design, (3) Explainability and (4) Evaluation Metrics. Our survey identifies interesting developments, but also remaining challenges. Among them, the current evaluation of generated reports is especially weak, since it mostly relies on t…
National University of Sciences and Technology (NUST)（国立科技大学（NUST））；Riphah International University（里法国际大学）
The accurate identification and precise localization of cephalometric landmarks enable the classification and quantification of anatomical abnormalities. The traditional way of marking cephalometric landmarks on lateral cephalograms is a monotonous and time-consuming job. Endeavours to develop automated landmark detection systems have persistently been made, however, they are inadequate for orthodontic applications due to unavailability of a reliable dataset. We proposed a new state-of-the-art dataset to facilitate the development of robust AI solutions for quantitative morphometric analysis. …
Anatomical landmarks are a crucial prerequisite for many medical imaging tasks. Usually, the set of landmarks for a given task is predefined by experts. The landmark locations for a given image are then annotated manually or via machine learning methods trained on manual annotations. In this paper, in contrast, we present a method to automatically discover and localize anatomical landmarks in medical images. Specifically, we consider landmarks that attract the visual attention of humans, which we term visually salient landmarks. We illustrate the method for fetal neurosonographic images. First…
Nanjing Medical University（南京医科大学）；Guangdong Women and Children Hospital（广东省妇幼保健院）；Shenzhen University（深圳大学）
The ultrasound (US) screening of the infant hip is vital for the early diagnosis of developmental dysplasia of the hip (DDH). The US diagnosis of DDH refers to measuring alpha and beta angles that quantify hip joint development. These two angles are calculated from key anatomical landmarks and structures of the hip. However, this measurement process is not trivial for sonographers and usually requires a thorough understanding of complex anatomical structures. In this study, we propose a multi-task framework to learn the relationships among landmarks and structures jointly and automatically eva…
Fundamental to improving Dental and Orthodontic treatments is the ability to quantitatively assess and cross-compare their outcomes. Such assessments require calculating distances and angles from 3D coordinates of dental landmarks. The costly and repetitive task of hand-labelling dental models impedes studies requiring large sample size to penetrate statistical noise. We have developed techniques and software implementing these techniques to map out automatically, 3D dental scans. This process is divided into consecutive steps - determining a model's orientation, separating and identifying the…
Graz University of Technology（格拉茨工业大学）；Ludwig Boltzmann Institute for Clinical Forensic Imaging（路德维希·玻尔兹曼临床法医成像研究所）
In many medical image analysis applications, often only a limited amount of training data is available, which makes training of convolutional neural networks (CNNs) challenging. In this work on anatomical landmark localization, we propose a CNN architecture that learns to split the localization task into two simpler sub-problems, reducing the need for large training datasets. Our fully convolutional SpatialConfiguration-Net (SCN) dedicates one component to locally accurate but ambiguous candidate predictions, while the other component improves robustness to ambiguities by incorporating the spa…
Recently, there is an increasing demand for automatically detecting anatomical landmarks which provide rich structural information to facilitate subsequent medical image analysis. Current methods related to this task often leverage the power of deep neural networks, while a major challenge in fine tuning such models in medical applications arises from insufficient number of labeled samples. To address this, we propose to regularize the knowledge transfer across source and target tasks through cross-task representation learning. The proposed method is demonstrated for extracting facial anatomic…
