Objective: Preterm infants' limb monitoring in neonatal intensive care units (NICUs) is of primary importance for assessing infants' health status and motor/cognitive development. Herein, we propose a new approach to preterm infants' limb pose estimation that features spatio-temporal information to detect and track limb joints from depth videos with high reliability. Methods: Limb-pose estimation is performed using a deep-learning framework consisting of a detection and a regression convolutional neural network (CNN) for rough and precise joint localization, respectively. The CNNs are implemented to encode connectivity in the temporal direction through 3D convolution. Assessment of the proposed framework is performed through a comprehensive study with sixteen depth videos acquired in the actual clinical practice from sixteen preterm infants (the babyPose dataset). Results: When applied to pose estimation, the median root mean squared distance, computed among all limbs, between the estimated and the ground-truth pose was 9.06 pixels, overcoming approaches based on spatial features only (11.27pixels). Conclusion: Results showed that the spatio-temporal features had a significant influence on the pose-estimation performance, especially in challenging cases (e.g., homogeneous image intensity). Significance: This paper significantly enhances the state of art in automatic assessment of preterm infants' health status by introducing the use of spatio-temporal features for limb detection and tracking, and by being the first study to use depth videos acquired in the actual clinical practice for limb-pose estimation. The babyPose dataset has been released as the first annotated dataset for infants' pose estimation.
University of Alberta（阿尔伯塔大学）；The Hospital for Sick Children（病童医院）；University of Toronto（多伦多大学）
White Matter Injury (WMI) is the most prevalent brain injury in the preterm neonate leading to developmental deficits. However, detecting WMI in Magnetic Resonance (MR) images of preterm neonate brains using traditional WM segmentation-based methods is difficult mainly due to lack of reliable preterm neonate brain atlases to guide segmentation. Hence, we propose a segmentation-free, fast, unsupervised, atlas-free WMI detection method. We detect the ventricles as blobs using a fast linear Maximally Stable Extremal Regions algorithm. A reference contour equidistant from the blobs and the brain-b…
Deep learning-based neurodevelopmental assessment in preterm infants
Image Computing Laboratory（图像计算实验室）；Shaanxi University of Science and Technology（陕西科技大学）；Department of Neonatology（新生儿科）；Shenzhen University of Advanced Technology General Hospital（深圳先进技术大学附属医院）；Department of Neurosurgery（神经外科）；The First Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第一附属医院）；Department of Radiotherapy（放射治疗科）；University of Groningen, University Medical Center Groningen（格罗宁根大学，格罗宁根大学医学中心）；CSIRO Data61
Universitá Politecnica delle Marche（马尔凯理工大学）；Istituto Italiano di Tencologia（意大利技术研究院）
Preterm infants' limb-pose estimation is a crucial but challenging task, which may improve patients' care and facilitate clinicians in infant's movements monitoring. Work in the literature either provides approaches to whole-body segmentation and tracking, which, however, has poor clinical value, or retrieve a posteriori limb pose from limb segmentation, increasing computational costs and introducing inaccuracy sources. In this paper, we address the problem of limb-pose estimation under a different point of view. We proposed a 2D fully-convolutional neural network for roughly detecting limb jo…
Xidian University（西安电子科技大学）；The First Affiliated Hospital of Xi'an Jiaotong University（西安交通大学第一附属医院）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Accurate segmentation of punctate white matter lesion (PWML) in infantile brains by an automatic algorithm can reduce the potential risk of postnatal development. How to segment PWML effectively has become one of the active topics in medical image segmentation in recent years. In this paper, we construct an efficient two-stage PWML semantic segmentation network based on the characteristics of the lesion, called refined segmentation R-CNN (RS RCNN). We propose a heuristic RPN (H-RPN) which can utilize surrounding information around the PWMLs for heuristic segmentation. Also, we design a lightwe…
The Blind Spot in 2D Infants' Pose Estimation:Robust Learning from Noisy Annotations
Università degli Studi “G. d’Annunzio” Chieti-Pescara（G. d'Annunzio 基耶蒂-佩斯卡拉大学）；University of Macerata（马切拉塔大学）；KU Leuven（鲁汶大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Università di Teramo（泰拉莫大学）
Site-Level Fine-Tuning with Progressive Layer Freezing: Towards Robust Prediction of Bronchopulmonary Dysplasia from Day-1 Chest Radiographs in Extremely Preterm Infants
Saarland University（萨尔兰大学）；University Hospital of Saarland（萨尔兰大学医院）；Clinical Centre Karlsruhe（卡尔斯鲁厄临床中心）；Helmholtz Institute for Pharmaceutical Research Saarland（亥姆霍兹萨尔兰药物研究所）；University Hospital of Augsburg（奥格斯堡大学医院）
GloResNet: A lightweight 3D CNN with global topological features for preterm brain injury prediction
Image Computing Laboratory, Shaanxi University of Science and Technology（陕西科技大学图像计算实验室）；Department of Neonatology, Shenzhen University of Advanced Technology General Hospital（深圳先进技术医院新生儿科）；Department of Neurosurgery, The First Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第一附属医院神经外科）；CSIRO Technology（澳大利亚CSIRO技术）
Cerebral Palsy Alliance Research Institute（脑瘫联盟研究所）；The University of Sydney（悉尼大学）；The Children’s Hospital at Westmead（韦斯特米德儿童医院）
The absence or abnormality of fidgety movements of joints or limbs is strongly indicative of cerebral palsy in infants. Developing computer-based methods for assessing infant movements in videos is pivotal for improved cerebral palsy screening. Most existing methods use appearance-based features and are thus sensitive to strong but irrelevant signals caused by background clutter or a moving camera. Moreover, these features are computed over the whole frame, thus they measure gross whole body movements rather than specific joint/limb motion. Addressing these challenges, we develop and validate …
MPFNet: A Multi-Prior Fusion Network with a Progressive Training Strategy for Micro-Expression Recognition
Defense Innovation Institute, Academy of Military Sciences (AMS) and Intelligent Game and Decision Laboratory（国防科技创新院、军事科学研究院（AMS）和智能游戏与决策实验室）；School of Computer Science and Technology, Dalian University of Technology（计算机科学与技术学院、大连理工大学）
University of California, Los Angeles（加州大学洛杉矶分校）；Shanghai Jiao Tong University（上海交通大学）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Peking University（北京大学）；Tsinghua University（清华大学）
To date, various 3D scene understanding tasks still lack practical and generalizable pre-trained models, primarily due to the intricate nature of 3D scene understanding tasks and their immense variations introduced by camera views, lighting, occlusions, etc. In this paper, we tackle this challenge by introducing a spatio-temporal representation learning (STRL) framework, capable of learning from unlabeled 3D point clouds in a self-supervised fashion. Inspired by how infants learn from visual data in the wild, we explore the rich spatio-temporal cues derived from the 3D data. Specifically, STRL…
