Image landmark detection aims to automatically identify the locations of predefined fiducial points. Despite recent success in this field, higher-ordered structural modeling to capture implicit or explicit relationships among anatomical landmarks has not been adequately exploited. In this work, we present a new topology-adapting deep graph learning approach for accurate anatomical facial and medical (e.g., hand, pelvis) landmark detection. The proposed method constructs graph signals leveraging both local image features and global shape features. The adaptive graph topology naturally explores and lands on task-specific structures which are learned end-to-end with two Graph Convolutional Networks (GCNs). Extensive experiments are conducted on three public facial image datasets (WFLW, 300W, and COFW-68) as well as three real-world X-ray medical datasets (Cephalometric (public), Hand and Pelvis). Quantitative results comparing with the previous state-of-the-art approaches across all studied datasets indicating the superior performance in both robustness and accuracy. Qualitative visualizations of the learned graph topologies demonstrate a physically plausible connectivity laying behind the landmarks.
Topology-Constrained Learning for Efficient Laparoscopic Liver Landmark Detection
The Hong Kong Polytechnic University, Hong Kong, China（香港理工大学）；National University of Singapore, Singapore（新加坡国立大学）；The Chinese University of Hong Kong, Hong Kong, China（香港中文大学）；Nanfang Hospital, Southern Medical University, Guangzhou, China（南方医科大学附属南华医院）
From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Computing Topology Preservation of RBF Transformations for Landmark-Based Image Registration
Department of Mathematics G. Peano , University of Torino（数学系G. Peano，托里诺大学）；Innsbruck Medical University（因斯布鲁克医学院）；University of Applied Science, J. Gutenberg（应用科学大学，J. Gutenberg）
FreeTalk: Emotional Topology-Free 3D Talking Heads
University of Copenhagen（哥本哈根大学）；IMT Nord Europe, Institut Mines-Télécom, Centre for Digital Systems（IMT北欧、 Mines-Telecom学院、数字系统中心）；Univ. Lille, CNRS, Centrale Lille, Institut Mines-Telecom, UMR 9189 CRIStAL, F-59000 Lille, France（里尔大学、CNRS、里尔中央大学、Mines-Telecom学院、UMR 9189 CRIStAL、法国里尔）
Landmark-based robot self-localization has recently garnered interest as a highly-compressive domain-invariant approach for performing visual place recognition (VPR) across domains (e.g., time of day, weather, and season). However, landmark-based self-localization can be an ill-posed problem for a passive observer (e.g., manual robot control), as many viewpoints may not provide an effective landmark view. In this study, we consider an active self-localization task by an active observer and present a novel reinforcement learning (RL)-based next-best-view (NBV) planner. Our contributions are as …
Institute of Theoretical Physics（理论物理研究所）；Faculty of Mathematics and Computer Science（数学与计算机科学学院）；Jagiellonian University（雅盖隆大学）
We introduce a flexible setup allowing for a neural network to learn both its size and topology during the course of a standard gradient-based training. The resulting network has the structure of a graph tailored to the particular learning task and dataset. The obtained networks can also be trained from scratch and achieve virtually identical performance. We explore the properties of the network architectures for a number of datasets of varying difficulty observing systematic regularities. The obtained graphs can be therefore understood as encoding nontrivial characteristics of the particular …
Ingenuity Labs Research Institute（英格纽蒂实验室研究所）；Irdeto Canada Corporation（加拿大爱迪德公司）
Prior work has shown that the order in which different components of the face are learned using a sequential learner can play an important role in the performance of facial expression recognition systems. We propose FaceTopoNet, an end-to-end deep model for facial expression recognition, which is capable of learning an effective tree topology of the face. Our model then traverses the learned tree to generate a sequence, which is then used to form an embedding to feed a sequential learner. The devised model adopts one stream for learning structure and one stream for learning texture. The struct…
DTIF: Robust Loop Closure Detection via Delaunay Triangle Topology in Complex Forests
School of Computer Science, Wuhan University（武汉大学计算机科学学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Urban Design, Wuhan University（武汉大学城市设计学院）；LIESMARS, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
Deep neural networks have been widely used for feature learning in facial expression recognition systems. However, small datasets and large intra-class variability can lead to overfitting. In this paper, we propose a method which learns an optimized compact network topology for real-time facial expression recognition utilizing localized facial landmark features. Our method employs a spatio-temporal bilinear layer as backbone to capture the motion of facial landmarks during the execution of a facial expression effectively. Besides, it takes advantage of Monte Carlo Dropout to capture the model'…
MedSapiens: Taking a Pose to Rethink Medical Imaging Landmark Detection
The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）；Erasmus MC（埃因霍温麦斯学院）；Medical University of Innsbruck（因斯布鲁克医科大学）；University of Barcelona（巴塞罗那大学）
Mask-HybridGNet: Graph-based segmentation with emergent anatomical correspondence from pixel-level supervision
Biomedical Image Technologies, ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain（生物医学影像技术，电信工程学院，马德里理工大学，马德里，西班牙）；IHU PRISM National Precision Medicine Center in Oncology, Cancer Data Science Unit, CentraleSupélec, Gustave Roussy, INSERM, Université Paris-Saclay, Gif-sur-Yvette, France（IHU PRISM国家精准医学中心（肿瘤学），癌症数据科学单元，中央圣艾尔布兰，格斯塔夫·罗西，国家医学研究所，巴黎-萨克雷大学，法国）
Shanghai Jiao Tong University（上海交通大学）；University of Sydney（悉尼大学）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院瑞金医院）
Motion estimation is a fundamental step in dynamic medical image processing for the assessment of target organ anatomy and function. However, existing image-based motion estimation methods, which optimize the motion field by evaluating the local image similarity, are prone to produce implausible estimation, especially in the presence of large motion. In this study, we provide a novel motion estimation framework of Dense-Sparse-Dense (DSD), which comprises two stages. In the first stage, we process the raw dense image to extract sparse landmarks to represent the target organ anatomical topology…
We propose an end-to-end architecture for facial expression recognition. Our model learns an optimal tree topology for facial landmarks, whose traversal generates a sequence from which we obtain an embedding to feed a sequential learner. The proposed architecture incorporates two main streams, one focusing on landmark positions to learn the structure of the face, while the other focuses on patches around the landmarks to learn texture information. Each stream is followed by an attention mechanism and the outputs are fed to a two-stream fusion component to perform the final classification. We c…
In this paper, we develop a robust 3D garment digitization solution that can generalize well on real-world fashion catalog images with cloth texture occlusions and large body pose variations. We assumed fixed topology parametric template mesh models for known types of garments (e.g., T-shirts, Trousers) and perform mapping of high-quality texture from an input catalog image to UV map panels corresponding to the parametric mesh model of the garment. We achieve this by first predicting a sparse set of 2D landmarks on the boundary of the garments. Subsequently, we use these landmarks to perform T…
Community-Level Modeling of Gyral Folding Patterns for Robust and Anatomically Informed Individualized Brain Mapping
Department of Computer Science and Engineering, University of Texas at Arlington（计算机科学与工程系，德克萨斯大学阿灵顿分校）；School of Computing, University of Georgia（计算学院，佐治亚大学）；Department of Computer Science, Indiana University Indianapolis（计算机科学系，印第安纳大学印第安纳波利斯分校）
Head pose estimation is a crucial challenge for many real-world applications, such as attention and human behavior analysis. This paper aims to estimate head pose from a single image by applying notions of network curvature. In the real world, many complex networks have groups of nodes that are well connected to each other with significant functional roles. Similarly, the interactions of facial landmarks can be represented as complex dynamic systems modeled by weighted graphs. The functionalities of such systems are therefore intrinsically linked to the topology and geometry of the underlying …
MS-DKC: A Dataset Knowledge Card Framework for Designing and Adapting Medical Image Segmentation Models
Center of Excellence in Precision Medicine and Digital Health, Faculty of Dentistry, Chulalongkorn University, Bangkok, Thailand（精准医学与数字健康中心，朱拉隆功大学牙科学院，泰国曼谷）；Department of Computer Engineering, COMSATS University Islamabad, Islamabad, Pakistan（计算机工程系，COMSATS伊斯兰堡大学，巴基斯坦伊斯兰堡）；School of Biomedical Engineering, UNSW, Sydney, NSW, Australia（生物医学工程学院，新南威尔士大学，澳大利亚悉尼，新南威尔士）；Visiting Scholar (Collaborative Projects), Center of Excellence in Precision Medicine and Digital Health, Chulalongkorn University, Bangkok, Thailand（访问学者（合作项目），精准医学与数字健康中心，朱拉隆功大学，泰国曼谷）；Department of Computing, Abasyn University Islamabad Campus (AUIC), Islamabad, Pakistan（计算系，阿巴斯扬大学伊斯兰堡校区（AUIC），巴基斯坦伊斯兰堡）；Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates（Mohamed bin Zayed人工智能大学，阿布扎比，阿拉伯联合酋长国）；College of Computer and Information Sciences, prince Sultan University, Riyadh, SAudi Arabia（计算机与信息科学学院，苏丹王子大学，沙特阿拉伯利雅得）
Designing Deep Neural Networks (DNNs) running on edge hardware remains a challenge. Standard designs have been adopted by the community to facilitate the deployment of Neural Network models. However, not much emphasis is put on adapting the network topology to fit hardware constraints. In this paper, we adapt one of the most widely used architectures for mobile hardware platforms, MobileNetV2, and study the impact of changing its topology and applying post-training quantization. We discuss the impact of the adaptations and the deployment of the model on an embedded hardware platform for face d…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；Tencent America（腾讯美国）
Caricature is an artistic representation that deliberately exaggerates the distinctive features of a human face to convey humor or sarcasm. However, reconstructing a 3D caricature from a 2D caricature image remains a challenging task, mostly due to the lack of data. We propose to fill this gap by introducing 3DCaricShop, the first large-scale 3D caricature dataset that contains 2000 high-quality diversified 3D caricatures manually crafted by professional artists. 3DCaricShop also provides rich annotations including a paired 2D caricature image, camera parameters and 3D facial landmarks. To dem…
