Location-guided Head Pose Estimation for Fisheye Image
School of Electronics and Information Technology, Sun Yat-sen University（中山大学电子与信息工程学院）；Data Science Research Center, Duke Kunshan University（昆山杜克大学数据科学研究中心）；Brigham Young University（杨百翰大学）
Bing Li, Dong Zhang, Cheng Huang, Yun Xian, Ming Li, Dah-Jye Lee
备注：
Revised Introduction and Related Work; Submitted to lEEE Transactions on Cognitive and Developmental Systems for review
Camera with a fisheye or ultra-wide lens covers a wide field of view that cannot be modeled by the perspective projection. Serious fisheye lens distortion in the peripheral region of the image leads to degraded performance of the existing head pose estimation models trained on undistorted images. This paper presents a new approach for head pose estimation that uses the knowledge of head location in the image to reduce the negative effect of fisheye distortion. We develop an end-to-end convolutional neural network to estimate the head pose with the multi-task learning of head pose and head location. Our proposed network estimates the head pose directly from the fisheye image without the operation of rectification or calibration. We also created a fisheye-distorted version of the three popular head pose estimation datasets, BIWI, 300W-LP, and AFLW2000 for our experiments. Experiments results show that our network remarkably improves the accuracy of head pose estimation compared with other state-of-the-art one-stage and two-stage methods.
Egocentric 3D human pose estimation using a single fisheye camera has become popular recently as it allows capturing a wide range of daily activities in unconstrained environments, which is difficult for traditional outside-in motion capture with external cameras. However, existing methods have several limitations. A prominent problem is that the estimated poses lie in the local coordinate system of the fisheye camera, rather than in the world coordinate system, which is restrictive for many applications. Furthermore, these methods suffer from limited accuracy and temporal instability due to a…
HUR-MACL: High-Uncertainty Region-Guided Multi-Architecture Collaborative Learning for Head and Neck Multi-Organ Segmentation
Digital Medical Research Center, School of Basic Medical Science, Fudan University, Shanghai 200032, China（复旦大学基础医学研究院数字医学研究中心）；Shanghai Key Lab of Medical Image Computing（上海医学图像计算重点实验室）；Radiation Oncology Center, Huashan Hospital Affiliated to Fudan University, Shanghai 200032, China（复旦大学附属华山医院放射肿瘤中心）
FONT: Flow-guided One-shot Talking Head Generation with Natural Head Motions
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；SZ DJI Technology Co., Ltd.（深圳市大疆创新科技有限公司）；The University of New South Wales（新南威尔士大学）
University of Science and Technology of China（中国科学技术大学）
Recovery of a 3D head model including the complete face and hair regions is still a challenging problem in computer vision and graphics. In this paper, we consider this problem using only a few multi-view portrait images as input. Previous multi-view stereo methods that have been based, either on optimization strategies or deep learning techniques, suffer from low-frequency geometric structures such as unclear head structures and inaccurate reconstruction in hair regions. To tackle this problem, we propose a prior-guided implicit neural rendering network. Specifically, we model the head geomet…
ByteDance Inc（字节跳动公司）；National University of Singapore（新加坡国立大学）
We present OmniAvatar, a novel geometry-guided 3D head synthesis model trained from in-the-wild unstructured images that is capable of synthesizing diverse identity-preserved 3D heads with compelling dynamic details under full disentangled control over camera poses, facial expressions, head shapes, articulated neck and jaw poses. To achieve such high level of disentangled control, we first explicitly define a novel semantic signed distance function (SDF) around a head geometry (FLAME) conditioned on the control parameters. This semantic SDF allows us to build a differentiable volumetric corres…
Tsinghua-Berkeley Shenzhen Institute, Tsinghua University（清华大学深圳国际研究生院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Deep neural networks (DNNs) have witnessed great successes in semantic segmentation, which requires a large number of labeled data for training. We present a novel learning framework called Uncertainty guided Cross-head Co-training (UCC) for semi-supervised semantic segmentation. Our framework introduces weak and strong augmentations within a shared encoder to achieve co-training, which naturally combines the benefits of consistency and self-training. Every segmentation head interacts with its peers and, the weak augmentation result is used for supervising the strong. The consistency training …
TalkCLIP: Talking Head Generation with Text-Guided Expressive Speaking Styles
Tsinghua University（清华大学）；BNRist（北京信息科学与技术国家研究中心）；State Key Laboratory of Intelligent Technology and Systems（智能技术与系统国家重点实验室）；THUAI（清华大学人工智能国际研究院）；Fuxi AI Lab（伏羲人工智能实验室）；Netease（网易公司）；School of Computer Science, the University of Queensland（昆士兰大学计算机科学学院）
SGRNet: Spatially Guided Radiology Network for Structured Radiological Reporting of Head and Neck Cancer
University of Strasbourg（斯特拉斯堡大学）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；Indian Institute of Technology (IIT) Madras（印度马德拉斯理工学院）；All India Institute of Medical Sciences（全印度医学科学院）
Monash University（莫纳什大学）；Indian Institute of Technology Ropar（印度理工学院罗帕尔分校）；University of Melbourne（墨尔本大学）
In challenging real-life conditions such as extreme head-pose, occlusions, and low-resolution images where the visual information fails to estimate visual attention/gaze direction, audio signals could provide important and complementary information. In this paper, we explore if audio-guided coarse head-pose can further enhance visual attention estimation performance for non-prolific faces. Since it is difficult to annotate audio signals for estimating the head-pose of the speaker, we use off-the-shelf state-of-the-art models to facilitate cross-modal weak-supervision. During the training phase…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
The accurate segmentation of organs-at-risk (OARs) in head and neck CT images is a critical step for radiation therapy of head and neck cancer patients. However, manual delineation for numerous OARs is time-consuming and laborious, even for expert oncologists. Moreover, manual delineation results are susceptible to high intra- and inter-variability. To this end, we propose a novel dual shape guided network (DSGnet) to automatically delineate nine important OARs in head and neck CT images. To deal with the large shape variation and unclear boundary of OARs in CT images, we represent the organ s…
Entropy-Guided Agreement-Diversity: A Semi-Supervised Active Learning Framework for Fetal Head Segmentation in Ultrasound
Taighde \' E ireann – Research Ireland Centre for Research Training in Machine Learning, Ireland School of Computer Science, University College Dublin, Ireland School of Medicine, University College Dublin, Ireland
Egocentric 3D human pose estimation with a single fisheye camera has drawn a significant amount of attention recently. However, existing methods struggle with pose estimation from in-the-wild images, because they can only be trained on synthetic data due to the unavailability of large-scale in-the-wild egocentric datasets. Furthermore, these methods easily fail when the body parts are occluded by or interacting with the surrounding scene. To address the shortage of in-the-wild data, we collect a large-scale in-the-wild egocentric dataset called Egocentric Poses in the Wild (EgoPW). This datase…
Max Plank Institute for Informatics（马克斯·普朗克信息学研究所）；Saarland Informatics Campus（萨尔兰信息学园区）；University of Maryland（马里兰大学）
We present EgoRenderer, a system for rendering full-body neural avatars of a person captured by a wearable, egocentric fisheye camera that is mounted on a cap or a VR headset. Our system renders photorealistic novel views of the actor and her motion from arbitrary virtual camera locations. Rendering full-body avatars from such egocentric images come with unique challenges due to the top-down view and large distortions. We tackle these challenges by decomposing the rendering process into several steps, including texture synthesis, pose construction, and neural image translation. For texture syn…
Frequency-guided Multi-level Reasoning for Scene Graph Generation in Video
Department of Electronic Engineering, Tsinghua University, Beijing, China（清华大学电子工程系）；State Key Laboratory of Space Network and Communications（空间网络与通信国家重点实验室）；Beijing National Research Center for Information Science and Technology (BNRist)（北京国家信息科学与技术研究中心）；School of Computer Science and Technology, Xinjiang University, Urumqi, China（新疆大学计算机科学与技术学院）
Ground-to-aerial geolocalization refers to localizing a ground-level query image by matching it to a reference database of geo-tagged aerial imagery. This is very challenging due to the huge perspective differences in visual appearances and geometric configurations between these two views. In this work, we propose a novel Transformer-guided convolutional neural network (TransGCNN) architecture, which couples CNN-based local features with Transformer-based global representations for enhanced representation learning. Specifically, our TransGCNN consists of a CNN backbone extracting feature map f…
A Gradient Mapping Guided Explainable Deep Neural Network for Extracapsular Extension Identification in 3D Head and Neck Cancer Computed Tomography Images
Department of Industrial and Systems Engineering, Mississippi State University, Mississippi State, MS, 39762, USA（工业与系统工程系，密苏里州立大学）；Department of Radiation Oncology, University of Mississippi Medical Center, Jackson, MS, 39216, USA（放射肿瘤学系，密苏里大学医学中心）
A Dual Path Framework with Hotspot Guided Fusion for Three Dimensional CT to PET Synthesis in Head and Neck Cancer
Advanced Molecular Imaging in Radiotherapy (AdMIRe) Research Laboratory, School of Health Sciences, Purdue University, West Lafayette, IN, 47907, USA（高级分子影像在放射治疗（AdMIRe）研究实验室，健康科学学院，普渡大学，西拉法叶，IN，47907，美国）；Department of Radiation Oncology, Indiana University School of Medicine, Indianapolis, IN 46202, USA（放射肿瘤学部，印第安纳大学医学院，印第安纳波利斯，IN，46202，美国）
LAF-YOLOv10 with Partial Convolution Backbone, Attention-Guided Feature Pyramid, Auxiliary P2 Head, and Wise-IoU Loss for Small Object Detection in Drone Aerial Imagery
Department of Information Technology（信息科技系）；Department of Computer Science and Engineering（计算机科学与工程系）
Improving Deep Learning-Based Target Volume Auto-Delineation for Adaptive MR-Guided Radiotherapy in Head and Neck Cancer: Impact of a Volume-Aware Dice Loss
