Motions of virtual characters in movies or video games are typically generated by recording actors using motion capturing methods. Animations generated this way often need postprocessing, such as improving the periodicity of cyclic animations or generating entirely new motions by interpolation of existing ones. Furthermore, search and classification of recorded motions becomes more and more important as the amount of recorded motion data grows.
In this paper, we will apply methods from shape analysis to the processing of animations. More precisely, we will use the by now classical elastic metric model used in shape matching, and extend it by incorporating additional inexact feature point information, which leads to an improved temporal alignment of different animations.
University of Kaiserslautern（凯泽斯劳滕大学）；Osaka Prefecture University（大阪府立大学）
The topic of facial landmark detection has been widely covered for pictures of human faces, but it is still a challenge for drawings. Indeed, the proportions and symmetry of standard human faces are not always used for comics or mangas. The personal style of the author, the limitation of colors, etc. makes the landmark detection on faces in drawings a difficult task. Detecting the landmarks on manga images will be useful to provide new services for easily editing the character faces, estimating the character emotions, or generating automatically some animations such as lip or eye movements. Th…
University of Trento（特伦托大学）；EPFL（洛桑联邦理工学院）；Inria（法国国家信息与自动化研究所）；Grenoble-Alpes University（格勒诺布尔阿尔卑斯大学）；Fondazione Bruno Kessler (FBK)（布鲁诺·凯斯勒基金会）
Each smile is unique: one person surely smiles in different ways (e.g., closing/opening the eyes or mouth). Given one input image of a neutral face, can we generate multiple smile videos with distinctive characteristics? To tackle this one-to-many video generation problem, we propose a novel deep learning architecture named Conditional Multi-Mode Network (CMM-Net). To better encode the dynamics of facial expressions, CMM-Net explicitly exploits facial landmarks for generating smile sequences. Specifically, a variational auto-encoder is used to learn a facial landmark embedding. This single emb…
Huazhong University of Science and Technology（华中科技大学）；Wuhan National Laboratory for Optoelectronics（武汉光电国家研究中心）；City University of Hong Kong（香港城市大学）；The Hong Kong University of Science and Technology（香港科技大学）；Union Hospital, Tongji Medical College, Huazhong University of Science and Technology（华中科技大学同济医学院附属协和医院）
Vertebral landmark localization is a crucial step for variant spine-related clinical applications, which requires detecting the corner points of 17 vertebrae. However, the neighbor landmarks often disturb each other for the homogeneous appearance of vertebrae, which makes vertebral landmark localization extremely difficult. In this paper, we propose multi-stage cascaded convolutional neural networks (CNNs) to split the single task into two sequential steps, i.e., center point localization to roughly locate 17 center points of vertebrae, and corner point localization to find 4 corner points for…
One usage of medical ultrasound imaging is to visualize and characterize human tongue shape and motion during a real-time speech to study healthy or impaired speech production. Due to the low-contrast characteristic and noisy nature of ultrasound images, it might require expertise for non-expert users to recognize tongue gestures in applications such as visual training of a second language. Moreover, quantitative analysis of tongue motion needs the tongue dorsum contour to be extracted, tracked, and visualized. Manual tongue contour extraction is a cumbersome, subjective, and error-prone task.…
In current biological and medical research, statistical shape modeling (SSM) provides an essential framework for the characterization of anatomy/morphology. Such analysis is often driven by the identification of a relatively small number of geometrically consistent features found across the samples of a population. These features can subsequently provide information about the population shape variation. Dense correspondence models can provide ease of computation and yield an interpretable low-dimensional shape descriptor when followed by dimensionality reduction. However, automatic methods for…
The goal of point set registration is to find point-by-point correspondences between point sets, each of which characterizes the shape of an object. Because local preservation of object geometry is assumed, prevalent algorithms in the area can often elegantly solve the problems without using geometric information specific to the objects. This means that registration performance can be further improved by using prior knowledge of object geometry. In this paper, we propose a novel point set registration method using the Gaussian mixture model with prior shape information encoded as a statistical…
Landmark-Guided Cross-Speaker Lip Reading with Mutual Information Regularization
School of Life Science and Technology, University of Electronic Science and Technology of China（电子科技大学生命科学与技术学院）；Defense Innovation Institute, Academy of Military Sciences（军事科学院国防创新研究院）；Tianjin Artificial Intelligence Innovation Center（天津人工智能创新中心）；High-tech Institute of Qingzhou（青州高新技术研究院）
On Procrustes Contamination in Machine Learning Applications of Geometric Morphometrics
PACEA UMR5199, CNRS Université de Bordeaux（PACEA UMR5199，法国国家科学研究中心波尔多大学）；Department d’Història i Història de l’Art Universitat Rovira i Virgili（历史与艺术史系罗维拉-维吉莉大学）
Institute for Systems and Robotics（系统与机器人研究所）；Instituto Superior Técnico（高等理工学院）
In image analysis, many tasks require representing two-dimensional (2D) shape, often specified by a set of 2D points, for comparison purposes. The challenge of the representation is that it must not only capture the characteristics of the shape but also be invariant to relevant transformations. Invariance to geometric transformations, such as translation, rotation, and scale, has received attention in the past, usually under the assumption that the points are previously labeled, i.e., that the shape is characterized by an ordered set of landmarks. However, in many practical scenarios, the poin…
