Recently, diffusion-based methods for monocular 3D human pose estimation have achieved state-of-the-art (SOTA) performance by directly regressing the 3D joint coordinates from the 2D pose sequence. Although some methods decompose the task into bone length and bone direction prediction based on the human anatomical skeleton to explicitly incorporate more human body prior constraints, the performance of these methods is significantly lower than that of the SOTA diffusion-based methods. This can be attributed to the tree structure of the human skeleton. Direct application of the disentangled method could amplify the accumulation of hierarchical errors, propagating through each hierarchy. Meanwhile, the hierarchical information has not been fully explored by the previous methods. To address these problems, a Disentangled Diffusion-based 3D Human Pose Estimation method with Hierarchical Spatial and Temporal Denoiser is proposed, termed DDHPose. In our approach: (1) We disentangle the 3D pose and diffuse the bone length and bone direction during the forward process of the diffusion model to effectively model the human pose prior. A disentanglement loss is proposed to supervise diffusion model learning. (2) For the reverse process, we propose Hierarchical Spatial and Temporal Denoiser (HSTDenoiser) to improve the hierarchical modeling of each joint. Our HSTDenoiser comprises two components: the Hierarchical-Related Spatial Transformer (HRST) and the Hierarchical-Related Temporal Transformer (HRTT). HRST exploits joint spatial information and the influence of the parent joint on each joint for spatial modeling, while HRTT utilizes information from both the joint and its hierarchical adjacent joints to explore the hierarchical temporal correlations among joints. Code and models are available at https://github.com/Andyen512/DDHPose
DiSa: Saliency-Aware Foreground-Background Disentangled Framework for Open-Vocabulary Semantic Segmentation
Department of Computer Science and Engineering, Lehigh University（计算机科学与工程系，莱恩大学）；Department of Computer Science（计算机科学系）；Engineering, Lehigh University（工程系，莱恩大学）；Qualcomm AI Research（高通人工智能研究）
Interpretable Single-View 3D Gaussian Splatting using Unsupervised Hierarchical Disentangled Representation Learning
Shanghai Jiao Tong University（上海交通大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（东方理工大学宁波数字孪生研究院）；Hong Kong Polytechnic University（香港理工大学）
Generating videos predicting the future of a given sequence has been an area of active research in recent years. However, an essential problem remains unsolved: most of the methods require large computational cost and memory usage for training. In this paper, we propose a novel method for generating future prediction videos with less memory usage than the conventional methods. This is a critical stepping stone in the path towards generating videos with high image quality, similar to that of generated images in the latest works in the field of image generation. We achieve high-efficiency by tra…
Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；University of Konstanz（康斯坦茨大学）；Centro de Investigaciones Científicas de las Huastecas Aguazarca, A.C.（华斯特卡斯阿瓜扎尔卡科学研究中心）；University of Würzburg（维尔茨堡大学）；Texas A&M University（得克萨斯农工大学）；Texas State University San Marcos（得克萨斯州立大学圣马科斯分校）
Apart from discriminative models for classification and object detection tasks, the application of deep convolutional neural networks to basic research utilizing natural imaging data has been somewhat limited; particularly in cases where a set of interpretable features for downstream analysis is needed, a key requirement for many scientific investigations. We present an algorithm and training paradigm designed specifically to address this: decontextualized hierarchical representation learning (DHRL). By combining a generative model chaining procedure with a ladder network architecture and late…
Hierarchical Context Alignment with Disentangled Geometric and Temporal Modeling for Semantic Occupancy Prediction
Shanghai Jiao Tong University（上海交通大学）；Eastern Institute of Technology（东部技术研究所）；Ningbo Institute of Digital Twin（宁波数字孪生研究所）；University of Adelaide（阿德莱德大学）；Tokyo Institute of Technology（东京技术大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
DifAttack++: Query-Efficient Black-Box Adversarial Attack via Hierarchical Disentangled Feature Space in Cross-Domain
State Key Laboratory of Internet of Things for Smart City（物联网智能城市国家重点实验室）；Department of Computer and Information Science（计算机与信息科学系）；Faculty of Science and Technology（科技学院）；University of Macau（澳门大学）；Echizen Laboratory（Echizen实验室）；National Institute of Informatics（国家信息 institute）；Institute of Artificial Intelligence and Future Networks（人工智能与未来网络研究院）；Beijing Normal University（北京师范大学）；School of Computer Science and Engineering（计算机科学与工程学院）；Macau University of Science and Technology（澳门科技大学）；University of Tokyo（东京大学）
Disentangled Hierarchical VAE for 3D Human-Human Interaction Generation
Department of CSSE, The University of Western Australia（西澳大学计算机科学与工程系）；Data61, CSIRO（澳大利亚联邦科学工业研究组织Data61部门）；Australian Institute for Machine Learning, The University of Adelaide（澳大利亚阿德莱德大学人工智能研究所）；NERC-RVC, Hunan University（湖南大学NERC-RVC部门）
Image denoising is a typical ill-posed problem due to complex degradation. Leading methods based on normalizing flows have tried to solve this problem with an invertible transformation instead of a deterministic mapping. However, the implicit bijective mapping is not explored well. Inspired by a latent observation that noise tends to appear in the high-frequency part of the image, we propose a fully invertible denoising method that injects the idea of disentangled learning into a general invertible neural network to split noise from the high-frequency part. More specifically, we decompose the …
University of Science and Technology of China（中国科学技术大学）；Monash University（莫纳什大学）；Nanyang Technological University（南洋理工大学）
Human bodies exhibit various shapes for different identities or poses, but the body shape has certain similarities in structure and thus can be embedded in a low-dimensional space. This paper presents an autoencoder-like network architecture to learn disentangled shape and pose embedding specifically for the 3D human body. This is inspired by recent progress of deformation-based latent representation learning. To improve the reconstruction accuracy, we propose a hierarchical reconstruction pipeline for the disentangling process and construct a large dataset of human body models with consistent…
Inverse problem regularization with hierarchical variational autoencoders
Univ. Bordeaux（波尔多大学）；CNRS（法国国家科学研究中心）；Bordeaux INP（波尔多国立综合理工学院）；IMB（数学与计算机科学研究所）；Ubisoft La Forge（育碧拉福奇工作室）；Université Paris Cité（巴黎城市大学）；MAP5（数学与应用概率科学研究所）
Samsung AI Center（三星人工智能中心）；Rutgers University（罗格斯大学）
We propose a family of novel hierarchical Bayesian deep auto-encoder models capable of identifying disentangled factors of variability in data. While many recent attempts at factor disentanglement have focused on sophisticated learning objectives within the VAE framework, their choice of a standard normal as the latent factor prior is both suboptimal and detrimental to performance. Our key observation is that the disentangled latent variables responsible for major sources of variability, the relevant factors, can be more appropriately modeled using long-tail distributions. The typical Gaussian…
HeRO: Hierarchical 3D Semantic Representation for Pose-aware Object Manipulation
College of Computer Science, Sichuan University（四川大学计算机学院）；School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）；Dexmal
DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
