Despite significant progress, controlled generation of complex images with interacting people remains difficult. Existing layout generation methods fall short of synthesizing realistic person instances; while pose-guided generation approaches focus on a single person and assume simple or known backgrounds. To tackle these limitations, we propose a new problem, \textbf{Persons in Context Synthesis}, which aims to synthesize diverse person instance(s) in consistent contexts, with user control over both. The context is specified by the bounding box object layout which lacks shape information, while pose of the person(s) by keypoints which are sparsely annotated. To handle the stark difference in input structures, we proposed two separate neural branches to attentively composite the respective (context/person) inputs into shared ``compositional structural space'', which encodes shape, location and appearance information for both context and person structures in a disentangled manner. This structural space is then decoded to the image space using multi-level feature modulation strategy, and learned in a self supervised manner from image collections and their corresponding inputs. Extensive experiments on two large-scale datasets (COCO-Stuff \cite{caesar2018cvpr} and Visual Genome \cite{krishna2017visual}) demonstrate that our framework outperforms state-of-the-art methods w.r.t. synthesis quality.
SenseNets Technology Limited（SenseNets科技有限公司）；SenseTime Group Limited（商汤集团有限公司）；The University of Sydney（悉尼大学）
Person re-identification (ReID) is to identify pedestrians observed from different camera views based on visual appearance. It is a challenging task due to large pose variations, complex background clutters and severe occlusions. Recently, human pose estimation by predicting joint locations was largely improved in accuracy. It is reasonable to use pose estimation results for handling pose variations and background clutters, and such attempts have obtained great improvement in ReID performance. However, we argue that the pose information was not well utilized and hasn't yet been fully exploited…
Technique Center Ihome Corporation（Ihome公司技术中心）；IBM Research（IBM研究院）
In this paper, we propose an assistive model that supports professional interior designers to produce industrial interior decoration solutions and to meet the personalized preferences of the property owners. The proposed model is able to automatically produce the layout of objects of a particular indoor scene according to property owners' preferences. In particular, the model consists of the extraction of abstract graph, conditional graph generation, and conditional scene instantiation. We provide an interior layout dataset that contains real-world 11000 designs from professional designers. Ou…
Resolving compositional and conformational heterogeneity in cryo-EM with deformable 3D Gaussian representations
Research Center for Mathematics and Interdisciplinary Sciences (Ministry of Education Frontiers Science Center for Nonlinear Expectations)（数学与交叉科学研究中心（教育部前沿非线性期望科学中心））；College of Medical Information and Engineering（医学信息与工程学院）；School of Medical Technology（医学技术学院）；State Key Laboratory of Microbial Technology（微生物技术国家重点实验室）；King Abdullah University of Science and Technology (KAUST)（国王 Abdullah 科学技术大学（KAUST））
VinAI Research（VinAI研究院）；University of Information Technology, VNU-HCM（越南国家大学胡志明市分校信息科技大学）；Stony Brook University（石溪大学）；University of Oregon（俄勒冈大学）
Personality image captioning (PIC) aims to describe an image with a natural language caption given a personality trait. In this work, we introduce a novel formulation for PIC based on a communication game between a speaker and a listener. The speaker attempts to generate natural language captions while the listener encourages the generated captions to contain discriminative information about the input images and personality traits. In this way, we expect that the generated captions can be improved to naturally represent the images and express the traits. In addition, we propose to adapt the la…
Institute for Information Technology Convergence（信息技术融合研究所）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
This paper addresses unsupervised person re-identification (Re-ID) using multi-label prediction and classification based on graph-structural insight. Our method extracts features from person images and produces a graph that consists of the features and a pairwise similarity of them as nodes and edges, respectively. Based on the graph, the proposed graph structure based multi-label prediction (GSMLP) method predicts multi-labels by considering the pairwise similarity and the adjacency node distribution of each node. The multi-labels created by GSMLP are applied to the proposed selective multi-l…
Compositional Inversion for Stable Diffusion Models
the Hong Kong Polytechnic University（香港理工大学）；Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人创新中心）；Sichuan University（四川大学）；UCAS（中国科学院大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；The Chinese University of Hong Kong（香港中文大学）；Lanzhou University（兰州大学）；Beijing Institute of Technology（北京理工大学）
Skeleton-based person re-identification (Re-ID) is an emerging open topic providing great value for safety-critical applications. Existing methods typically extract hand-crafted features or model skeleton dynamics from the trajectory of body joints, while they rarely explore valuable relation information contained in body structure or motion. To fully explore body relations, we construct graphs to model human skeletons from different levels, and for the first time propose a Multi-level Graph encoding approach with Structural-Collaborative Relation learning (MG-SCR) to encode discriminative gra…
Structural-Semantic Reciprocal Learning for Unsupervised Visible-Infrared Person Re-Identification
School of Computer Science and Technology, Wuhan University of Science and Technology（武汉科技大学计算机科学与技术学院）；Hubei Province Key Laboratory of Intelligent Information Processing and Real-Time Industrial System, Wuhan University of Science and Technology（武汉科技大学智能信息处理与实时工业系统湖北省重点实验室）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；China University of Chinese Academy of Sciences（中国科学院大学）
SceneTransporter: Optimal Transport-Guided Compositional Latent Diffusion for Single-Image Structured 3D Scene Generation
Xi’an Research Institute of Hi-Tech（西安高新技术研究院）；Tsinghua University（清华大学）；SkyWork AI；Tongji University（同济大学）；Fudan University（复旦大学）；Beijing Normal University（北京师范大学）
