Unified Structured Learning for Simultaneous Human Pose Estimation and Garment Attribute Classification
Shanghai Jiao Tong University(上海交通大学);Nanjing University of Information Science and Technology(南京信息工程大学);National University of Defense Technology(国防科技大学);National University of Singapore(新加坡国立大学)
Jie Shen, Guangcan Liu, Jia Chen, Yuqiang Fang, Jianbin Xie, Yong Yu, Shuicheng Yan
In this paper, we utilize structured learning to simultaneously address two intertwined problems: human pose estimation (HPE) and garment attribute classification (GAC), which are valuable for a variety of computer vision and multimedia applications. Unlike previous works that usually handle the two problems separately, our approach aims to produce a jointly optimal estimation for both HPE and GAC via a unified inference procedure. To this end, we adopt a preprocessing step to detect potential human parts from each image (i.e., a set of "candidates") that allows us to have a manageable input space. In this way, the simultaneous inference of HPE and GAC is converted to a structured learning problem, where the inputs are the collections of candidate ensembles, the outputs are the joint labels of human parts and garment attributes, and the joint feature representation involves various cues such as pose-specific features, garment-specific features, and cross-task features that encode correlations between human parts and garment attributes. Furthermore, we explore the "strong edge" evidence around the potential human parts so as to derive more powerful representations for oriented human parts. Such evidences can be seamlessly integrated into our structured learning model as a kind of energy function, and the learning process could be performed by standard structured Support Vector Machines (SVM) algorithm. However, the joint structure of the two problems is a cyclic graph, which hinders efficient inference. To resolve this issue, we compute instead approximate optima by using an iterative procedure, where in each iteration the variables of one problem are fixed. In this way, satisfactory solutions can be efficiently computed by dynamic programming. Experimental results on two benchmark datasets show the state-of-the-art performance of our approach.
ChatGarment: Garment Estimation, Generation and Editing via Large Language Models
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Shanghai Jiao Tong University(上海交通大学);EPFL(洛桑联邦理工学院);Westlake University(西湖大学);ETH Zürich(苏黎世联邦理工学院);Mila, University of Montreal(蒙特利尔大学米拉研究所);Meshcapade(Meshcapade公司);Stanford University(斯坦福大学)
On Symbiosis of Attribute Prediction and Semantic Segmentation
University of Central Florida(中佛罗里达大学)
在本文中,我们提出采用语义分割,以改善个人相关的属性预测。其核心思想在于,一个属性出现在图像中的概率在空间域中是不均匀的。我们建立我们的属性预测模型与深度语义分割网络。这利用由语义分割学习的定位线索来引导属性预测的注意力到不同属性自然出现的区域。因此,除了预测之外,我们还能够定位属性,尽管在训练期间只能访问图像级标签(弱监督)。我们首先提出了基于语义分割的池化和门控,分别表示为SSP和SSG。在前者中,估计的分割掩模用于从多个语义同质区域汇集属性预测网络的最终激活。在SSG中,同样的想法也适用于网络的中间层。SSP和SSG虽然有效,但由于激活的每个通道都与所有语义分割掩码一起被池化/门控,因此会造成大量的存储器利用。为了解决这个问题,我们提出了共生增强(SA),其中我们每个激活通道只学习一个掩码。SA允许模型选择一个或组合(加权叠加)多个语义映射,以便为每个通道生成适当的掩码。SA同时通过利用属性预测的输出逻辑来指导语义分割任务,将相同的机制应用于反向问题。我们评估了我们提出的方法在CelebA和LFWA数据集上的面部属性,同时对WIDER Attribute和Berkeley Attributes of People的全身属性进行基准测试。我们提出的方法取得了优越的结果相比,以前的作品。
Single Stage Warped Cloth Learning and Semantic-Contextual Attention Feature Fusion for Virtual TryOn
Bharti School of Telecommunications, Indian Institute of Technology, Delhi, India(印度德里印度理工学院巴蒂电信学院);Indian Institute of Information Technology, Sonepat, Haryana, India(印度哈里亚纳邦索内帕特印度信息技术学院);Department of Electrical Engineering, Indian Institute of Technology, Delhi, India(印度德里印度理工学院电气工程系)
ClothFit: Cloth-Human-Attribute Guided Virtual Try-On Network Using 3D Simulated Dataset
RPTU Kaiserslautern-Landau(凯泽斯劳滕-兰道莱茵兰-普法尔茨技术大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Shapematchr GmbH(Shapematchr有限公司)
Beijing University of Posts and Telecommunications(北京邮电大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);ETH Zurich(苏黎世联邦理工学院);University of Tübingen(图宾根大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);City University of Hong Kong(香港城市大学);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学)
Attribute Prototype Network for Zero-Shot Learning
Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);University of Chinese Academy of Sciences(中国科学院大学);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Tübingen(图宾根大学)