Multi-label learning has attracted significant interests in computer vision recently, finding applications in many vision tasks such as multiple object recognition and automatic image annotation. Associating multiple labels to a complex image is very difficult, not only due to the intricacy of describing the image, but also because of the incompleteness nature of the observed labels. Existing works on the problem either ignore the label-label and instance-instance correlations or just assume these correlations are linear and unstructured. Considering that semantic correlations between images are actually structured, in this paper we propose to incorporate structured semantic correlations to solve the missing label problem of multi-label learning. Specifically, we project images to the semantic space with an effective semantic descriptor. A semantic graph is then constructed on these images to capture the structured correlations between them. We utilize the semantic graph Laplacian as a smooth term in the multi-label learning formulation to incorporate the structured semantic correlations. Experimental results demonstrate the effectiveness of the proposed semantic descriptor and the usefulness of incorporating the structured semantic correlations. We achieve better results than state-of-the-art multi-label learning methods on four benchmark datasets.
Reliable Representation Learning for Incomplete Multi-View Missing Multi-Label Classification
Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology(深圳视觉目标检测与识别重点实验室,哈尔滨工业大学);PAMI Research Group, Department of Computer and Information Science, University of Macau(PAMI研究组,澳门大学计算机与信息科学系)
Label Structure Preserving Contrastive Embedding for Multi-Label Learning with Missing Labels
Jiangsu University(江苏大学);Jiangsu Engineering Research Center of Big Data Ubiquitous Perception and Intelligent Agriculture Applications(江苏省大数据泛在感知与智慧农业应用工程研究中心);MIIT Key Laboratory of Pattern Analysis and Machine Intelligence(工信部模式分析与机器智能重点实验室);Nanjing University of Aeronautics and Astronautics(南京航空航天大学)
Robust Multi-Modal Face Anti-Spoofing with Domain Adaptation: Tackling Missing Modalities, Noisy Pseudo-Labels, and Model Degradation
College of Computer and Cyber Security, Fujian Normal University(计算机与网络安全部,福建师范大学);Department of Computer Science, National Tsing Hua University(计算机科学系,国立清华大学)
Multi-label Learning with Missing Labels using Mixed Dependency Graphs
Tencent AI Lab(腾讯人工智能实验室);KAUST(阿卜杜拉国王科技大学);SUNY-Albany(纽约州立大学奥尔巴尼分校)
这项工作的重点是缺失标签的多标签学习(MLML)问题,其目的是在给定具有这些标签的不完整/部分集合的训练实例的情况下,用多个类标签标记每个测试实例。处理缺失标签的关键是通过依赖图将每个实例的每个标签视为一个节点,将标签信息从提供的标签传播到缺失的标签。我们通过利用不同类型的标签依赖关系来构建这个图。具体地说,实例级相似性被用作无向边来连接跨不同实例的标签节点,而语义标签层次结构被用作有向边来连接不同的类。这个基础图被称为混合依赖图,因为它包括无向边和有向边。此外,我们提出了另外两种类型的标签依赖关系,以连接不同类的标签节点。一个是类同现,它也被编码为无向边。结合基图,我们得到了一个新的混合图MG-CO(mixed graph with co-occurrence)。另一种是对整个标签矩阵进行稀疏和低秩分解,以嵌入所有标签上的高阶依赖。结合基图,新的混合图被称为MG-SL(稀疏低秩分解混合图)。基于MG-CO和MG-SL,我们提出了MLML问题的两个凸直推公式,分别记为MLMG-CO和MLMG-SL。两个重要的应用,包括图像标注和基于标签的图像检索,可以联合使用我们提出的方法处理。基准数据集上的实验表明,我们的方法在性能和鲁棒性方面都有显着的改进。
Exploring Structured Semantic Prior for Multi Label Recognition with Incomplete Labels
Xidian University(西安电子科技大学);Hangzhou Zhuoxi Institute of Brain and Intelligence(杭州卓兮脑与智能研究院);Tsinghua University(清华大学);BNRist(北京国家信息科学与技术研究中心);the University of Sheffield(谢菲尔德大学)
Mitigating representation bias caused by missing pixels in methane plume detection
Leiden Institute for Advanced Computer Science (LIACS)(莱顿高级计算机科学研究所);SRON Space Research Organization Netherlands(荷兰空间研究组织SRON);Department of Earth Sciences(地球科学系);Φ \Phi -lab, ESA-ESRIN(Φ实验室,ESA-ESRIN);Chair for AI Methodology (AIM) at RWTH Aachen(RWTH亚琛大学人工智能方法学教授职位)
Simple and Robust Loss Design for Multi-Label Learning with Missing Labels
OPPO Research Institute(OPPO研究院);School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University(上海交通大学电子信息与电气工程学院);Academy for Engineering and Technology, Fudan University(复旦大学工程与应用技术研究院)
Structured Set Matching Networks for One-Shot Part Labeling
Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Semantic Machines(语义机器公司);University of Washington(华盛顿大学)
图表通常描述复杂的现象,并作为视觉和文本推理的良好测试平台。然而,使用自然图像理解方法来理解图需要大量的图训练数据集,这是非常难以获得的。相反,这可以作为标记图、图像或两者之间的匹配问题来解决。这个问题是非常具有挑战性的,因为缺乏显着的颜色和纹理呈现局部线索模糊,需要全局推理。我们考虑一次性部分标记的问题:在目标图像中标记对象的多个部分,仅给出该类别的单个源图像。对于这个集合到集合的匹配问题,我们引入了结构化集合匹配网络(SSMN),这是一种结合了卷积神经网络的结构化预测模型。使用全局归一化来训练SSMN,以最大化对应元素之间的局部匹配分数和所有匹配元素之间的全局一致性分数,同时还强制执行两个集合之间的匹配约束。SSMN在三个标签传输场景中的表现明显优于几个强大的基线:在200多个类别的新图表数据集上评估的图表到图表;在Pascal Part Dataset之上构建的数据集上评估的图像到图像;以及在这些数据集之间传输标签时评估的图像到图表。
When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identification
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Glasgow(格拉斯哥大学);Boston University(波士顿大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection
Department of Computer Engineering, Ankara University(安卡拉大学计算机工程系);Department of Modeling and Simulation, Graduate School of Informatics, METU(信息学院建模与模拟系,METU)
A Concise yet Effective model for Non-Aligned Incomplete Multi-view and Missing Multi-label Learning
College of Computer Science and Technology(计算机科学与技术学院);College of Artificial Intelligence(人工智能学院);Nanjing University of Aeronautics and Astronautics(南京航空航天大学)
Learning deep structured network for weakly supervised change detection
Data61-CSIRO(澳大利亚联邦科学与工业研究组织数据61);Australian National University(澳大利亚国立大学);ShanghaiTech University(上海科技大学);The University of Western Australia(西澳大学);Murdoch University(莫道克大学)
Hyperbolic Structured Classification for Robust Single Positive Multi-label Learning
School of Advanced Technology(高级技术学院);Xi'an Jiaotong-Liverpool University(西安交通大学利物浦大学);School of Data Science(数据科学学院);Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Digital Innovation Research Center(数字创新研究中心);Duke Kunshan University(杜克大学昆山分校)