arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.08677cs.CV
收藏

Label2Label:一个面向多属性学习的语言建模框架

Label2Label: A Language Modeling Framework for Multi-Attribute Learning

Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)

Wanhua Li, Zhexuan Cao, Jianjiang Feng, Jie Zhou, Jiwen Lu

摘要:

对象通常与多个属性相关联,并且这些属性通常表现出高度的相关性。属性之间复杂关系的建模对多属性学习提出了巨大挑战。本文提出了一个简单而通用的框架Label2Label利用复杂的属性相关性。Label2Label是从语言建模的角度进行多属性预测的第一次尝试。具体来说,它将每个属性标签视为描述样本的“单词”。由于每个样本都被标注了多个属性标签,这些“词”将自然形成一个无序但有意义的“句子”,它描绘了相应样本的语义信息。受NLP中预训练语言模型的显著成功的启发,Label2Label引入了一种图像条件掩码语言模型,它随机地从标签“句子”中屏蔽一些“单词”标记,并旨在基于被屏蔽的“句子”和图像特征所传达的上下文来恢复它们。我们的直觉是,如果神经网络可以根据上下文和剩余的属性提示推断出缺失的属性,那么就可以很好地把握实例属性关系。Label2Label概念简单,经验强大。在不结合特定于任务的先验知识和高度专业化的网络设计的情况下,与高度定制的特定于领域的方法相比,我们的方法在三种不同的多属性学习任务上实现了最先进的结果。代码可在https://github.com/Li-Wanhua/Label2Label上获得。

英文摘要:

Objects are usually associated with multiple attributes, and these attributes often exhibit high correlations. Modeling complex relationships between attributes poses a great challenge for multi-attribute learning. This paper proposes a simple yet generic framework named Label2Label to exploit the complex attribute correlations. Label2Label is the first attempt for multi-attribute prediction from the perspective of language modeling. Specifically, it treats each attribute label as a "word" describing the sample. As each sample is annotated with multiple attribute labels, these "words" will naturally form an unordered but meaningful "sentence", which depicts the semantic information of the corresponding sample. Inspired by the remarkable success of pre-training language models in NLP, Label2Label introduces an image-conditioned masked language model, which randomly masks some of the "word" tokens from the label "sentence" and aims to recover them based on the masked "sentence" and the context conveyed by image features. Our intuition is that the instance-wise attribute relations are well grasped if the neural net can infer the missing attributes based on the context and the remaining attribute hints. Label2Label is conceptually simple and empirically powerful. Without incorporating task-specific prior knowledge and highly specialized network designs, our approach achieves state-of-the-art results on three different multi-attribute learning tasks, compared to highly customized domain-specific methods. Code is available at https://github.com/Li-Wanhua/Label2Label.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑