arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.07009cs.CV
收藏

视觉-语言辅助的属性学习

Vision-language Assisted Attribute Learning

Beijing University of Posts and Telecommunications(北京邮电大学);China Mobile Research Institute(中国移动研究院);University of Surrey(萨里大学)

Kongming Liang, Xinran Wang, Rui Wang, Donghui Gao, Ling Jin, Weidong Liu, Xiatian Zhu, Zhanyu Ma, Jun Guo

AI总结:

针对大规模属性学习中标签缺失的问题,提出利用视觉-语言模型预测缺失属性概率、随机忽略高置信度缺失标签的辅助损失方法,在VAW数据集上达SOTA,可预测更完整属性。

摘要:

大规模属性标注通常存在不完整、局部缺失的问题,给模型优化带来了重大挑战。现有的属性学习方法在训练时往往将缺失标签视为负样本,或是直接全部忽略,这两种处理方式都会极大地制约模型性能。为突破这些局限,本文利用已有的视觉-语言知识,明确补全缺失标签以强化模型学习。针对输入图像,我们借助现成的视觉-语言模型预测每个缺失属性标签的存在概率,并在训练中随机选择忽略高置信度的缺失标签。该策略在完全忽略缺失标签与将其视为负样本之间取得了良好平衡,因为这些高得分标签能有效反映标签的模糊性。大量实验表明,本文提出的视觉-语言辅助损失函数在新整理的VAW数据集上取得了当前最优性能。定性评估结果验证了该方法能够预测更完整的属性。

英文摘要:

Attribute labeling at large scale is typically incomplete and partial, posing significant challenges to model optimization. Existing attribute learning methods often treat the missing labels as negative or simply ignore them all during training, either of which could hamper the model performance to a great extent. To overcome these limitations, in this paper we leverage the available vision-language knowledge to explicitly disclose the missing labels for enhancing model learning. Given an image, we predict the likelihood of each missing attribute label assisted by an off-the-shelf vision-language model, and randomly select to ignore those with high scores in training. Our strategy strikes a good balance between fully ignoring and negatifying the missing labels, as these high scores are found to be informative on revealing label ambiguity. Extensive experiments show that our proposed vision-language assisted loss can achieve state-of-the-art performance on the newly cleaned VAW dataset. Qualitative evaluation demonstrates the ability of the proposed method in predicting more complete attributes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑