arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1411.7798cs.CV
收藏

基于成对约束的跨模态学习

Cross-Modal Learning via Pairwise Constraints

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shandong University(山东大学)

Ran He, Man Zhang, Liang Wang, Ye Ji, Qiyue Yin

AI总结:

该研究针对多媒体中文本图像成对约束的跨模态学习问题,提出复合正则化框架及无监督聚类、有监督匹配方法,可缩小语义鸿沟,提升聚类与检索准确率。

摘要:

在多媒体应用中,网页文档中的文本和图像组件构成成对约束,潜在指示相同的语义概念。本文研究基于成对约束的跨模态学习,旨在挖掘不同模态中隐藏的共同结构。我们首先提出一种复合正则化框架来处理成对约束,该框架可作为开发跨模态算法的通用平台。针对无监督学习,我们提出一种跨模态子空间聚类方法,用于学习不同模态的共同结构。针对有监督学习,为缩小语义鸿沟并减少成对约束中的离群点,我们提出一种基于复合ℓ21正则化的跨模态匹配方法,并搭配迭代重加权算法以求得全局最优解。大量实验证明了用语义驱动的成对约束进行文本与图像联合建模的优势,且表明所提出的跨模态方法可进一步缩小不同模态间的语义鸿沟,提升聚类/检索准确率。

英文摘要:

In multimedia applications, the text and image components in a web document form a pairwise constraint that potentially indicates the same semantic concept. This paper studies cross-modal learning via the pairwise constraint, and aims to find the common structure hidden in different modalities. We first propose a compound regularization framework to deal with the pairwise constraint, which can be used as a general platform for developing cross-modal algorithms. For unsupervised learning, we propose a cross-modal subspace clustering method to learn a common structure for different modalities. For supervised learning, to reduce the semantic gap and the outliers in pairwise constraints, we propose a cross-modal matching method based on compound ?21 regularization along with an iteratively reweighted algorithm to find the global optimum. Extensive experiments demonstrate the benefits of joint text and image modeling with semantically induced pairwise constraints, and show that the proposed cross-modal methods can further reduce the semantic gap between different modalities and improve the clustering/retrieval accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑