arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1901.01860cs.LGstat.ML
收藏

JECL:图文对的联合嵌入和聚类学习

JECL: Joint Embedding and Cluster Learning for Image-Text Pairs

University of Washington(华盛顿大学);University of California, Los Angeles(加州大学洛杉矶分校)

Sean T. Yang, Kuan-Hao Huang, Bill Howe

摘要:

我们提出了JECL,一种通过训练具有正则化聚类和对齐目标的并行编码器来聚类图像标题对的方法,同时学习表示和聚类分配。这些图像-字幕对经常出现在高价值的应用程序中,在这些应用程序中,生成结构化训练数据的成本很高,但自由文本描述很常见。JECL通过最小化图像和文本的分布与组合的联合目标分布之间的Kullback-Leibler发散,并优化图像和文本的软聚类分配之间的Jensen-Shannon发散来进行训练。正则化也适用于JECL,以防止平凡的解决方案。实验表明,JECL优于单视图和多视图方法在大型基准图像字幕数据集,并显着鲁棒丢失的字幕和不同的数据大小。

英文摘要:

We propose JECL, a method for clustering image-caption pairs by training parallel encoders with regularized clustering and alignment objectives, simultaneously learning both representations and cluster assignments. These image-caption pairs arise frequently in high-value applications where structured training data is expensive to produce, but free-text descriptions are common. JECL trains by minimizing the Kullback-Leibler divergence between the distribution of the images and text to that of a combined joint target distribution and optimizing the Jensen-Shannon divergence between the soft cluster assignments of the images and text. Regularizers are also applied to JECL to prevent trivial solutions. Experiments show that JECL outperforms both single-view and multi-view methods on large benchmark image-caption datasets, and is remarkably robust to missing captions and varying data sizes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑