arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1207.1423cs.LGcs.DBstat.ML
收藏

基于双翼和声的文本与图像关联挖掘

Mining Associated Text and Images with Dual-Wing Harmoniums

Carnegie Mellon University(卡内基梅隆大学)

Eric P. Xing, Rong Yan, Alexander G. Hauptmann

摘要:

我们提出了一个多翼风琴模型挖掘多媒体数据,扩展和改进早期模型的基础上,双层随机场,捕捉隐藏的主题方面和观察到的输入之间的双向依赖关系。该模型可以看作是两层有向模型(如LDA)的无向对应物,用于类似的任务,但在推理/学习成本权衡,潜在主题表示和主题混合机制方面存在显着差异。特别是,我们的模型有利于有效的推理和强大的主题混合,并可能提供高的灵活性,在建模的潜在主题空间。一个对比发散和变分算法的学习。我们将我们的模型专门用于标题图像的双翼风琴,将多变量泊松用于单词计数,多变量高斯用于颜色直方图。我们给出了该模型在新闻视频集分类、检索和图像注释中应用的实证结果,并报告了与各种现存模型的广泛比较。

英文摘要:

We propose a multi-wing harmonium model for mining multimedia data that extends and improves on earlier models based on two-layer random fields, which capture bidirectional dependencies between hidden topic aspects and observed inputs. This model can be viewed as an undirected counterpart of the two-layer directed models such as LDA for similar tasks, but bears significant difference in inference/learning cost tradeoffs, latent topic representations, and topic mixing mechanisms. In particular, our model facilitates efficient inference and robust topic mixing, and potentially provides high flexibilities in modeling the latent topic spaces. A contrastive divergence and a variational algorithm are derived for learning. We specialized our model to a dual-wing harmonium for captioned images, incorporating a multivariate Poisson for word-counts and a multivariate Gaussian for color histogram. We present empirical results on the applications of this model to classification, retrieval and image annotation on news video collections, and we report an extensive comparison with various extant models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑