用于有效多模态检索的复合相关Quantization
Composite Correlation Quantization for Efficient Multimodal Retrieval
摘要：
在现代搜索引擎和社交网络中，从大规模多模态数据库中进行高效的相似性检索是普遍存在的。为了支持跨内容模态的查询，系统应该启用跨模态相关性和计算高效的索引。虽然散列方法在实现这一目标方面显示出巨大的潜力，但目前的尝试通常无法以无缝方案学习同构散列码，即，它们在连续同构空间中嵌入多个模态，并将阈值嵌入到二进制码中，这会导致检索准确性的大幅损失。在本文中，我们提出了一种新的复合相关量化（CCQ）模型的无缝多模式哈希。具体来说，CCQ共同找到将不同模态转换为同构潜在空间的相关最大映射，并学习将同构潜在特征转换为紧凑二进制代码的复合量化器。设计了一个优化框架，通过最小化重建和量化误差来保持模态内相似性和模态间相关性，该框架可以在线性时间内从配对和部分配对数据进行训练。一组全面的实验清楚地表明，CCQ对最先进的散列方法的单峰和跨模态检索的优越效果和效率。
英文摘要：
Efficient similarity retrieval from large-scale multimodal database is pervasive in modern search engines and social networks. To support queries across content modalities, the system should enable cross-modal correlation and computation-efficient indexing. While hashing methods have shown great potential in achieving this goal, current attempts generally fail to learn isomorphic hash codes in a seamless scheme, that is, they embed multiple modalities in a continuous isomorphic space and separately threshold embeddings into binary codes, which incurs substantial loss of retrieval accuracy. In this paper, we approach seamless multimodal hashing by proposing a novel Composite Correlation Quantization (CCQ) model. Specifically, CCQ jointly finds correlation-maximal mappings that transform different modalities into isomorphic latent space, and learns composite quantizers that convert the isomorphic latent features into compact binary codes. An optimization framework is devised to preserve both intra-modal similarity and inter-modal correlation through minimizing both reconstruction and quantization errors, which can be trained from both paired and partially paired data in linear time. A comprehensive set of experiments clearly show the superior effectiveness and efficiency of CCQ against the state of the art hashing methods for both unimodal and cross-modal retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
