arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.10264cs.LGcs.AIcs.IR
收藏

面向多模态语义嵌入的保序降维方法

Order-Preserving Dimension Reduction for Multimodal Semantic Embedding

Chengyu Gong, Gefei Shen, Luanzheng Guo, Nathan Tallent, Dongfang Zhao

AI总结:

本文提出保序降维方法OPDR,通过新的KNN质量全局度量与闭式映射,在降低多模态嵌入维度的同时保持近邻排序,实验证明其能在显著减少计算成本的同时维持高召回准确率。

摘要:

在多模态数据检索中搜索k近邻(KNN)的计算代价高昂,尤其是由于跨模态相似度度量之间固有的比较困难。近年来多模态机器学习的进展通过将数据映射到共享嵌入空间来解决这一问题;然而,这些嵌入的高维性(数百到数千维)对时间敏感的视觉应用构成了挑战。本工作提出了保序降维(Order-Preserving Dimension Reduction,OPDR),旨在降低嵌入维度的同时在低维空间中保持KNN的排序。OPDR的一个显著组成部分是一种新的度量函数,用于将KNN质量量化为全局指标,并基于该指标推导出目标维度与关键上下文参数之间的闭式映射关系。我们已将OPDR与多种最先进的降维技术、距离函数和嵌入模型集成;在多种多模态数据集上的实验表明,OPDR能够在显著降低计算成本的同时有效保持较高的召回准确率。

英文摘要:

Searching for the $k$-nearest neighbors (KNN) in multimodal data retrieval is computationally expensive, particularly due to the inherent difficulty in comparing similarity measures across different modalities. Recent advances in multimodal machine learning address this issue by mapping data into a shared embedding space; however, the high dimensionality of these embeddings (hundreds to thousands of dimensions) presents a challenge for time-sensitive vision applications. This work proposes Order-Preserving Dimension Reduction (OPDR), aiming to reduce the dimensionality of embeddings while preserving the ranking of KNN in the lower-dimensional space. One notable component of OPDR is a new measure function to quantify KNN quality as a global metric, based on which we derive a closed-form map between target dimensionality and key contextual parameters. We have integrated OPDR with multiple state-of-the-art dimension-reduction techniques, distance functions, and embedding models; experiments on a variety of multimodal datasets demonstrate that OPDR effectively retains recall high accuracy while significantly reducing computational costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑