arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.05597cs.LG
收藏

一种用于多模态特征表示的判别向量框架

A Discriminative Vectorial Framework for Multi-modal Feature Representation

Ryerson University(瑞尔森大学)

Lei Gao, Ling Guan

摘要:

随着感知技术和计算技术的飞速发展,表征同一模式或现象的多模态数据源越来越受到人们的关注。因此,找到从这些多模态数据源中探索有用信息的方法已迅速成为必要。本文提出了一种基于多模态散列(MH)和判别相关最大化(DCM)分析的多模态特征表示的判别向量框架。具体而言,所提出的框架是能够最大限度地减少不同模态之间的语义相似性MH和严格的内在歧视性表示跨多个数据源DCM分析联合,使一个新的矢量框架的多模态特征表示。此外,所提出的特征表示策略进行了分析,并进一步优化基于规范和非规范的情况下,分别。因此,所生成的特征表示导致高质量的输入数据源的有效利用,从而在各种应用中产生改进的、有时相当令人印象深刻的结果。利用经典特征和基于深度神经网络(DNN)的特征,应用于图像和多媒体分析和识别任务,包括数据可视化,人脸识别,对象识别,跨模态(文本-图像)识别和音频情感识别,证明了所提出的框架的有效性和通用性。实验结果表明,所提出的解决方案优于国家的最先进的统计机器学习(SML)和DNN算法。

英文摘要:

Due to the rapid advancements of sensory and computing technology, multi-modal data sources that represent the same pattern or phenomenon have attracted growing attention. As a result, finding means to explore useful information from these multi-modal data sources has quickly become a necessity. In this paper, a discriminative vectorial framework is proposed for multi-modal feature representation in knowledge discovery by employing multi-modal hashing (MH) and discriminative correlation maximization (DCM) analysis. Specifically, the proposed framework is capable of minimizing the semantic similarity among different modalities by MH and exacting intrinsic discriminative representations across multiple data sources by DCM analysis jointly, enabling a novel vectorial framework of multi-modal feature representation. Moreover, the proposed feature representation strategy is analyzed and further optimized based on canonical and non-canonical cases, respectively. Consequently, the generated feature representation leads to effective utilization of the input data sources of high quality, producing improved, sometimes quite impressive, results in various applications. The effectiveness and generality of the proposed framework are demonstrated by utilizing classical features and deep neural network (DNN) based features with applications to image and multimedia analysis and recognition tasks, including data visualization, face recognition, object recognition; cross-modal (text-image) recognition and audio emotion recognition. Experimental results show that the proposed solutions are superior to state-of-the-art statistical machine learning (SML) and DNN algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑