多模态大模型 - arXivDaily 专题

2203.13815 2022-03-28 cs.CV 83%

Versatile Multi-Modal Pre-Training for Human-Centric Perception

Fangzhou Hong, Liang Pan, Zhongang Cai, Ziwei Liu

专题命中跨模态检索：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments CVPR 2022; Project Page https://hongfz16.github.io/projects/HCMoCo.html; Codes available at https://github.com/hongfz16/HCMoCo

2201.08125 2022-01-21 cs.CV 83%

Deep Unsupervised Contrastive Hashing for Large-Scale Cross-Modal Text-Image Retrieval in Remote Sensing

Georgii Mikriukov, Mahdyar Ravanbakhsh, Begüm Demir

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2112.07209 2021-12-15 cs.IR cs.AI cs.LG 83%

ACE-BERT: Adversarial Cross-modal Enhanced BERT for E-commerce Retrieval

Boxuan Zhang, Chao Wei, Yan Jin, Weiru Zhang

专题命中跨模态检索：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.AI

2111.07180 2021-11-16 cs.CL cs.LG 83%

Explainable Semantic Space by Grounding Language to Vision with Cross-Modal Contrastive Learning

Yizhen Zhang, Minkyu Choi, Kuan Han, Zhongming Liu

专题命中跨模态检索：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CL

Comments 10 pages, 7 figures, 1 appendix, to be published in Neurips 2021

2012.01345 2021-10-01 cs.CV cs.IR cs.LG 83%

Cross-Modal Retrieval and Synthesis (X-MRS): Closing the Modality Gap in Shared Representation Learning

Ricardo Guerrero, Hai Xuan Pham, Vladimir Pavlovic

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2109.01797 2021-09-07 cs.AI 83%

Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis

Sijie Mai, Ying Zeng, Shuangjia Zheng, Haifeng Hu

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

Comments Under Review

2107.14572 2021-08-10 cs.CV 83%

Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretraining

Xunlin Zhan, Yangxin Wu, Xiao Dong, Yunchao Wei, Minlong Lu, Yichi Zhang, Hang Xu, Xiaodan Liang

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2108.02417 2021-08-06 cs.CV 83%

Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrieval

Xuri Ge, Fuhai Chen, Joemon M. Jose, Zhilong Ji, Zhongqin Wu, Xiao Liu

专题命中跨模态检索：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments 9 pages, 7 figures, Accepted by ACM MM 2021

2108.00724 2021-08-03 cs.CV cs.IR 83%

Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval Service

Zhongwei Xie, Ling Liu, Yanzhao Wu, Lin Li, Luo Zhong

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments accepted by IEEE Transactions on Services Computing

2006.11149 2021-06-02 cs.CV cs.IR 83%

Compositional Learning of Image-Text Query for Image Retrieval

Muhammad Umer Anwaar, Egor Labintcev, Martin Kleinsteuber

专题命中跨模态检索：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Published at IEEE WACV 2021

2105.04242 2021-05-11 stat.ML cs.CV cs.LG 83%

T-EMDE: Sketching-based global similarity for cross-modal retrieval

Barbara Rychalska, Mikolaj Wieczorek, Jacek Dabrowski

专题命中跨模态检索：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 10 pages,5 figures, 4 tables, 1 code snippet

2104.13748 2021-04-29 cs.IR cs.MM 83%

QuTI! Quantifying Text-Image Consistency in Multimodal Documents

Matthias Springstein, Eric Müller-Budack, Ralph Ewerth

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Comments Accepted for publication in: International ACM SIGIR Conference on Research and Development in Information Retrieval 2021

2104.12836 2021-04-28 cs.CV 83%

Multimodal Contrastive Training for Visual Representation Learning

Xin Yuan, Zhe Lin, Jason Kuen, Jianming Zhang, Yilin Wang, Michael Maire, Ajinkya Kale, Baldo Faieta

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

1904.04794 2021-04-22 eess.IV cs.CV cs.IR 83%

CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing

Ushasi Chaudhuri, Biplab Banerjee, Avik Bhattacharya, Mihai Datcu

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

2008.05231 2021-03-03 cs.CV 83%

Fine-grained Visual Textual Alignment for Cross-Modal Retrieval using Transformer Encoders

Nicola Messina, Giuseppe Amato, Andrea Esuli, Fabrizio Falchi, Claudio Gennaro, Stéphane Marchand-Maillet

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Accepted in ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM). arXiv admin note: text overlap with arXiv:2004.09144

2004.09144 2021-01-27 cs.CV 83%

Transformer Reasoning Network for Image-Text Matching and Retrieval

Nicola Messina, Fabrizio Falchi, Andrea Esuli, Giuseppe Amato

专题命中跨模态检索：image-text(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments Presented at ICPR 2020

2011.08899 2020-11-19 cs.CV cs.LG 83%

Multimodal Prototypical Networks for Few-shot Learning

Frederik Pahde, Mihai Puscas, Tassilo Klein, Moin Nabi

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments To appear at WACV 2021

2010.08189 2020-10-19 cs.CV 83%

New Ideas and Trends in Deep Multimodal Content Understanding: A Review

Wei Chen, Weiping Wang, Li Liu, Michael S. Lew

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

Comments Accepted by Neurocomputing

1812.01391 2020-01-03 cs.CV 83%

Semi-Supervised Cross-Modal Retrieval with Label Prediction

Devraj Mandal, Pramod Rao, Soma Biswas

专题命中跨模态检索：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments Updated Version of the Paper has been accepted in IEEE Transactions on Multimedia {https://ieeexplore.ieee.org/document/8907496/}

1909.13689 2019-10-01 cs.MM 83%

Diachronic Cross-modal Embeddings

David Semedo, João Magalhães

专题命中跨模态检索：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.MM

Comments To appear in ACM MM 2019

1805.01963 2019-09-13 cs.CV 83%

MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval

Xin Liu, Zhikai Hu, Haibin Ling, Yiu-ming Cheung

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

Comments 16 pages, accepted by IEEE T-PAMI

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence,2019

1908.07673 2019-08-22 cs.IR cs.MM 83%

Learning Joint Embedding for Cross-Modal Retrieval

Donghuo Zeng

专题命中跨模态检索：cross-modal(title,abstract)；image-text(abstract)；分类 cs.MM

Comments 3 pages, 1 figure, Submitted to ICDM2019 Ph.D. Forum session

1908.07388 2019-08-21 cs.LG cs.CV 83%

Cross-modal Zero-shot Hashing

Xuanwu Liu, Zhao Li, Jun Wang, Guoxian Yu, Carlotta Domeniconi, Xiangliang Zhang

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CV

1908.04979 2019-08-15 cs.LG cs.CV cs.IR stat.ML 83%

Harmonized Multimodal Learning with Gaussian Process Latent Variable Models

Guoli Song, Shuhui Wang, Qingming Huang, Qi Tian

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

1906.04402 2019-07-18 cs.CV 83%

Polysemous Visual-Semantic Embedding for Cross-Modal Retrieval

Yale Song, Mohammad Soleymani

专题命中跨模态检索：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2019. Includes supplementary material. Have updated results on TGIF and MRW

1810.13151 2019-06-13 cs.MM 83%

Semantic Modeling of Textual Relationships in Cross-Modal Retrieval

Jing Yu, Chenghao Yang, Zengchang Qin, Zhuoqian Yang, Yue Hu, Weifeng Zhang

专题命中跨模态检索：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.MM

Comments To appear in KSEM 2019

1805.02997 2018-05-09 cs.CV 83%

Category-Based Deep CCA for Fine-Grained Venue Discovery from Multimodal Data

Yi Yu, Suhua Tang, Kiyoharu Aizawa, Akiko Aizawa

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CV

1710.06582 2017-10-19 cs.MM cs.LG stat.ML 83%

Learning Social Image Embedding with Deep Multimodal Attention Networks

Feiran Huang, Xiaoming Zhang, Zhoujun Li, Tao Mei, Yueying He, Zhonghua Zhao

专题命中跨模态检索：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.MM

Journal ref Proceedings of Thematic Workshops of the 25th ACM Multimedia 2017

1705.09888 2017-05-30 cs.CV 83%

Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval

Peng Xu, Qiyue Yin, Yongye Huang, Yi-Zhe Song, Zhanyu Ma, Liang Wang, Tao Xiang, W. Bastiaan Kleijn, Jun Guo

专题命中跨模态检索：cross-modal(title,abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by Neurocomputing

1602.06697 2017-02-21 cs.CV 83%

Correlation Hashing Network for Efficient Cross-Modal Retrieval

Yue Cao, Mingsheng Long, Jianmin Wang, Philip S. Yu

专题命中跨模态检索：cross-modal(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments 7 pages

1. 跨模态检索 3460 篇

Versatile Multi-Modal Pre-Training for Human-Centric Perception

Deep Unsupervised Contrastive Hashing for Large-Scale Cross-Modal Text-Image Retrieval in Remote Sensing

ACE-BERT: Adversarial Cross-modal Enhanced BERT for E-commerce Retrieval

Explainable Semantic Space by Grounding Language to Vision with Cross-Modal Contrastive Learning

Cross-Modal Retrieval and Synthesis (X-MRS): Closing the Modality Gap in Shared Representation Learning

Hybrid Contrastive Learning of Tri-Modal Representation for Multimodal Sentiment Analysis

Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-modal Pretraining

Structured Multi-modal Feature Embedding and Alignment for Image-Sentence Retrieval

Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval Service

Compositional Learning of Image-Text Query for Image Retrieval

T-EMDE: Sketching-based global similarity for cross-modal retrieval

QuTI! Quantifying Text-Image Consistency in Multimodal Documents

Multimodal Contrastive Training for Visual Representation Learning

CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing

Fine-grained Visual Textual Alignment for Cross-Modal Retrieval using Transformer Encoders

Transformer Reasoning Network for Image-Text Matching and Retrieval

Multimodal Prototypical Networks for Few-shot Learning

New Ideas and Trends in Deep Multimodal Content Understanding: A Review

Semi-Supervised Cross-Modal Retrieval with Label Prediction

Diachronic Cross-modal Embeddings

MTFH: A Matrix Tri-Factorization Hashing Framework for Efficient Cross-Modal Retrieval

Learning Joint Embedding for Cross-Modal Retrieval

Cross-modal Zero-shot Hashing

Harmonized Multimodal Learning with Gaussian Process Latent Variable Models

Polysemous Visual-Semantic Embedding for Cross-Modal Retrieval

Semantic Modeling of Textual Relationships in Cross-Modal Retrieval

Category-Based Deep CCA for Fine-Grained Venue Discovery from Multimodal Data

Learning Social Image Embedding with Deep Multimodal Attention Networks

Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval

Correlation Hashing Network for Efficient Cross-Modal Retrieval