多模态大模型 - arXivDaily 专题

2108.11656 2022-02-15 cs.CL cs.IR 79%

AR-BERT: Aspect-relation enhanced Aspect-level Sentiment Classification with Multi-modal Explanations

Sk Mainul Islam, Sourangshu Bhattacharya

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Comments Accepted at: The Webconf 2022

2202.05465 2022-02-14 cs.CV 79%

WAD-CMSN: Wasserstein Distance based Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

Guanglong Xu, Zhensheng Hu, Jia Cai

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2202.03587 2022-02-09 eess.AS cs.SD eess.SP 79%

CALM: Contrastive Aligned Audio-Language Multirate and Multimodal Representations

Vin Sachidananda, Shao-Yen Tseng, Erik Marchi, Sachin Kajarekar, Panayiotis Georgiou

专题命中跨模态检索：multimodal(title,abstract)；分类 eess.AS

2201.06779 2022-01-19 cs.AI 79%

Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records

Shuai Niu, Qing Yin, Yunya Song, Yike Guo, Xian Yang

专题命中跨模态检索：multimodal(title)；multi-modal(abstract)；分类 cs.AI

2201.06734 2022-01-19 cs.CV 79%

Cross-modal Contrastive Distillation for Instructional Activity Anticipation

Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2112.01131 2021-12-29 cs.MM cs.CY cs.LG cs.SI 79%

FNR: A Similarity and Transformer-Based Approach to Detect Multi-Modal Fake News in Social Media

Faeze Ghorbanpour, Maryam Ramezani, Mohammad A. Fazli, Hamid R. Rabiee

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.MM

Comments 10 pages, 11 figures, 4 tables and 20 references

2112.10384 2021-12-21 cs.LG cs.CV 79%

Multimodal Adversarially Learned Inference with Factorized Discriminators

Wenxue Chen, Jianke Zhu

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments 9 pages, 6 figures

2112.05827 2021-12-14 cs.CV cs.LG 79%

Quality-Aware Multimodal Biometric Recognition

Sobhan Soleymani, Ali Dabouei, Fariborz Taherkhani, Seyed Mehdi Iranmanesh, Jeremy Dawson, Nasser M. Nasrabadi

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments IEEE Transactions on Biometrics, Behavior, and Identity Science

2112.04182 2021-12-09 cs.CV 79%

Unimodal Face Classification with Multimodal Training

Wenbin Teng, Chongyang Bai

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE International Conference On Automatic Face and Gesture Recognition 2021

2110.12812 2021-10-26 cs.CV cs.LG 79%

Domain Adaptation in Multi-View Embedding for Cross-Modal Video Retrieval

Jonathan Munro, Michael Wray, Diane Larlus, Gabriela Csurka, Dima Damen

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 15 pages

2110.11601 2021-10-26 cs.CV 79%

Multimodal Semi-Supervised Learning for 3D Objects

Zhimin Chen, Longlong Jing, Yang Liang, YingLi Tian, Bing Li

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments BMVC 2021 poster

2109.12750 2021-10-20 cs.LG cs.AI cs.RO 79%

Learning Multimodal Rewards from Rankings

Vivek Myers, Erdem Bıyık, Nima Anari, Dorsa Sadigh

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments 17 pages, 12 figures, 2 tables. Published at Conference on Robot Learning (CoRL) 2021

2108.04049 2021-10-20 cs.CL cs.IR 79%

Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models

Bogdan Kostić, Julian Risch, Timo Möller

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Comments Accepted at the 3rd Workshop on Machine Reading for Question Answering (MRQA) at EMNLP 2021

2110.04599 2021-10-12 cs.LG cs.MM 79%

Embed Everything: A Method for Efficiently Co-Embedding Multi-Modal Spaces

Sarah Di, Robin Yu, Amol Kapoor

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.MM

Comments 7 pages, 4 figures

2109.15153 2021-10-01 cs.CL 79%

Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism

Xiaoqiang Zhang, Ying Chen, Guangyuan Li

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Comments NLPCC 2021

2109.01850 2021-09-13 cs.CL 79%

Supervised Contrastive Learning for Multimodal Unreliable News Detection in COVID-19 Pandemic

Wenjia Zhang, Lin Gui, Yulan He

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CL

Comments Accepted by the CIKM 2021

2109.00796 2021-09-06 cs.CV 79%

Multi-Modal Zero-Shot Sign Language Recognition

Razieh Rastgoo, Kourosh Kiani, Sergio Escalera, Mohammad Sabokrou

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2108.10059

2108.03788 2021-08-19 cs.CV cs.IR 79%

Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images

Zhongwei Xie, Ling Liu, Lin Li, Luo Zhong

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments accepted by CIKM 2021. arXiv admin note: substantial text overlap with arXiv:2108.00705

2102.08655 2021-08-11 cs.CL 79%

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

Nora Hollenstein, Cedric Renggli, Benjamin Glaus, Maria Barrett, Marius Troendle, Nicolas Langer, Ce Zhang

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CL

Journal ref Frontiers of Human Neuroscience 2021

1911.12763 2021-07-14 cs.CV 79%

Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA

Mikhail Fain, Niall Twomey, Andrey Ponikar, Ryan Fox, Danushka Bollegala

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

2107.02575 2021-07-07 cs.CV 79%

Contrastive Multimodal Fusion with TupleInfoNCE

Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

2105.07921 2021-06-18 cs.CV 79%

BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval

Gencer Sumbul, Arne de Wall, Tristan Kreuziger, Filipe Marcelino, Hugo Costa, Pedro Benevides, Mário Caetano, Begüm Demir, Volker Markl

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at the IEEE Geoscience and Remote Sensing Magazine. Our code is available online at https://git.tu-berlin.de/rsim/BigEarthNet-MM_19-classes_models. arXiv admin note: substantial text overlap with arXiv:2001.06372

2101.05068 2021-06-15 cs.CV 79%

Probabilistic Embeddings for Cross-Modal Retrieval

Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio de Rezende, Yannis Kalantidis, Diane Larlus

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted to CVPR 2021; Code is available at https://github.com/naver-ai/pcme

2105.07128 2021-05-18 cs.CV 79%

FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal Retrieval

Xin Liu, Xingzhi Wang, Yiu-ming Cheung

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments 16 pages, 7 figures

Journal ref IEEE Transactions on Neural Networks and Learning Systems, 2021

2105.03567 2021-05-11 cs.LG cs.AI cs.CR 79%

Multimodal and Contrastive Learning for Click Fraud Detection

Weibin Li, Qiwei Zhong, Qingyang Zhao, Hongchun Zhang, Xiaonan Meng

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.AI

Comments Accepted to DeMal@WWW 2021

2104.07098 2021-04-16 cs.CV 79%

StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis

Moustafa Meshry, Yixuan Ren, Larry S Davis, Abhinav Shrivastava

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

2104.04991 2021-04-13 cs.CV 79%

Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval

Wei Chen, Yu Liu, Erwin M. Bakker, Michael S. Lew

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by Pattern Recognition

2103.13061 2021-03-25 cs.CV 79%

Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning

Amaia Salvador, Erhan Gundogdu, Loris Bazzani, Michael Donoser

专题命中跨模态检索：cross-modal(title,abstract)；分类 cs.CV

Comments CVPR 2021

2103.04731 2021-03-09 cs.CV 79%

Self-Augmented Multi-Modal Feature Embedding

Shinnosuke Matsuo, Seiichi Uchida, Brian Kenji Iwana

专题命中跨模态检索：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at ICASSP2021

2006.06325 2021-01-14 cs.CV cs.LG eess.IV 79%

CoMIR: Contrastive Multimodal Image Representation for Registration

Nicolas Pielawski, Elisabeth Wetzer, Johan Öfverstedt, Jiahao Lu, Carolina Wählby, Joakim Lindblad, Nataša Sladoje

专题命中跨模态检索：multimodal(title,abstract)；分类 cs.CV

Comments 21 pages, 11 figures

Journal ref NeurIPS 2020

1. 跨模态检索 3475 篇

AR-BERT: Aspect-relation enhanced Aspect-level Sentiment Classification with Multi-modal Explanations

WAD-CMSN: Wasserstein Distance based Cross-Modal Semantic Network for Zero-Shot Sketch-Based Image Retrieval

CALM: Contrastive Aligned Audio-Language Multirate and Multimodal Representations

Label Dependent Attention Model for Disease Risk Prediction Using Multimodal Electronic Health Records

Cross-modal Contrastive Distillation for Instructional Activity Anticipation

FNR: A Similarity and Transformer-Based Approach to Detect Multi-Modal Fake News in Social Media

Multimodal Adversarially Learned Inference with Factorized Discriminators

Quality-Aware Multimodal Biometric Recognition

Unimodal Face Classification with Multimodal Training

Domain Adaptation in Multi-View Embedding for Cross-Modal Video Retrieval

Multimodal Semi-Supervised Learning for 3D Objects

Learning Multimodal Rewards from Rankings

Multi-modal Retrieval of Tables and Texts Using Tri-encoder Models

Embed Everything: A Method for Efficiently Co-Embedding Multi-Modal Spaces

Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism

Supervised Contrastive Learning for Multimodal Unreliable News Detection in COVID-19 Pandemic

Multi-Modal Zero-Shot Sign Language Recognition

Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images

Decoding EEG Brain Activity for Multi-Modal Natural Language Processing

Dividing and Conquering Cross-Modal Recipe Retrieval: from Nearest Neighbours Baselines to SoTA

Contrastive Multimodal Fusion with TupleInfoNCE

BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval

Probabilistic Embeddings for Cross-Modal Retrieval

FDDH: Fast Discriminative Discrete Hashing for Large-Scale Cross-Modal Retrieval

Multimodal and Contrastive Learning for Click Fraud Detection

StEP: Style-based Encoder Pre-training for Multi-modal Image Synthesis

Integrating Information Theory and Adversarial Learning for Cross-modal Retrieval

Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning

Self-Augmented Multi-Modal Feature Embedding

CoMIR: Contrastive Multimodal Image Representation for Registration