多模态大模型

MAViC: Multimodal Active Learning for Video Captioning

Gyanendra Das, Xavier Thomas, Anant Raj, Vikram Gupta

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Cross-Modal Similarity-Based Curriculum Learning for Image Captioning

Hongkuan Zhang, Saku Sugawara, Akiko Aizawa, Lei Zhou, Ryohei Sasano, Koichi Takeda

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2022

DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention

Fenglin Liu, Xian Wu, Shen Ge, Xuancheng Ren, Wei Fan, Xu Sun, Yuexian Zou

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published in ACM TKDD2022 (ACM Transactions on Knowledge Discovery from Data)

On the Use of Modality-Specific Large-Scale Pre-Trained Encoders for Multimodal Sentiment Analysis

Atsushi Ando, Ryo Masumura, Akihiko Takashima, Satoshi Suzuki, Naoki Makishima, Keita Suzuki, Takafumi Moriya, Takanori Ashihara, Hiroshi Sato

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to SLT 2022

Visual Answer Localization with Cross-modal Mutual Knowledge Transfer

Yixuan Weng, Bin Li

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 4 pages, 3 figures, 2 tables

Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks

Colin Leong, Joshua Nemecek, Jacob Mansdorfer, Anna Filighera, Abraham Owodunni, Daniel Whitenack

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 1 figure, 3 tables, accepted to and presented at EMNLP 2022

Journal ref EMNLP 2022

Cross-Modal Alignment Learning of Vision-Language Conceptual Systems

Taehyeong Kim, Hyeonseop Song, Byoung-Tak Zhang

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 19 pages, 4 figures

Towards Multimodal Vision-Language Models Generating Non-Generic Text

Wes Robbins, Zanyar Zohourianshahzadi, Jugal Kalita

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref 2021 International Conference on Natural Language Processing

Generalizing Multimodal Pre-training into Multilingual via Language Acquisition

Liang Zhang, Anwen Hu, Qin Jin

专题命中 图文多模态 ：multimodal(title)；image-text(abstract)；分类 cs.CL、cs.AI

Comments 14 pages, 5 figures

Beyond a Pre-Trained Object Detector: Cross-Modal Textual and Visual Context for Image Captioning

Chia-Wen Kuo, Zsolt Kira

专题命中 图文多模态 ：cross-modal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Comments paper accepted in CVPR 2022

On the Importance of Karaka Framework in Multi-modal Grounding

Sai Kiran Gorthi, Radhika Mamidi

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Unified Contrastive Learning in Image-Text-Label Space

Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, Jianfeng Gao

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2022

Assessing Multilingual Fairness in Pre-trained Multimodal Representations

Jialu Wang, Yang Liu, Xin Eric Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 15 pages, 18 figures

Two-stream Hierarchical Similarity Reasoning for Image-text Matching

Ran Chen, Hanli Wang, Lei Wang, Sam Kwong

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.MM

MAGIC: Multimodal relAtional Graph adversarIal inferenCe for Diverse and Unpaired Text-based Image Captioning

Wenqiao Zhang, Haochen Shi, Jiannan Guo, Shengyu Zhang, Qingpeng Cai, Juncheng Li, Sihui Luo, Yueting Zhuang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref AAAI 2022

RedCaps: web-curated image-text data created by the people, for the people

Karan Desai, Gaurav Kaul, Zubin Aysola, Justin Johnson

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Comments NeurIPS 2021 Datasets and Benchmarks. Website: https://redcaps.xyz

Negative Sample is Negative in Its Own Way: Tailoring Negative Sentences for Image-Text Retrieval

Zhihao Fan, Zhongyu Wei, Zejun Li, Siyuan Wang, Jianqing Fan

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Exploiting Cross-Modal Prediction and Relation Consistency for Semi-Supervised Image Captioning

Yang Yang, Hongchen Wei, Hengshu Zhu, Dianhai Yu, Hui Xiong, Jian Yang

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching

Ali Furkan Biten, Andres Mafla, Lluis Gomez, Dimosthenis Karatzas

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted WACV 2022

Hierarchical Multimodal Transformer to Summarize Videos

Bin Zhao, Maoguo Gong, Xuelong Li

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models

Po-Yao Huang, Mandela Patrick, Junjie Hu, Graham Neubig, Florian Metze, Alexander Hauptmann

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments accepted by NAACL 2021

Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts

Soravit Changpinyo, Piyush Sharma, Nan Ding, Radu Soricut

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Comments IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021). Our dataset is available at https://github.com/google-research-datasets/conceptual-12m

Confidence-aware Non-repetitive Multimodal Transformers for TextCaps

Zhaokai Wang, Renda Bao, Qi Wu, Si Liu

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 9 pages; Accepted by AAAI 2021

Reading Isn't Believing: Adversarial Attacks On Multi-Modal Neurons

David A. Noever, Samantha E. Miller Noever

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Similarity Reasoning and Filtration for Image-Text Matching

Haiwen Diao, Ying Zhang, Lin Ma, Huchuan Lu

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.MM

Comments 14 pages, 8 figures, Accepted by AAAI2021

Multimodal Learning for Hateful Memes Detection

Yi Zhou, Zhenhao Chen

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Pretraining for Dense Video Captioning

Gabriel Huang, Bo Pang, Zhenhai Zhu, Clara Rivera, Radu Soricut

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments AACL-IJCNLP 2020

Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting

Po-Yao Huang, Junjie Hu, Xiaojun Chang, Alexander Hauptmann

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACL 2020

More Grounded Image Captioning by Distilling Image-Text Matching Model

Yuanen Zhou, Meng Wang, Daqing Liu, Zhenzhen Hu, Hanwang Zhang

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted by CVPR 2020

Vision-Dialog Navigation by Exploring Cross-modal Memory

Yi Zhu, Fengda Zhu, Zhaohuan Zhan, Bingqian Lin, Jianbin Jiao, Xiaojun Chang, Xiaodan Liang

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments CVPR2020