多模态大模型

Multimodal Parameter-Efficient Few-Shot Class Incremental Learning

Marco D'Alessandro, Alberto Alonso, Enrique Calabrés, Mikel Galar

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Paris, France, 2023, pp. 3385-3395

Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training

Longtian Qiu, Shan Ning, Xuming He

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments AAAI 2024.Open sourced, Code and Model Available

UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts

Chenlu Zhan, Yufei Zhang, Yu Lin, Gaoang Wang, Hongwei Wang

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Structural Information Guided Multimodal Pre-training for Vehicle-centric Perception

Xiao Wang, Wentao Wu, Chenglong Li, Zhicheng Zhao, Zhe Chen, Yukai Shi, Jin Tang

专题命中 图文多模态 ：multimodal(title)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI-2024

Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models

Xinwei Li, Li Lin, Shuai Wang, Chen Qian

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Multimodal Data Augmentation for Image Captioning using Diffusion Models

Changrong Xiao, Sean Xin Xu, Kunpeng Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Socratis: Are large multimodal models emotionally aware?

Katherine Deng, Arijit Ray, Reuben Tan, Saadia Gabriel, Bryan A. Plummer, Kate Saenko

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023 WECIA

Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts

Deepanway Ghosal, Navonil Majumder, Roy Ka-Wei Lee, Rada Mihalcea, Soujanya Poria

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text

Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Youngjae Yu, Ludwig Schmidt, William Yang Wang, Yejin Choi

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments NeurIPS D&B 2023. Project homepage: https://github.com/allenai/mmc4

Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality

Harman Singh, Pengchuan Zhang, Qifan Wang, Mengjiao Wang, Wenhan Xiong, Jingfei Du, Yu Chen

专题命中 图文多模态 ：image-text(title)；multimodal(abstract)；分类 cs.CV、cs.CL

Comments EMNLP 2023 (long paper, main conference)

Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis

Pin-Er Chen, Po-Ya Angela Wang, Hsin-Yu Chou, Yu-Hsiang Tseng, Shu-Kai Hsieh

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 10 pages, 9 figures

Leveraging Image-Text Similarity and Caption Modification for the DataComp Challenge: Filtering Track and BYOD Track

Shuhei Yokoo, Peifei Zhu, Yuchi Ishikawa, Mikihiro Tanaka, Masayoshi Kondo, Hirokatsu Kataoka

专题命中 图文多模态 ：image-text(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at the ICCV 2023 Workshop on Towards the Next Generation of Computer Vision Datasets: DataComp Track

Knowledge Extraction and Distillation from Large-Scale Image-Text Colonoscopy Records Leveraging Large Language and Vision Models

Shuo Wang, Yan Zhu, Xiaoyuan Luo, Zhiwei Yang, Yizhe Zhang, Peiyao Fu, Manning Wang, Zhijian Song, Quanlin Li, Pinghong Zhou, Yike Guo

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Multimodal Analysis Of Google Bard And GPT-Vision: Experiments In Visual Reasoning

David Noever, Samantha Elizabeth Miller Noever

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistants

Tianyu Yu, Jinyi Hu, Yuan Yao, Haoye Zhang, Yue Zhao, Chongyi Wang, Shan Wang, Yinxv Pan, Jiao Xue, Dahai Li, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Neurons in Pretrained Text-Only Transformers

Sarah Schwettmann, Neil Chowdhury, Samuel Klein, David Bau, Antonio Torralba

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Oral presentation at ICCV CLVL 2023

Noise-aware Learning from Web-crawled Image-Text Data for Image Captioning

Wooyoung Kang, Jonghwan Mun, Sungjun Lee, Byungseok Roh

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Findings of Factify 2: Multimodal Fake News Detection

S Suryavardan, Shreyash Mishra, Megha Chakraborty, Parth Patwa, Anku Rani, Aman Chadha, Aishwarya Reganti, Amitava Das, Amit Sheth, Manoj Chinnakotla, Asif Ekbal, Srijan Kumar

专题命中 图文多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments Defactify2 @AAAI 2023

FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning

Haokun Chen, Yao Zhang, Denis Krompass, Jindong Gu, Volker Tresp

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.AI、cs.MM

Multi-modal Pre-training for Medical Vision-language Understanding and Generation: An Empirical Study with A New Benchmark

Li Xu, Bo Liu, Ameer Hamza Khan, Lu Fan, Xiao-Ming Wu

专题命中 图文多模态 ：multi-modal(title)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Published as oral paper in CHIL 2023

CLIP Multi-modal Hashing: A new baseline CLIPMH

Jian Zhu, Mingkai Sheng, Mingda Ke, Zhangmin Huang, Jingfei Chang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.MM

Comments submit to ICASSP2024

Evaluating Picture Description Speech for Dementia Detection using Image-text Alignment

Youxiang Zhu, Nana Lin, Xiaohui Liang, John A. Batsis, Robert M. Roth, Brian MacWhinney

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination

Haoxuan Li, Yi Bin, Junrong Liao, Yang Yang, Heng Tao Shen

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted at ACM MM 2023

Multimodal Color Recommendation in Vector Graphic Documents

Qianru Qiu, Xueting Wang, Mayu Otani

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Comments Accepted to ACM MM 2023

Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment

Shengqiong Wu, Hao Fei, Wei Ji, Tat-Seng Chua

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping

Junyang Wang, Ming Yan, Yi Zhang, Jitao Sang

专题命中 图文多模态 ：cross-modal(title)；image-text(abstract)；分类 cs.CV、cs.CL

Comments 11 pages, 15 figures, has been accepted by IJCAI 2023

Multi-modal reward for visual relationships-based image captioning

Ali Abedi, Hossein Karshenas, Peyman Adibi

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 18 pages, 10 figures

INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification

Yinuo Zhang, Zhulin Tao, Xi Wang, Tongyue Wang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments AAAI-23 DeFactify 2 Workshop (2nd Prize)

Selectively Hard Negative Mining for Alleviating Gradient Vanishing in Image-Text Matching

Zheng Li, Caili Guo, Xin Wang, Zerun Feng, Zhongtian Du

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.MM

Video Captioning with Guidance of Multimodal Latent Topics

Shizhe Chen, Jia Chen, Qin Jin, Alexander Hauptmann

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ACM Multimedia 2017