多模态大模型

Cross-Modal Attention Alignment Network with Auxiliary Text Description for zero-shot sketch-based image retrieval

Hanwen Su, Ge Song, Kai Huang, Jiyan Wang, Ming Yang

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Zero-shot Composed Image Retrieval Considering Query-target Relationship Leveraging Masked Image-text Pairs

Huaying Zhang, Rintaro Yanagi, Ren Togo, Takahiro Ogawa, Miki Haseyama

专题命中 跨模态检索 ：image-text(title,abstract)；分类 cs.CV

Comments Accepted as a conference paper in IEEE ICIP 2024

Data curation via joint example selection further accelerates multimodal learning

Talfan Evans, Nikhil Parthasarathy, Hamza Merzic, Olivier J. Henaff

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments Main text: 9 pages, 5 figures, 3 tables, 1 algorithm. Appendix: 7 pages, 5 figures, 1 table, 2. algorithm

Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning

Chi-Sheng Chen, Chun-Shu Wei

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments 19 pages, 14 figures

Towards a multimodal framework for remote sensing image change retrieval and captioning

Roger Ferrod, Luigi Di Caro, Dino Ienco

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

High-level Codes and Fine-grained Weights for Online Multi-modal Hashing Retrieval

Yu-Wei Zhan, Xiao-Ming Wu, Xin Luo, Yinwei Wei, Xin-Shun Xu

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.MM

Comments 32 pages, 4 figures

MORE: Multi-mOdal REtrieval Augmented Generative Commonsense Reasoning

Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CL

Comments Published as a conference paper at ACL Findings 2024

Open-World Human-Object Interaction Detection via Multi-modal Prompts

Jie Yang, Bingliang Li, Ailing Zeng, Lei Zhang, Ruimao Zhang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Comments CVPR24. arXiv admin note: text overlap with arXiv:2305.12252

Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities

Mingcheng Li, Dingkang Yang, Xiao Zhao, Shuaibing Wang, Yan Wang, Kun Yang, Mingyang Sun, Dongliang Kou, Ziyun Qian, Lihua Zhang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2024

Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning

Alex Jinpeng Wang, Linjie Li, Yiqi Lin, Min Li, Lijuan Wang, Mike Zheng Shou

专题命中 跨模态检索 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV

Comments 12 pages. The website is \url{https://fingerrec.github.io/visincontext}

FinEmbedDiff: A Cost-Effective Approach of Classifying Financial Documents with Vector Sampling using Multi-modal Embedding Models

Anjanava Biswas, Wrick Talukdar

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI

Comments 10 pages, 3 figures

Journal ref International Research Journal of Modernization in Engineering Technology and Science: Vol. 06 (2024): No. 5, 6142-6152

MGI: Multimodal Contrastive pre-training of Genomic and Medical Imaging

Jiaying Zhou, Mingzhou Jiang, Junde Wu, Jiayuan Zhu, Ziyue Wang, Yueming Jin

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoning

Cheng Tan, Jingxuan Wei, Linzhuang Sun, Zhangyang Gao, Siyuan Li, Bihui Yu, Ruifeng Guo, Stan Z. Li

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Under review

Reverse Image Retrieval Cues Parametric Memory in Multimodal LLMs

Jialiang Xu, Michael Moor, Jure Leskovec

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

${M^2D}$NeRF: Multi-Modal Decomposition NeRF with 3D Feature Fields

Ning Wang, Lefei Zhang, Angel X Chang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

COM3D: Leveraging Cross-View Correspondence and Cross-Modal Mining for 3D Retrieval

Hao Wu, Ruochong LI, Hao Wang, Hui Xiong

专题命中 跨模态检索 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by ICME 2024 oral

Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum

Tao Meng, Fuchen Zhang, Yuntao Shou, Wei Ai, Nan Yin, Keqin Li

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Comments 10 pages, 4 figures

Next Visit Diagnosis Prediction via Medical Code-Centric Multimodal Contrastive EHR Modelling with Hierarchical Regularisation

Heejoon Koo

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

Comments Accepted to EACL 2024 (The 18th Conference of the European Chapter of the Association for Computational Linguistics)

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs

Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, Saining Xie

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments Project page: https://tsb0601.github.io/mmvp_blog/

Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Compositional Understanding

Le Zhang, Rabiul Awal, Aishwarya Agrawal

专题命中 跨模态检索 ：cross-modal(title)；image-text(abstract)；分类 cs.CV

Comments CVPR 2024

Large Language Models and Multimodal Retrieval for Visual Word Sense Disambiguation

Anastasia Kritharoula, Maria Lymperaiou, Giorgos Stamou

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Comments Conference on Empirical Methods in Natural Language Processing (EMNLP) 2023

Journal ref Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

Global Contrastive Training for Multimodal Electronic Health Records with Language Supervision

Yingbo Ma, Suraj Kolla, Zhenhong Hu, Dhruv Kaliraman, Victoria Nolan, Ziyuan Guan, Yuanfang Ren, Brooke Armfield, Tezcan Ozrazgat-Baslanti, Jeremy A. Balch, Tyler J. Loftus, Parisa Rashidi, Azra Bihorac, Benjamin Shickel

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Comments 12 pages, 3 figures. arXiv admin note: text overlap with arXiv:2403.04012

MMIDR: Teaching Large Language Model to Interpret Multimodal Misinformation via Knowledge Distillation

Longzheng Wang, Xiaohan Xu, Lei Zhang, Jiarui Lu, Yongxiu Xu, Hongbo Xu, Minghao Tang, Chuang Zhang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Comments 10 pages, 3 figures

KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning

Cong-Duy Nguyen, Thong Nguyen, Xiaobao Wu, Anh Tuan Luu

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to NAACL 2024

CMViM: Contrastive Masked Vim Autoencoder for 3D Multi-modal Representation Learning for AD classification

Guangqian Yang, Kangrui Du, Zhihan Yang, Ye Du, Yongping Zheng, Shujun Wang

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Comments 11 pages, 1 figure

Path-GPTOmic: A Balanced Multi-modal Learning Framework for Survival Outcome Prediction

Hongxiao Wang, Yang Yang, Zhuo Zhao, Pengfei Gu, Nishchal Sapkota, Danny Z. Chen

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE International Symposium on Biomedical Imaging (ISBI 2024)

SnapNTell: Enhancing Entity-Centric Visual Question Answering with Retrieval Augmented Multimodal LLM

Jielin Qiu, Andrea Madotto, Zhaojiang Lin, Paul A. Crook, Yifan Ethan Xu, Xin Luna Dong, Christos Faloutsos, Lei Li, Babak Damavandi, Seungwhan Moon

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

A Privacy-Preserving Framework with Multi-Modal Data for Cross-Domain Recommendation

Li Wang, Lei Sang, Quangui Zhang, Qiang Wu, Min Xu

专题命中 跨模态检索 ：multi-modal(title,abstract)；分类 cs.AI

BiVRec: Bidirectional View-based Multimodal Sequential Recommendation

Jiaxi Hu, Jingtong Gao, Xiangyu Zhao, Yuehong Hu, Yuxuan Liang, Yiqi Wang, Ming He, Zitao Liu, Hongzhi Yin

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.AI

BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning

Siyuan Liang, Mingli Zhu, Aishan Liu, Baoyuan Wu, Xiaochun Cao, Ee-Chien Chang

专题命中 跨模态检索 ：multimodal(title,abstract)；分类 cs.CV

Comments The paper lacks some work that needs to be cited

Journal ref CVPR 2024