多模态大模型

Rebalanced Vision-Language Retrieval Considering Structure-Aware Distillation

Yang Yang, Wenjuan Xi, Luping Zhou, Jinhui Tang

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Wills Aligner: Multi-Subject Collaborative Brain Visual Decoding

Guangyin Bao, Qi Zhang, Zixuan Gong, Jialei Zhou, Wei Fan, Kun Yi, Usman Naseem, Liang Hu, Duoqian Miao

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments AAAI 2025, 16 pages

A dual contrastive framework

Yuan Sun, Zhao Zhang, Jorge Ortiz

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

WordVIS: A Color Worth A Thousand Words

Umar Khan, Saifullah, Stefan Agne, Andreas Dengel, Sheraz Ahmed

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Dynamic Modality-Camera Invariant Clustering for Unsupervised Visible-Infrared Person Re-identification

Yiming Yang, Weipeng Hu, Haifeng Hu

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Emotion-Aligned Contrastive Learning Between Images and Music

Shanti Stewart, Kleanthis Avramidis, Tiantian Feng, Shrikanth Narayanan

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments Published at ICASSP 2024. Code: https://github.com/shantistewart/Emo-CLIM

A Framework For Image Synthesis Using Supervised Contrastive Learning

Yibin Liu, Jianyu Zhang, Li Zhang, Shijian Li, Gang Pan

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Relation-Aware Meta-Learning for Zero-shot Sketch-Based Image Retrieval

Yang Liu, Jiale Du, Xinbo Gao, Jungong Han

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Exploring Optimal Transport-Based Multi-Grained Alignments for Text-Molecule Retrieval

Zijun Min, Bingshuai Liu, Liang Zhang, Jia Song, Jinsong Su, Song He, Xiaochen Bo

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Comments BIBM 2024 Regular Paper

Towards Neural Foundation Models for Vision: Aligning EEG, MEG, and fMRI Representations for Decoding, Encoding, and Modality Conversion

Matteo Ferrante, Tommaso Boccato, Grigorii Rashkov, Nicola Toschi

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

CleanerCLIP: Fine-grained Counterfactual Semantic Augmentation for Backdoor Defense in Contrastive Learning

Yuan Xun, Siyuan Liang, Xiaojun Jia, Xinwei Liu, Xiaochun Cao

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Pseudo-triplet Guided Few-shot Composed Image Retrieval

Bohan Hou, Haoqiang Lin, Haokun Wen, Meng Liu, Mingzhu Xu, Xuemeng Song

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 10pages

Plug, Play, and Fuse: Zero-Shot Joint Decoding via Word-Level Re-ranking Across Diverse Vocabularies

Sai Koneru, Matthias Huck, Miriam Exel, Jan Niehues

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments WMT 2024

BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval

Imanol Miranda, Ander Salaberria, Eneko Agirre, Gorka Azkune

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to NeurIPS 24 Datasets and Benchmarks Track; Project page at: https://imirandam.github.io/BiVLC_project_page/

G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models

Pengyue Jia, Yiding Liu, Xiaopeng Li, Yuhao Wang, Yantong Du, Xiao Han, Xuetao Wei, Shuaiqiang Wang, Dawei Yin, Xiangyu Zhao

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS2024

Improving Hateful Meme Detection through Retrieval-Guided Contrastive Learning

Jingbiao Mei, Jinghong Chen, Weizhe Lin, Bill Byrne, Marcus Tomalin

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2024 Main. The code is available from: https://github.com/JingbiaoMei/RGCL

Multi-path Exploration and Feedback Adjustment for Text-to-Image Person Retrieval

Bin Kang, Bin Chen, Junjie Wang, Yong Xu

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Eliminating the Language Bias for Visual Question Answering with fine-grained Causal Intervention

Ying Liu, Ge Bai, Chenji Lu, Shilong Li, Zhang Zhang, Ruifang Liu, Wenbin Guo

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Journal ref 2024 IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, ON, Canada, 2024, pp. 1-6

VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding

Houlun Chen, Xin Wang, Hong Chen, Zeyang Zhang, Wei Feng, Bin Huang, Jia Jia, Wenwu Zhu

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by 38th NeurIPS Datasets & Benchmarks Track (NeurIPS 2024)

LLaMP: Large Language Model Made Powerful for High-fidelity Materials Knowledge Retrieval and Distillation

Yuan Chiang, Elvis Hsieh, Chia-Hong Chou, Janosh Riebesell

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 32 pages, 5 figures

Temporal Image Caption Retrieval Competition -- Description and Results

Jakub Pokrywka, Piotr Wierzchoń, Kornel Weryszko, Krzysztof Jassem

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Journal ref Proceedings of the 18th Conference on Computer Science and Intelligence Systems, M. Ganzha, L. Maciaszek, M. Paprzycki, D. Ślęzak (eds). ACSIS, Vol. 35, pages 1331-1336 (2023)

Coarse-to-fine Alignment Makes Better Speech-image Retrieval

Lifeng Zhou, Yuke Li

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition -- And Ways to Overcome Them

Harish Haresamudram, Apoorva Beedu, Mashfiqui Rabbi, Sankalita Saha, Irfan Essa, Thomas Ploetz

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning

Yingjin Song, Denis Paperno, Albert Gatt

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 18 pages, 12 figures, accepted by INLG 2024

Improving Composed Image Retrieval via Contrastive Learning with Scaling Positives and Negatives

Zhangchi Feng, Richong Zhang, Zhijie Nie

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted to ACM MM 2024 Regular Papers

X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs

Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments Accepted at ECCV2024

Reproducible scaling laws for contrastive language-image learning

Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, Jenia Jitsev

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments CVPR 2023. Version with minor extension. Original: CVPR2023/html/Cherti_Reproducible_Scaling_Laws_for_Contrastive_Language-Image_Learning_CVPR_2023_paper" target="_blank" rel="noopener">https://openaccess.thecvf.com/content/CVPR2023/html/Cherti_Reproducible_Scaling_Laws_for_Contrastive_Language-Image_Learning_CVPR_2023_paper

Journal ref Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 2818-2829

CEIA: CLIP-Based Event-Image Alignment for Open-World Event-Based Understanding

Wenhao Xu, Wenming Weng, Yueyi Zhang, Zhiwei Xiong

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Semantic Compositions Enhance Vision-Language Contrastive Learning

Maxwell Aladago, Lorenzo Torresani, Soroush Vosoughi

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Nomic Embed Vision: Expanding the Latent Space

Zach Nussbaum, Brandon Duderstadt, Andriy Mulyar

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI