多模态大模型

BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving

Tao Tang, Dafeng Wei, Zhengyu Jia, Tian Gao, Changwei Cai, Chengkai Hou, Peng Jia, Kun Zhan, Haiyang Sun, Jingchen Fan, Yixing Zhao, Fu Liu, Xiaodan Liang, Xianpeng Lang, Yang Wang

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

It's Not a Modality Gap: Characterizing and Addressing the Contrastive Gap

Abrar Fahim, Alex Murphy, Alona Fyshe

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.CL

CSS: Contrastive Semantic Similarity for Uncertainty Quantification of LLMs

Shuang Ao, Stefan Rueger, Advaith Siddharthan

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CL、cs.AI

Comments The paper is accepted by The Conference on Uncertainty in Artificial Intelligence (UAI), 2024

Retrieval Augmented Structured Generation: Business Document Information Extraction As Tool Use

Franz Louis Cesista, Rui Aguiar, Jason Kim, Paolo Acilo

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted by IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR), 2024

Invisible Relevance Bias: Text-Image Retrieval Models Prefer AI-Generated Images

Shicheng Xu, Danyang Hou, Liang Pang, Jingcheng Deng, Jun Xu, Huawei Shen, Xueqi Cheng

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by SIGIR 2024

MVBIND: Self-Supervised Music Recommendation For Videos Via Embedding Space Binding

Jiajie Teng, Huiyu Duan, Yucheng Zhu, Sijing Wu, Guangtao Zhai

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models

Yifei Ming, Yixuan Li

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments The paper is accepted at ICML 2024

Large Language Model Informed Patent Image Retrieval

Hao-Cheng Lo, Jung-Mei Chu, Jieh Hsiang, Chun-Chieh Cho

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 8 pages. Under review

Fooling Contrastive Language-Image Pre-trained Models with CLIPMasterPrints

Matthias Freiberger, Peter Kun, Christian Igel, Anders Sundnes Løvlie, Sebastian Risi

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments This work was supported by a research grant (40575) from VILLUM FONDEN

Vision-and-Language Navigation via Causal Learning

Liuyi Wang, Zongtao He, Ronghao Dang, Mengjiao Shen, Chengju Liu, Qijun Chen

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents

Michael Lutz, Arth Bohra, Manvel Saroyan, Artem Harutyunyan, Giovanni Campagna

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CL、cs.AI

PetKaz at SemEval-2024 Task 3: Advancing Emotion Classification with an LLM for Emotion-Cause Pair Extraction in Conversations

Roman Kazakov, Kseniia Petukhova, Ekaterina Kochmar

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 7 figures, 2 tables, to be published in the Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), for associated code, see https://github.com/sachertort/petkaz-semeval-ecac

Sparse Concept Bottleneck Models: Gumbel Tricks in Contrastive Learning

Andrei Semenov, Vladimir Ivanov, Aleksandr Beznosikov, Alexander Gasnikov

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 23 pages, 1 algorithm, 36 figures

CFIR: Fast and Effective Long-Text To Image Retrieval for Large Corpora

Zijun Long, Xuri Ge, Richard Mccreadie, Joemon Jose

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling

Chengxu Zhuang, Evelina Fedorenko, Jacob Andreas

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CL、cs.AI

MOFI: Learning Image Representations from Noisy Entity Annotated Images

Wentao Wu, Aleksei Timofeev, Chen Chen, Bowen Zhang, Kun Duan, Shuangning Liu, Yantao Zheng, Jonathon Shlens, Xianzhi Du, Zhe Gan, Yinfei Yang

专题命中 跨模态检索 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ICLR 2024

Tri-Modal Motion Retrieval by Learning a Joint Embedding Space

Kangning Yin, Shihao Zou, Yuxuan Ge, Zheng Tian

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Contrastive Learning Is Spectral Clustering On Similarity Graph

Zhiquan Tan, Yifan Zhang, Jingqin Yang, Yang Yuan

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments ICLR 2024; We express our gratitude to the anonymous reviewers for their valuable feedback

AM^2-EmoJE: Adaptive Missing-Modality Emotion Recognition in Conversation via Joint Embedding Learning

Naresh Kumar Devulapally, Sidharth Anand, Sreyasee Das Bhattacharjee, Junsong Yuan

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Enhancing Image Retrieval : A Comprehensive Study on Photo Search using the CLIP Mode

Naresh Kumar Lahajal, Harini S

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Are we describing the same sound? An analysis of word embedding spaces of expressive piano performance

Silvan David Peter, Shreyan Chowdhury, Carlos Eduardo Cancino-Chacón, Gerhard Widmer

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CL、cs.AI

Journal ref Proceedings of the Forum for Information Retrieval Evaluation, FIRE, 2023, Panjim, India

TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection

Hao Sun, Mingyao Zhou, Wenjing Chen, Wei Xie

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments Accepted by AAAI-24

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

Wenhan Yang, Jingdong Gao, Baharan Mirzasoleiman

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.CL

Towards Generalized Multi-stage Clustering: Multi-view Self-distillation

Jiatai Wang, Zhiwei Xu, Xin Wang, Tao Li

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding

Yingjie Niu, Ming Ding, Keisuke Fujii, Kento Ohtani, Alexander Carballo, Kazuya Takeda

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

HKUST at SemEval-2023 Task 1: Visual Word Sense Disambiguation with Context Augmentation and Visual Assistance

Zhuohao Yin, Xin Huang

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection

Yicheng Xiao, Zhuoyan Luo, Yong Liu, Yue Ma, Hengwei Bian, Yatai Ji, Yujiu Yang, Xiu Li

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

3D-MIR: A Benchmark and Empirical Study on 3D Medical Image Retrieval in Radiology

Asma Ben Abacha, Alberto Santamaria-Pang, Ho Hin Lee, Jameson Merkow, Qin Cai, Surya Teja Devarakonda, Abdullah Islam, Julia Gong, Matthew P. Lungren, Thomas Lin, Noel C Codella, Ivan Tarapov

专题命中 跨模态检索 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Visualizing the Diversity of Representations Learned by Bayesian Neural Networks

Dennis Grinwald, Kirill Bykov, Shinichi Nakajima, Marina M. -C. Höhne

专题命中 跨模态检索 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 16 pages, 18 figures

Journal ref Published in Transactions on Machine Learning Research (11/2023)

(Un)likelihood Training for Interpretable Embedding

Jiaxin Wu, Chong-Wah Ngo, Wing-Kwong Chan, Zhijian Hou

专题命中 跨模态检索 ：cross-modal(abstract)；分类 cs.CV、cs.MM

Comments accepted in ACM Transactions on Information Systems