多模态大模型

Cephalo: Multi-Modal Vision-Language Models for Bio-Inspired Materials Analysis and Design

Markus J. Buehler

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL

MuDPT: Multi-modal Deep-symphysis Prompt Tuning for Large Pre-trained Vision-Language Models

Yongzhu Miao, Shasha Li, Jintao Tang, Ting Wang

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Comments The paper has been accepted by ICME 2023

Vision-Language Consistency Guided Multi-modal Prompt Learning for Blind AI Generated Image Quality Assessment

Jun Fu, Wei Zhou, Qiuping Jiang, Hantao Liu, Guangtao Zhai

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE Signal Processing Letter

Solution for SMART-101 Challenge of CVPR Multi-modal Algorithmic Reasoning Task 2024

Jinwoo Ahn, Junhyeok Park, Min-Jun Kim, Kang-Hyeon Kim, So-Yeong Sohn, Yun-Ji Lee, Du-Seong Chang, Yu-Jung Heo, Eun-Sol Kim

专题命中 图文多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.AI

Assessing News Thumbnail Representativeness: Counterfactual text can enhance the cross-modal matching ability

Yejun Yoon, Seunghyun Yoon, Kunwoo Park

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.CL

Comments ACL 2024 (findings), 16 pages

GLaMM: Pixel Grounding Large Multimodal Model

Hanoona Rasheed, Muhammad Maaz, Sahal Shaji Mullappilly, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M. Anwer, Erix Xing, Ming-Hsuan Yang, Fahad S. Khan

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments CVPR 2024

Concept Visualization: Explaining the CLIP Multi-modal Embedding Using WordNet

Loris Giulivi, Giacomo Boracchi

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Accepted for publication at IJCNN 2024

Understanding Multimodal Deep Neural Networks: A Concept Selection View

Chenming Shang, Hengyuan Zhang, Hao Wen, Yujiu Yang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

On Speculative Decoding for Multimodal Large Language Models

Mukul Gagrani, Raghavv Goel, Wonseok Jeon, Junyoung Park, Mingu Lee, Christopher Lott

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments Accepted as a spotlight paper to ELVM workshop at CVPR 2024

A Review of Multi-Modal Large Language and Vision Models

Kilian Carolan, Laura Fennelly, Alan F. Smeaton

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CL、cs.AI

Comments 33 pages, 1 figure

Open-Vocabulary Federated Learning with Multimodal Prototyping

Huimin Zeng, Zhenrui Yue, Dong Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at NAACL 2024

Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction

Inhwan Bae, Junoh Lee, Hae-Gon Jeon

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Accepted at CVPR 2024

m3P: Towards Multimodal Multilingual Translation with Multimodal Prompt

Jian Yang, Hongcheng Guo, Yuwei Yin, Jiaqi Bai, Bing Wang, Jiaheng Liu, Xinnian Liang, Linzheng Cahi, Liqun Yang, Zhoujun Li

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments COLING 2024

Multi-Modal Hallucination Control by Visual Information Grounding

Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, Stefano Soatto

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Journal ref IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024

Leveraging Chat-Based Large Vision Language Models for Multimodal Out-Of-Context Detection

Fatma Shalabi, Hichem Felouat, Huy H. Nguyen, Isao Echizen

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 13 pages, 6 figures , conference

Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipation

Razvan-George Pasca, Alexey Gavryushin, Muhammad Hamza, Yen-Ling Kuo, Kaichun Mo, Luc Van Gool, Otmar Hilliges, Xi Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Towards Grounded Visual Spatial Reasoning in Multi-Modal Vision Language Models

Navid Rajabi, Jana Kosecka

专题命中 图文多模态 ：multi-modal(title)；image-text(abstract)；分类 cs.CV、cs.CL

Comments Accepted to DMLR @ ICLR 2024

ITEm: Unsupervised Image-Text Embedding Learning for eCommerce

Baohao Liao, Michael Kozielski, Sanjika Hewavitharana, Jiangbo Yuan, Shahram Khadivi, Tomer Lancewicki

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.CL

OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network

Tiancheng Zhao, Peng Liu, Kyusong Lee

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Published at IET CV

Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models

Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, Yang Liu

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings

Daniel Rose, Vaishnavi Himakunthala, Andy Ouyang, Ryan He, Alex Mei, Yujie Lu, Michael Saxon, Chinmay Sonar, Diba Mirza, William Yang Wang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Zoom-shot: Fast and Efficient Unsupervised Zero-Shot Transfer of CLIP to Vision Encoders with Multimodal Loss

Jordan Shipard, Arnold Wiliem, Kien Nguyen Thanh, Wei Xiang, Clinton Fookes

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 15 pages

Multimodal Parameter-Efficient Few-Shot Class Incremental Learning

Marco D'Alessandro, Alberto Alonso, Enrique Calabrés, Mikel Galar

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Journal ref 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Paris, France, 2023, pp. 3385-3395

Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training

Longtian Qiu, Shan Ning, Xuming He

专题命中 图文多模态 ：image-text(title,abstract)；分类 cs.CV、cs.AI

Comments AAAI 2024.Open sourced, Code and Model Available

UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts

Chenlu Zhan, Yufei Zhang, Yu Lin, Gaoang Wang, Hongwei Wang

专题命中 图文多模态 ：cross-modal(title,abstract)；分类 cs.CV、cs.AI

Structural Information Guided Multimodal Pre-training for Vehicle-centric Perception

Xiao Wang, Wentao Wu, Chenglong Li, Zhicheng Zhao, Zhe Chen, Yukai Shi, Jin Tang

专题命中 图文多模态 ：multimodal(title)；image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by AAAI-2024

Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models

Xinwei Li, Li Lin, Shuai Wang, Chen Qian

专题命中 图文多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.CL

Multimodal Data Augmentation for Image Captioning using Diffusion Models

Changrong Xiao, Sean Xin Xu, Kunpeng Zhang

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Socratis: Are large multimodal models emotionally aware?

Katherine Deng, Arijit Ray, Reuben Tan, Saadia Gabriel, Bryan A. Plummer, Kate Saenko

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments ICCV 2023 WECIA

Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts

Deepanway Ghosal, Navonil Majumder, Roy Ka-Wei Lee, Rada Mihalcea, Soujanya Poria

专题命中 图文多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.AI