多模态大模型

LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding

Senqiao Yang, Jiaming Liu, Ray Zhang, Mingjie Pan, Zoey Guo, Xiaoqi Li, Zehui Chen, Peng Gao, Yandong Guo, Shanghang Zhang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Bootstrapping Vision-Language Learning with Decoupled Language Pre-training

Yiren Jian, Chongyang Gao, Soroush Vosoughi

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2023 (spotlight). The code is available at https://github.com/yiren-jian/BLIText

Simple Image-level Classification Improves Open-vocabulary Object Detection

Ruohuan Fang, Guansong Pang, Xiao Bai

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted at AAAI 2024

Relax Image-Specific Prompt Requirement in SAM: A Single Generic Prompt for Segmenting Camouflaged Objects

Jian Hu, Jiayi Lin, Weitong Cai, Shaogang Gong

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI2024

Image Captioning with Multi-Context Synthetic Data

Feipeng Ma, Yizhou Zhou, Fengyun Rao, Yueyi Zhang, Xiaoyan Sun

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by AAAI 2024

p-Laplacian Adaptation for Generative Pre-trained Vision-Language Models

Haoyuan Wu, Xinyun Zhang, Peng Xu, Peiyu Liao, Xufeng Yao, Bei Yu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI24. The first two authors contributed equally to this paper

Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation

Bichen Wu, Ruizhe Cheng, Peizhao Zhang, Tianren Gao, Peter Vajda, Joseph E. Gonzalez

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 19 pages, 6 figures

DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations

Ping Hu, Ximeng Sun, Stan Sclaroff, Kate Saenko

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments TPAMI. arXiv admin note: substantial text overlap with arXiv:2206.09541

Uni3DL: Unified Model for 3D and Language Understanding

Xiang Li, Jian Ding, Zhaoyang Chen, Mohamed Elhoseiny

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Self-Supervised Open-Ended Classification with Small Visual Language Models

Mohammad Mahdi Derakhshani, Ivona Najdenkoska, Cees G. M. Snoek, Marcel Worring, Yuki M. Asano

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

LightCLIP: Learning Multi-Level Interaction for Lightweight Vision-Language Models

Ying Nie, Wei He, Kai Han, Yehui Tang, Tianyu Guo, Fanyi Du, Yunhe Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuning

Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, Yu-Gang Jiang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments techical report; work in progress

Transform, Contrast and Tell: Coherent Entity-Aware Multi-Image Captioning

Jingqiang Chen

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 32 pages, 11 tables, 3 figures

Improving Image Captioning via Predicting Structured Concepts

Ting Wang, Weidong Chen, Yuanhe Tian, Yan Song, Zhendong Mao

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by EMNLP 2023 (Main Conference, Oral)

Text and Click inputs for unambiguous open vocabulary instance segmentation

Nikolai Warner, Meera Hahn, Jonathan Huang, Irfan Essa, Vighnesh Birodkar

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 20 pages, 9 figures, 8 tables

Understanding the Vulnerability of CLIP to Image Compression

Cangxiong Chen, Vinay P. Namboodiri, Julian Padget

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments R0-FoMo: Workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models at NeurIPS 2023

POAR: Towards Open Vocabulary Pedestrian Attribute Recognition

Yue Zhang, Suchen Wang, Shichao Kan, Zhenyu Weng, Yigang Cen, Yap-peng Tan

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement

Yanyan Wei, Zhao Zhang, Jiahuan Ren, Xiaogang Xu, Richang Hong, Yi Yang, Shuicheng Yan, Meng Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Domain Aligned CLIP for Few-shot Classification

Muhammad Waleed Gondal, Jochen Gast, Inigo Alonso Ruiz, Richard Droste, Tommaso Macri, Suren Kumar, Luitpold Staudigl

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments To appear at WACV 2024

Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Clutter

Georgios Tziafas, Yucheng Xu, Arushi Goel, Mohammadreza Kasaei, Zhibin Li, Hamidreza Kasaei

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Poster CoRL 2023. Dataset and code available here: https://github.com/gtziafas/OCID-VLG

MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, Mohamed Elhoseiny

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

UMDFood: Vision-language models boost food composition compilation

Peihua Ma, Yixin Wu, Ning Yu, Yang Zhang, Michael Backes, Qin Wang, Cheng-I Wei

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments 13 pages, 9 figures

Uniformly Distributed Category Prototype-Guided Vision-Language Framework for Long-Tail Recognition

Siming Fu, Xiaoxuan He, Xinpeng Ding, Yuchen Cao, Hualiang Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 11pages, 5figures

Journal ref ACM MM2023

Customize StyleGAN with One Hand Sketch

Shaocong Zhang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments preprint

JourneyDB: A Benchmark for Generative Image Understanding

Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, Jifeng Dai, Yu Qiao, Limin Wang, Hongsheng Li

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted to the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS 2023)

Semi-Supervised Panoptic Narrative Grounding

Danni Yang, Jiayi Ji, Xiaoshuai Sun, Haowei Wang, Yinan Li, Yiwei Ma, Rongrong Ji

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments ACM MM 2023

Impressions: Understanding Visual Semiotics and Aesthetic Impact

Julia Kruk, Caleb Ziems, Diyi Yang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments To be published in EMNLP 2023

ArchBERT: Bi-Modal Understanding of Neural Architectures and Natural Languages

Mohammad Akbari, Saeed Ranjbar Alvar, Behnam Kamranian, Amin Banitalebi-Dehkordi, Yong Zhang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CL

Comments CoNLL 2023

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jinwoo Shin

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments NeurIPS 2023

Meta-learning For Vision-and-language Cross-lingual Transfer

Hanxu Hu, Frank Keller

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CL

Comments MRL2023 (co-located with EMNLP2023)