视觉大模型 / VLM - arXivDaily 专题

2406.14481 2024-06-21 cs.LG cs.AI cs.NE q-bio.NC 62%

Revealing Vision-Language Integration in the Brain with Multimodal Networks

Vighnesh Subramaniam, Colin Conwell, Christopher Wang, Gabriel Kreiman, Boris Katz, Ignacio Cases, Andrei Barbu

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments ICML 2024; 23 pages, 11 figures

2406.12946 2024-06-21 eess.AS cs.AI cs.CL cs.LG 62%

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models

Vahid Noroozi, Zhehuai Chen, Somshubra Majumdar, Steve Huang, Jagadeesh Balam, Boris Ginsburg

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted for Interspeech 2024

2406.10328 2024-06-18 cs.CV cs.CL cs.LG 62%

From Pixels to Prose: A Large Dataset of Dense Image Captions

Vasu Singla, Kaiyu Yue, Sukriti Paul, Reza Shirkavand, Mayuka Jayawardhana, Alireza Ganjdanesh, Heng Huang, Abhinav Bhatele, Gowthami Somepalli, Tom Goldstein

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments pixelprose 16M dataset

2406.10318 2024-06-18 cs.CV cs.AI 62%

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding

Tuo Zhang, Tiantian Feng, Yibin Ni, Mengqin Cao, Ruying Liu, Katharine Butler, Yanjun Weng, Mi Zhang, Shrikanth S. Narayanan, Salman Avestimehr

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2404.16829 2024-05-27 cs.CV cs.AI cs.CL 62%

Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials

Ye Fang, Zeyi Sun, Tong Wu, Jiaqi Wang, Ziwei Liu, Gordon Wetzstein, Dahua Lin

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

Comments Project Page: https://sunzey.github.io/Make-it-Real/

2403.15593 2024-05-20 cs.CV cs.LG 62%

FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs

Sepehr Dehdashtian, Lan Wang, Vishnu Naresh Boddeti

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments The Twelfth International Conference on Learning Representations (ICLR) 2024

2310.20225 2024-04-30 cs.CV cs.AI 62%

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

Yu Hao, Fan Yang, Hao Huang, Shuaihang Yuan, Sundeep Rangan, John-Ross Rizzo, Yao Wang, Yi Fang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2404.15532 2024-04-25 cs.HC cs.AI cs.CL cs.CV cs.MA 62%

BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis

Shuhang Lin, Wenyue Hua, Lingyao Li, Che-Jui Chang, Lizhou Fan, Jianchao Ji, Hang Hua, Mingyu Jin, Jiebo Luo, Yongfeng Zhang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments 26 pages, 14 figures The data and code for this project are accessible at https://github.com/agiresearch/battleagent

2302.10174 2024-04-02 cs.CV cs.LG 62%

Towards Universal Fake Image Detectors that Generalize Across Generative Models

Utkarsh Ojha, Yuheng Li, Yong Jae Lee

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2403.18525 2024-03-28 cs.CV cs.CL cs.LG 62%

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP

Reza Abbasi, Mohammad Samiei, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Oral accepted at OODCV 2023(http://www.ood-cv.org)

2403.12910 2024-03-20 cs.RO cs.AI cs.LG 62%

Yell At Your Robot: Improving On-the-Fly from Language Corrections

Lucy Xiaoyang Shi, Zheyuan Hu, Tony Z. Zhao, Archit Sharma, Karl Pertsch, Jianlan Luo, Sergey Levine, Chelsea Finn

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments Project website: https://yay-robot.github.io/

2311.07593 2024-03-18 cs.CL cs.CV cs.LG 62%

Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification

Reza Esfandiarpoor, Stephen H. Bach

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments ICLR 2024

2403.09410 2024-03-15 cs.CV cs.AI 62%

XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization

Yequan Bie, Luyang Luo, Zhixuan Chen, Hao Chen

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2311.15108 2024-02-13 cs.CV cs.AI 62%

Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision

Nicholas Lui, Bryan Chia, William Berrios, Candace Ross, Douwe Kiela

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments The Appendix can be found at https://bit.ly/dp-appendix; Added link to code and fixed formatting (Feb 10 2024)

2402.02662 2024-02-07 cs.CV cs.CL cs.LG 62%

Image-Caption Encoding for Improving Zero-Shot Generalization

Eric Yang Yu, Christopher Liao, Sathvik Ravi, Theodoros Tsiligkaridis, Brian Kulis

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2401.02600 2024-01-08 cs.CV cs.AI 62%

Object-oriented backdoor attack against image captioning

Meiling Li, Nan Zhong, Xinpeng Zhang, Zhenxing Qian, Sheng Li

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

2308.06595 2023-12-27 cs.CL cs.AI cs.CV 62%

VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use

Yonatan Bitton, Hritik Bansal, Jack Hessel, Rulin Shao, Wanrong Zhu, Anas Awadalla, Josh Gardner, Rohan Taori, Ludwig Schmidt

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted to NeurIPS 2023, Datasets and Benchmarks. Website: https://visit-bench.github.io/

2303.06854 2023-12-21 cs.CV cs.CL cs.CR cs.LG 62%

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

Wenhan Yang, Jingdong Gao, Baharan Mirzasoleiman

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2312.12383 2023-12-20 cs.AI cs.CL cs.CV 62%

Visual AI and Linguistic Intelligence Through Steerability and Composability

David Noever, Samantha Elizabeth Miller Noever

专题命中其他VLM ：multimodal large language model(abstract)；分类 cs.CV、cs.AI

2312.08958 2023-12-15 cs.LG cs.AI cs.RO 62%

LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers

Taewook Nam, Juyong Lee, Jesse Zhang, Sung Ju Hwang, Joseph J. Lim, Karl Pertsch

专题命中其他VLM ：vision-language model(abstract)；分类 cs.AI、cs.LG

Comments 2nd Workshop on Agent Learning in Open-Endedness (ALOE) at NeurIPS 2023

2312.08878 2023-12-15 cs.CV cs.LG stat.AP 62%

Domain Prompt Learning with Quaternion Networks

Qinglong Cao, Zhengqin Xu, Yuntian Chen, Chao Ma, Xiaokang Yang

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2308.03793 2023-12-15 cs.CV cs.LG 62%

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

Xuefeng Hu, Ke Zhang, Lu Xia, Albert Chen, Jiajia Luo, Yuyin Sun, Ken Wang, Nan Qiao, Xiao Zeng, Min Sun, Cheng-Hao Kuo, Ram Nevatia

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted as Oral Paper by 2024 IEEE CVF Winter Conference on Applications of Computer Vision (WACV)

2307.00910 2023-12-13 cs.CV cs.AI 62%

CoPL: Contextual Prompt Learning for Vision-Language Understanding

Koustava Goswami, Srikrishna Karanam, Prateksha Udhayanan, K J Joseph, Balaji Vasan Srinivasan

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.AI

Comments Accepted at AAAI 2024

2311.13258 2023-11-23 cs.CV cs.CL cs.LG 62%

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation

Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to EMNLP 2023

2307.10350 2023-10-27 cs.LG cs.CV 62%

Improving Multimodal Datasets with Image Captioning

Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco, Sewoong Oh, Ludwig Schmidt

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted at NeurIPS 2023 Datasets & Benchmarks

2305.14095 2023-10-26 cs.CV cs.LG 62%

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

Sangwoo Mo, Minkyu Kim, Kyungmin Lee, Jinwoo Shin

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments NeurIPS 2023

2305.01278 2023-10-25 cs.CV cs.AI cs.CL 62%

VPGTrans: Transfer Visual Prompt Generator across LLMs

Ao Zhang, Hao Fei, Yuan Yao, Wei Ji, Li Li, Zhiyuan Liu, Tat-Seng Chua

专题命中其他VLM ：MLLM(abstract)；分类 cs.CV、cs.AI

Comments Project Website: https://vpgtrans.github.io Code: https://github.com/VPGTrans/VPGTrans NeurIPS 2023

2310.03957 2023-10-09 cs.LG cs.CV 62%

Understanding prompt engineering may not require rethinking generalization

Victor Akinwande, Yiding Jiang, Dylan Sam, J. Zico Kolter

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

2307.10504 2023-09-11 cs.CV cs.LG 62%

Identifying Interpretable Subspaces in Image Representations

Neha Kalibhat, Shweta Bhardwaj, Bayan Bruss, Hamed Firooz, Maziar Sanjabi, Soheil Feizi

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Published at ICML 2023 Code: https://github.com/NehaKalibhat/falcon-explain

2306.07282 2023-08-21 cs.CV cs.LG 62%

Waffling around for Performance: Visual Classification with Random Words and Broad Concepts

Karsten Roth, Jae Myung Kim, A. Sophia Koepke, Oriol Vinyals, Cordelia Schmid, Zeynep Akata

专题命中其他VLM ：vision-language model(abstract)；分类 cs.CV、cs.LG

Comments Accepted to ICCV 2023. Main paper with 9 pages

1. 其他VLM 1578 篇

Revealing Vision-Language Integration in the Brain with Multimodal Networks

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models

From Pixels to Prose: A Large Dataset of Dense Image Captions

Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding

Make-it-Real: Unleashing Large Multimodal Model for Painting 3D Objects with Realistic Materials

FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs

A Multi-Modal Foundation Model to Assist People with Blindness and Low Vision in Environmental Interaction

BattleAgent: Multi-modal Dynamic Emulation on Historical Battles to Complement Historical Analysis

Towards Universal Fake Image Detectors that Generalize Across Generative Models

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP

Yell At Your Robot: Improving On-the-Fly from Language Corrections

Follow-Up Differential Descriptions: Language Models Resolve Ambiguities for Image Classification

XCoOp: Explainable Prompt Learning for Computer-Aided Diagnosis via Concept-guided Context Optimization

Leveraging Diffusion Perturbations for Measuring Fairness in Computer Vision

Image-Caption Encoding for Improving Zero-Shot Generalization

Object-oriented backdoor attack against image captioning

VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use

Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks

Visual AI and Linguistic Intelligence Through Steerability and Composability

LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers

Domain Prompt Learning with Quaternion Networks

ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

CoPL: Contextual Prompt Learning for Vision-Language Understanding

ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation

Improving Multimodal Datasets with Image Captioning

S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions

VPGTrans: Transfer Visual Prompt Generator across LLMs

Understanding prompt engineering may not require rethinking generalization

Identifying Interpretable Subspaces in Image Representations

Waffling around for Performance: Visual Classification with Random Words and Broad Concepts