视觉大模型 / VLM

MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control

Enshen Zhou, Yiran Qin, Zhenfei Yin, Yuzhou Huang, Ruimao Zhang, Lu Sheng, Yu Qiao, Jing Shao

专题命中 其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments Project page: https://sites.google.com/view/minedreamer/main

EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognition

Niki Maria Foteinopoulou, Ioannis Patras

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at FG'2024

CoLeCLIP: Open-Domain Continual Learning via Joint Task Prompt and Vocabulary Learning

Yukun Li, Guansong Pang, Wei Suo, Chenchen Jing, Yuling Xi, Lingqiao Liu, Hao Chen, Guoqiang Liang, Peng Wang

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Split to Merge: Unifying Separated Modalities for Unsupervised Domain Adaptation

Xinyao Li, Yuke Li, Zhekai Du, Fengling Li, Ke Lu, Jingjing Li

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments CVPR 2024 camera ready

FontCLIP: A Semantic Typography Visual-Language Model for Multilingual Font Applications

Yuki Tatsukawa, I-Chao Shen, Anran Qi, Yuki Koyama, Takeo Igarashi, Ariel Shamir

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments 11 pages. Eurographics 2024. https://yukistavailable.github.io/fontclip.github.io/

A streamlined Approach to Multimodal Few-Shot Class Incremental Learning for Fine-Grained Datasets

Thang Doan, Sima Behpour, Xin Li, Wenbin He, Liang Gou, Liu Ren

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

CLIP-Gaze: Towards General Gaze Estimation via Visual-Linguistic Model

Pengwei Yin, Guanzhong Zeng, Jingjing Wang, Di Xie

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted to AAAI 2024

MeaCap: Memory-Augmented Zero-shot Image Captioning

Zequn Zeng, Yan Xie, Hao Zhang, Chiyu Chen, Zhengjue Wang, Bo Chen

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by CVPR2024

Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation

Zhekai Du, Xinyao Li, Fengling Li, Ke Lu, Lei Zhu, Jingjing Li

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.AI

DomainVerse: A Benchmark Towards Real-World Distribution Shifts For Tuning-Free Adaptive Domain Generalization

Feng Hou, Jin Yuan, Ying Yang, Yang Liu, Yang Zhang, Cheng Zhong, Zhongchao Shi, Jianping Fan, Yong Rui, Zhiqiang He

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Currently in review for ICML 2024

Vision-Language Interpreter for Robot Task Planning

Keisuke Shirai, Cristian C. Beltran-Hernandez, Masashi Hamaya, Atsushi Hashimoto, Shohei Tanaka, Kento Kawaharazuka, Kazutoshi Tanaka, Yoshitaka Ushiku, Shinsuke Mori

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.AI

Comments ICRA 2024

PathAsst: A Generative Foundation AI Assistant Towards Artificial General Intelligence of Pathology

Yuxuan Sun, Chenglu Zhu, Sunyi Zheng, Kai Zhang, Lin Sun, Zhongyi Shui, Yunlong Zhang, Honglin Li, Lin Yang

专题命中 其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments 9 pages, 7 figures

Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probing

Yong Cao, Wenyan Li, Jiaang Li, Yifei Yuan, Antonia Karamolegkou, Daniel Hershcovich

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments work in process

Object-Centric Instruction Augmentation for Robotic Manipulation

Junjie Wen, Yichen Zhu, Minjie Zhu, Jinming Li, Zhiyuan Xu, Zhengping Che, Chaomin Shen, Yaxin Peng, Dong Liu, Feifei Feng, Jian Tang

专题命中 其他VLM ：MLLM(abstract)；分类 cs.CV

Comments accepted to ICRA2024

PVLR: Prompt-driven Visual-Linguistic Representation Learning for Multi-Label Image Recognition

Hao Tan, Zichang Tan, Jun Li, Jun Wan, Zhen Lei

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments 15 pages, 8 figures

GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewear

Robert Konrad, Nitish Padmanaban, J. Gabriel Buckmaster, Kevin C. Boyle, Gordon Wetzstein

专题命中 其他VLM ：multimodal large language model(abstract)；分类 cs.CV

Comments Project video: https://youtu.be/AuDFHHTK_m8

InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model

Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Code and models are available at https://github.com/InternLM/InternLM-XComposer

The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification

Linhao Qu, Xiaoyuan Luo, Kexue Fu, Manning Wang, Zhijian Song

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by NeurIPS 2023

CBVS: A Large-Scale Chinese Image-Text Benchmark for Real-World Short Video Search Scenarios

Xiangshuo Qiao, Xianxin Li, Xiaozhe Qu, Jie Zhang, Yang Liu, Yu Luo, Cihang Jin, Jin Ma

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

An Axiomatic Approach to Model-Agnostic Concept Explanations

Zhili Feng, Michal Moshkovitz, Dotan Di Castro, J. Zico Kolter

专题命中 其他VLM ：vision language model(abstract)；分类 cs.LG

Vocabulary-free Image Classification

Alessandro Conti, Enrico Fini, Massimiliano Mancini, Paolo Rota, Yiming Wang, Elisa Ricci

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted at NeurIPS2023, 19 pages, 8 figures, code is available at https://github.com/altndrr/vic

DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Driving

Wencheng Han, Dongqian Guo, Cheng-Zhong Xu, Jianbing Shen

专题命中 其他VLM ：vision language model(abstract)；分类 cs.CV

Exploring Boundary of GPT-4V on Marine Analysis: A Preliminary Case Study

Ziqiang Zheng, Yiwei Chen, Jipeng Zhang, Tuan-Anh Vu, Huimin Zeng, Yue Him Wong Tim, Sai-Kit Yeung

专题命中 其他VLM ：MLLM(abstract)；分类 cs.CV

Comments 51 pages, 36 figures, Repository: https://github.com/hkust-vgd/Marine_GPT-4V_Eval

Vision-language Assisted Attribute Learning

Kongming Liang, Xinran Wang, Rui Wang, Donghui Gao, Ling Jin, Weidong Liu, Xiatian Zhu, Zhanyu Ma, Jun Guo

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Accepted by IEEE IC-NIDC 2023

Domain-Controlled Prompt Learning

Qinglong Cao, Zhengqin Xu, Yuntian Chen, Chao Ma, Xiaokang Yang

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

ConVRT: Consistent Video Restoration Through Turbulence with Test-time Optimization of Neural Video Representations

Haoming Cai, Jingxi Chen, Brandon Y. Feng, Weiyun Jiang, Mingyang Xie, Kevin Zhang, Ashok Veeraraghavan, Christopher Metzler

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments https://convrt-2024.github.io/

ECLIPSE: A Resource-Efficient Text-to-Image Prior for Image Generations

Maitreya Patel, Changhoon Kim, Sheng Cheng, Chitta Baral, Yezhou Yang

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Project Page: https://eclipse-t2i.vercel.app/

GenZI: Zero-Shot 3D Human-Scene Interaction Generation

Lei Li, Angela Dai

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments Project page: https://craigleili.github.io/projects/genzi/ Video: https://youtu.be/ozfs6E0JIMY

Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement

Yanyan Wei, Zhao Zhang, Jiahuan Ren, Xiaogang Xu, Richang Hong, Yi Yang, Shuicheng Yan, Meng Wang

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

GIPCOL: Graph-Injected Soft Prompting for Compositional Zero-Shot Learning

Guangyue Xu, Joyce Chai, Parisa Kordjamshidi

专题命中 其他VLM ：vision-language model(abstract)；分类 cs.CV

Comments WACV24