多模态大模型 - arXivDaily 专题

2211.15398 2023-09-06 cs.CV cs.LG 57%

Leveraging per Image-Token Consistency for Vision-Language Pre-training

Yunhao Gou, Tom Ko, Hansi Yang, James Kwok, Yu Zhang, Mingxuan Wang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by CVPR 2023

2308.16777 2023-09-04 cs.CV 57%

Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

Minheng Ni, Yabo Zhang, Kailai Feng, Xiaoming Li, Yiwen Guo, Wangmeng Zuo

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.16689 2023-09-01 cs.CV 57%

ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation

Weihan Wang, Zhen Yang, Bin Xu, Juanzi Li, Yankui Sun

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 15 pages, 5 figures

2308.16493 2023-09-01 cs.AI cs.RO 57%

Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perception

Riley Tavassoli, Mani Amani, Reza Akhavian

专题命中图文多模态：multi-modal(abstract)；分类 cs.AI

Comments Preprint submitted to Information Fusion

2304.06028 2023-09-01 cs.CV 57%

RECLIP: Resource-efficient CLIP by Training with Small Images

Runze Li, Dahun Kim, Bir Bhanu, Weicheng Kuo

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Published at Transactions on Machine Learning Research

2308.14212 2023-08-29 cs.CV 57%

Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy

Sanoojan Baliah, Fadillah A. Maani, Santosh Sanjeev, Muhammad Haris Khan

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.12757 2023-08-25 cs.CV 57%

PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning

Mengya Han, Heliang Zheng, Chaoyue Wang, Yong Luo, Han Hu, Jing Zhang, Yonggang Wen

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2308.11605 2023-08-23 cs.CV 57%

GOPro: Generate and Optimize Prompts in CLIP using Self-Supervised Learning

Mainak Singha, Ankit Jha, Biplab Banerjee

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at BMVC 2023

2306.00409 2023-08-23 cs.CV 57%

Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting

Shubin Huang, Qiong Wu, Yiyi Zhou, Weijie Chen, Rongsheng Zhang, Xiaoshuai Sun, Rongrong Ji

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.10240 2023-08-22 cs.CV 57%

Generic Attention-model Explainability by Weighted Relevance Accumulation

Yiming Huang, Aozhe Jia, Xiaodan Zhang, Jiawei Zhang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2308.10061 2023-08-22 cs.CV 57%

DPL: Decoupled Prompt Learning for Vision-Language Models

Chen Xu, Yuhan Zhu, Guozhen Zhang, Haocheng Shen, Yixuan Liao, Xiaoxin Chen, Gangshan Wu, Limin Wang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments 11 pages, 5 figures, 8 tables

2308.08428 2023-08-21 cs.CV 57%

ALIP: Adaptive Language-Image Pre-training with Synthetic Caption

Kaicheng Yang, Jiankang Deng, Xiang An, Jiawei Li, Ziyong Feng, Jia Guo, Jing Yang, Tongliang Liu

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 15pages, 10figures, ICCV2023

2305.20087 2023-08-21 cs.CV 57%

Too Large; Data Reduction for Vision-Language Pre-Training

Alex Jinpeng Wang, Kevin Qinghong Lin, David Junhao Zhang, Stan Weixian Lei, Mike Zheng Shou

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments ICCV2023. Code: https://github.com/showlab/datacentric.vlp

2304.06712 2023-08-21 cs.CV 57%

What does CLIP know about a red circle? Visual prompt engineering for VLMs

Aleksandar Shtedritski, Christian Rupprecht, Andrea Vedaldi

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments ICCV 2023 Oral

2308.08881 2023-08-21 cs.CV 57%

Text-Only Training for Visual Storytelling

Yuechen Wang, Wengang Zhou, Zhenbo Lu, Houqiang Li

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments ACM MM 2023

2303.06571 2023-08-21 cs.CV 57%

Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models

Juncheng Li, Minghe Gao, Longhui Wei, Siliang Tang, Wenqiao Zhang, Mengze Li, Wei Ji, Qi Tian, Tat-Seng Chua, Yueting Zhuang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted by ICCV 2023

2308.07648 2023-08-16 cs.CV 57%

Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval

Chaorui Deng, Qi Chen, Pengda Qin, Da Chen, Qi Wu

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments to be appeared in ICCV2023

2303.06628 2023-08-14 cs.CV cs.LG 57%

Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models

Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xiangyu Yue, Yang You

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted by ICCV 2023

2308.00353 2023-08-02 cs.CV 57%

Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understanding

Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, Xiaojuan Qi

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments submit to TPAMI

2307.15983 2023-08-01 cs.MM 57%

Instance-Wise Adaptive Tuning and Caching for Vision-Language Models

Chunjin Yang, Fanman Meng, Shuai Chen, Mingyu Liu, Runtong Zhang

专题命中图文多模态：image-text(abstract)；分类 cs.MM

2306.00988 2023-07-24 eess.IV cs.CV cs.LG 57%

Continual Learning for Abdominal Multi-Organ and Tumor Segmentation

Yixiao Zhang, Xinyi Li, Huimiao Chen, Alan Yuille, Yaoyao Liu, Zongwei Zhou

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments MICCAI-2023

2307.10046 2023-07-20 cs.CV 57%

Divert More Attention to Vision-Language Object Tracking

Mingzhe Guo, Zhipeng Zhang, Liping Jing, Haibin Ling, Heng Fan

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments 16 pages, 9 figures

2307.03339 2023-07-10 cs.CV 57%

Open-Vocabulary Object Detection via Scene Graph Discovery

Hengcan Shi, Munawar Hayat, Jianfei Cai

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

2306.17659 2023-07-03 cs.CV 57%

Zero-shot Nuclei Detection via Visual-Language Pre-trained Models

Yongjian Wu, Yang Zhou, Jiya Saiyin, Bingzheng Wei, Maode Lai, Jianzhong Shou, Yubo Fan, Yan Xu

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments This article has been accepted by MICCAI 2023,but has not been fully edited. Content may change prior to final publication

2306.16774 2023-06-30 cs.CL 57%

Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages

Yasmine Karoui, Rémi Lebret, Negar Foroutan, Karl Aberer

专题命中图文多模态：image-text(abstract)；分类 cs.CL

Comments Accepted to ACL 2023 as short paper

2306.14406 2023-06-30 cs.CV 57%

TCEIP: Text Condition Embedded Regression Network for Dental Implant Position Prediction

Xinquan Yang, Jinheng Xie, Xuguang Li, Xuechen Li, Xin Li, Linlin Shen, Yongqiang Deng

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments MICCAI 2023

2306.16658 2023-06-30 cs.CV 57%

Prompt Ensemble Self-training for Open-Vocabulary Domain Adaptation

Jiaxing Huang, Jingyi Zhang, Han Qiu, Sheng Jin, Shijian Lu

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2306.15658 2023-06-28 cs.CV 57%

CLIPA-v2: Scaling CLIP Training with 81.1% Zero-shot ImageNet Accuracy within a \$10,000 Budget; An Extra \$4,000 Unlocks 81.8% Accuracy

Xianhang Li, Zeyu Wang, Cihang Xie

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Tech Report. Code is available at https://github.com/UCSC-VLAA/CLIPA

2306.07279 2023-06-19 cs.CV 57%

Scalable 3D Captioning with Pretrained Models

Tiange Luo, Chris Rockwell, Honglak Lee, Justin Johnson

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Dataset link: https://huggingface.co/datasets/tiange/Cap3D

2302.14042 2023-06-16 cs.CV 57%

Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images

Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, Weidi Xie

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

1. 图文多模态 4657 篇

Leveraging per Image-Token Consistency for Vision-Language Pre-training

Ref-Diff: Zero-shot Referring Image Segmentation with Generative Models

ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation

Expanding Frozen Vision-Language Models without Retraining: Towards Improved Robot Perception

RECLIP: Resource-efficient CLIP by Training with Small Images

Exploring the Transfer Learning Capabilities of CLIP in Domain Generalization for Diabetic Retinopathy

PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning

GOPro: Generate and Optimize Prompts in CLIP using Self-Supervised Learning

Adapting Pre-trained Language Models to Vision-Language Tasks via Dynamic Visual Prompting

Generic Attention-model Explainability by Weighted Relevance Accumulation

DPL: Decoupled Prompt Learning for Vision-Language Models

ALIP: Adaptive Language-Image Pre-training with Synthetic Caption

Too Large; Data Reduction for Vision-Language Pre-Training

What does CLIP know about a red circle? Visual prompt engineering for VLMs

Text-Only Training for Visual Storytelling

Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models

Prompt Switch: Efficient CLIP Adaptation for Text-Video Retrieval

Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models

Lowis3D: Language-Driven Open-World Instance-Level 3D Scene Understanding

Instance-Wise Adaptive Tuning and Caching for Vision-Language Models

Continual Learning for Abdominal Multi-Organ and Tumor Segmentation

Divert More Attention to Vision-Language Object Tracking

Open-Vocabulary Object Detection via Scene Graph Discovery

Zero-shot Nuclei Detection via Visual-Language Pre-trained Models

Stop Pre-Training: Adapt Visual-Language Models to Unseen Languages

TCEIP: Text Condition Embedded Regression Network for Dental Implant Position Prediction

Prompt Ensemble Self-training for Open-Vocabulary Domain Adaptation

CLIPA-v2: Scaling CLIP Training with 81.1% Zero-shot ImageNet Accuracy within a \$10,000 Budget; An Extra \$4,000 Unlocks 81.8% Accuracy

Scalable 3D Captioning with Pretrained Models

Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images