多模态大模型 - arXivDaily 专题

2401.09748 2024-01-19 cs.SC cs.AI cs.LG 70%

Bootstrapping OTS-Funcimg Pre-training Model (Botfip) -- A Comprehensive Symbolic Regression Framework

Tianhao Chen, Pengbo Xu, Haibiao Zheng

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.AI

2311.17812 2024-01-01 cs.CV 70%

DAP: Domain-aware Prompt Learning for Vision-and-Language Navigation

Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments 4 pages. arXiv admin note: substantial text overlap with arXiv:2309.03661

2306.07915 2023-12-22 cs.CV 70%

Image Captioners Are Scalable Vision Learners Too

Michael Tschannen, Manoj Kumar, Andreas Steiner, Xiaohua Zhai, Neil Houlsby, Lucas Beyer

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2023. v2 adds SugarCrepe results and more ablations, v3 has minor fixes. v4 adds a code link ( https://github.com/google-research/big_vision ). v5 has minor fixes

2309.15112 2023-12-15 cs.CV 70%

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Haodong Duan, Songyang Zhang, Shuangrui Ding, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, Jiaqi Wang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Code and models are available at https://github.com/InternLM/InternLM-XComposer

2312.04403 2023-12-08 cs.CV 70%

OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization

Dongchen Han, Xiaojun Jia, Yang Bai, Jindong Gu, Yang Liu, Xiaochun Cao

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

2312.04248 2023-12-08 cs.CV 70%

TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes

Xuying Zhang, Bo-Wen Yin, Yuming Chen, Zheng Lin, Yunheng Li, Qibin Hou, Ming-Ming Cheng

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2111.12727 2023-12-01 cs.CV cs.AI cs.CL cs.MM 70%

Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets

Marcella Cornia, Lorenzo Baraldi, Giuseppe Fiameni, Rita Cucchiara

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to IJCV

2308.02487 2023-11-16 cs.CV 70%

Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP

Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, Liang-Chieh Chen

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments NeurIPS 2023 camera ready. code and model available at https://github.com/bytedance/fc-clip

2309.17425 2023-11-07 cs.AI cs.LG 70%

Data Filtering Networks

Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, Vaishaal Shankar

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.AI

2305.08675 2023-11-07 cs.CV 70%

Improved baselines for vision-language pre-training

Enrico Fini, Pietro Astolfi, Adriana Romero-Soriano, Jakob Verbeek, Michal Drozdzal

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments TMLR, featured certification; changelog at https://openreview.net/forum?id=a7nvXxNmdV

Journal ref Transactions on Machine Learning Research, 10/2023, issn 2835-8856

2310.08166 2023-11-01 cs.CL 70%

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Junyu Lu, Dixiang Zhang, Xiaojun Wu, Xinyu Gao, Ruyi Gan, Jiaxing Zhang, Yan Song, Pingjian Zhang

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CL

2310.18049 2023-10-30 cs.CV 70%

Text Augmented Spatial-aware Zero-shot Referring Image Segmentation

Yucheng Suo, Linchao Zhu, Yi Yang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Findings of EMNLP2023

2305.15023 2023-10-25 cs.CV 70%

Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models

Gen Luo, Yiyi Zhou, Tianhe Ren, Shengxin Chen, Xiaoshuai Sun, Rongrong Ji

专题命中图文多模态：multimodal(abstract)；multi-modal(abstract)；分类 cs.CV

2309.14491 2023-09-27 cs.CV 70%

Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving

Mahyar Najibi, Jingwei Ji, Yin Zhou, Charles R. Qi, Xinchen Yan, Scott Ettinger, Dragomir Anguelov

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ICCV 2023

2308.16354 2023-09-01 cs.CV 70%

Catalog Phrase Grounding (CPG): Grounding of Product Textual Attributes in Product Images for e-commerce Vision-Language Applications

Wenyi Wu, Karim Bouyarmane, Ismail Tutar

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments KDD 2022 Workshop on First Content Understanding and Generation for e-Commerce

2308.13853 2023-08-29 cs.CV 70%

Beyond One-to-One: Rethinking the Referring Image Segmentation

Yutao Hu, Qixiong Wang, Wenqi Shao, Enze Xie, Zhenguo Li, Jungong Han, Ping Luo

专题命中图文多模态：multi-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments ICCV 2023

2308.08754 2023-08-21 cs.CV 70%

Fine-grained Text and Image Guided Point Cloud Completion with CLIP Model

Wei Song, Jun Zhou, Mingjie Wang, Hongchen Tan, Nannan Li, Xiuping Liu

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2307.12980 2023-07-25 cs.CV 70%

A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models

Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2303.12112 2023-07-21 cs.CV cs.AI cs.CL cs.MM 70%

Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation

Sara Sarto, Manuele Barraco, Marcella Cornia, Lorenzo Baraldi, Rita Cucchiara

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR 2023 (highlight paper)

2306.09085 2023-06-16 cs.CV cs.AI cs.CL cs.LG cs.MM 70%

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

Sihan Chen, Xingjian He, Handong Li, Xiaojie Jin, Jiashi Feng, Jing Liu

专题命中图文多模态：image-text(abstract)；分类 cs.CV、cs.CL、cs.AI

2306.04715 2023-06-09 cs.CV 70%

UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks

Yanan Sun, Zihan Zhong, Qi Fan, Chi-Keung Tang, Yu-Wing Tai

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

2303.14302 2023-06-06 cs.CV 70%

VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining

Junjie Ke, Keren Ye, Jiahui Yu, Yonghui Wu, Peyman Milanfar, Feng Yang

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2023, https://github.com/google-research/google-research/tree/master/vila

2305.19595 2023-06-02 cs.CV 70%

Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models

Sivan Doveh, Assaf Arbelle, Sivan Harary, Roei Herzig, Donghyun Kim, Paola Cascante-bonilla, Amit Alfassy, Rameswar Panda, Raja Giryes, Rogerio Feris, Shimon Ullman, Leonid Karlinsky

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

2305.15483 2023-05-26 cs.CV 70%

Weakly Supervised Vision-and-Language Pre-training with Relative Representations

Chi Chen, Peng Li, Maosong Sun, Yang Liu

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted by ACL 2023

2305.12223 2023-05-24 cs.CV 70%

What Makes for Good Visual Tokenizers for Large Language Models?

Guangzhi Wang, Yixiao Ge, Xiaohan Ding, Mohan Kankanhalli, Ying Shan

专题命中图文多模态：multimodal(abstract)；MLLM(abstract)；分类 cs.CV

Comments 15 pages, 3 figures. Project released at: https://github.com/TencentARC/GVT

2304.13583 2023-04-27 eess.IV cs.CV 70%

Multi-Modality Deep Network for Extreme Learned Image Compression

Xuhao Jiang, Weimin Tan, Tian Tan, Bo Yan, Liquan Shen

专题命中图文多模态：multimodal(abstract)；image-text(abstract)；分类 cs.CV

Comments 13 pages, 14 figures, accepted by AAAI 2023

2301.04926 2023-04-07 cs.CV 70%

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, Wenping Wang

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments CVPR 2023

2302.12248 2023-03-30 cs.CV 70%

Learning Visual Representations via Language-Guided Sampling

Mohamed El Banani, Karan Desai, Justin Johnson

专题命中图文多模态：cross-modal(abstract)；image-text(abstract)；分类 cs.CV

Comments Accepted to CVPR 2023. v2 is camera-ready version with additional ImageNet evaluations. Project page: https://github.com/mbanani/lgssl

2303.02483 2023-03-07 cs.CV 70%

FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks

Xiao Han, Xiatian Zhu, Licheng Yu, Li Zhang, Yi-Zhe Song, Tao Xiang

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments CVPR 2023

2301.09045 2023-02-21 cs.CV 70%

Champion Solution for the WSDM2023 Toloka VQA Challenge

Shengyi Gao, Zhe Chen, Guo Chen, Wenhai Wang, Tong Lu

专题命中图文多模态：multi-modal(abstract)；cross-modal(abstract)；分类 cs.CV

Comments Technical report in WSDM Cup 2023

1. 图文多模态 4672 篇

Bootstrapping OTS-Funcimg Pre-training Model (Botfip) -- A Comprehensive Symbolic Regression Framework

DAP: Domain-aware Prompt Learning for Vision-and-Language Navigation

Image Captioners Are Scalable Vision Learners Too

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization

TeMO: Towards Text-Driven 3D Stylization for Multi-Object Meshes

Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets

Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP

Data Filtering Networks

Improved baselines for vision-language pre-training

Ziya-Visual: Bilingual Large Vision-Language Model via Multi-Task Instruction Tuning

Text Augmented Spatial-aware Zero-shot Referring Image Segmentation

Cheap and Quick: Efficient Vision-Language Instruction Tuning for Large Language Models

Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous Driving

Catalog Phrase Grounding (CPG): Grounding of Product Textual Attributes in Product Images for e-commerce Vision-Language Applications

Beyond One-to-One: Rethinking the Referring Image Segmentation

Fine-grained Text and Image Guided Point Cloud Completion with CLIP Model

A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models

Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation

COSA: Concatenated Sample Pretrained Vision-Language Foundation Model

UniBoost: Unsupervised Unimodal Pre-training for Boosting Zero-shot Vision-Language Tasks

VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretraining

Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Models

Weakly Supervised Vision-and-Language Pre-training with Relative Representations

What Makes for Good Visual Tokenizers for Large Language Models?

Multi-Modality Deep Network for Extreme Learned Image Compression

CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

Learning Visual Representations via Language-Guided Sampling

FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks

Champion Solution for the WSDM2023 Toloka VQA Challenge