多模态大模型 - arXivDaily 专题

2301.06958 2023-03-01 cs.CV 57%

RILS: Masked Visual Reconstruction in Language Semantic Space

Shusheng Yang, Yixiao Ge, Kun Yi, Dian Li, Ying Shan, Xiaohu Qie, Xinggang Wang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2212.06710 2023-03-01 cs.LG cs.CV 57%

TIER: Text-Image Entropy Regularization for CLIP-style models

Anil Palepu, Andrew L. Beam

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Submitted to CHIL 2023 conference

2302.13996 2023-02-28 cs.CV 57%

Aligning Bag of Regions for Open-Vocabulary Object Detection

Size Wu, Wenwei Zhang, Sheng Jin, Wentao Liu, Chen Change Loy

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2302.12066 2023-02-24 cs.CV 57%

Teaching CLIP to Count to Ten

Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, Tali Dekel

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2302.03432 2023-02-08 cs.CV 57%

SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation

Yash Patel, Yusheng Xie, Yi Zhu, Srikar Appalaraju, R. Manmatha

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2109.02748 2023-02-06 cs.CV cs.LG 57%

Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP

Sepideh Esmaeilpour, Bing Liu, Eric Robertson, Lei Shu

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2208.05516 2023-02-02 cs.LG cs.CV 57%

Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP

Thao Nguyen, Gabriel Ilharco, Mitchell Wortsman, Sewoong Oh, Ludwig Schmidt

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Oral paper at NeurIPS 2022

2301.10916 2023-01-27 cs.CV 57%

ITstyler: Image-optimized Text-based Style Transfer

Yunpeng Bai, Jiayue Liu, Chao Dong, Chun Yuan

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments 8 pages, 10 figures

2211.13977 2023-01-03 cs.CV 57%

CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels

Siyuan Li, Li Sun, Qingli Li

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by AAAI23

2211.12739 2022-12-21 cs.CV 57%

Texts as Images in Prompt Tuning for Multi-Label Image Recognition

Zixian Guo, Bowen Dong, Zhilong Ji, Jinfeng Bai, Yiwen Guo, Wangmeng Zuo

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2205.14100 2022-12-19 cs.CV 57%

GIT: A Generative Image-to-text Transformer for Vision and Language

Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, Lijuan Wang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2212.00836 2022-12-05 cs.CV 57%

UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding

Dave Zhenyu Chen, Ronghang Hu, Xinlei Chen, Matthias Nießner, Angel X. Chang

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

2212.00638 2022-12-02 cs.CV cs.LG 57%

Finetune like you pretrain: Improved finetuning of zero-shot vision models

Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, Aditi Raghunathan

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments 20 Pages, 7 Tables, 5 Figures

2211.14958 2022-11-29 cs.CV 57%

MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding

Zilong Wang, Jiuxiang Gu, Chris Tensmeyer, Nikolaos Barmpalios, Ani Nenkova, Tong Sun, Jingbo Shang, Vlad I. Morariu

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments EMNLP 2022

2211.14843 2022-11-29 cs.CV 57%

Learning Object-Language Alignments for Open-Vocabulary Object Detection

Chuang Lin, Peize Sun, Yi Jiang, Ping Luo, Lizhen Qu, Gholamreza Haffari, Zehuan Yuan, Jianfei Cai

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Technical Report

2210.15871 2022-11-28 cs.CV 57%

VLT: Vision-Language Transformer and Query Generation for Referring Segmentation

Henghui Ding, Chang Liu, Suchen Wang, Xudong Jiang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments TPAMI

2211.12494 2022-11-23 cs.CV cs.LG 57%

On the Transferability of Visual Features in Generalized Zero-Shot Learning

Paola Cascante-Bonilla, Leonid Karlinsky, James Seale Smith, Yanjun Qi, Vicente Ordonez

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

2211.12148 2022-11-23 cs.CV cs.LG 57%

Aligning Source Visual and Target Language Domains for Unpaired Video Captioning

Fenglin Liu, Xian Wu, Chenyu You, Shen Ge, Yuexian Zou, Xu Sun

专题命中图文多模态：multimodal(abstract)；分类 cs.CV

Comments Published at IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)

2211.06679 2022-11-22 cs.CL 57%

AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities

Zhongzhi Chen, Guang Liu, Bo-Wen Zhang, Fulong Ye, Qinghong Yang, Ledell Wu

专题命中图文多模态：multimodal(abstract)；分类 cs.CL

2211.07275 2022-11-15 cs.CV 57%

Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignment

Junyang Wang, Yi Zhang, Ming Yan, Ji Zhang, Jitao Sang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

2209.03594 2022-11-15 cs.CV 57%

Levenshtein OCR

Cheng Da, Peng Wang, Cong Yao

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted by ECCV2022

2203.03911 2022-11-15 cs.CV 57%

Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting

Chuhui Xue, Wenqing Zhang, Yu Hao, Shijian Lu, Philip Torr, Song Bai

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted by ECCV2022 for oral presentation

2211.05521 2022-11-11 cs.CV cs.CY 57%

Zero-shot Visual Commonsense Immorality Prediction

Yujin Jeong, Seongbeom Park, Suhong Moon, Jinkyu Kim

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments BMVC2022

2211.04785 2022-11-10 cs.CV 57%

Masked Vision-Language Transformers for Scene Text Recognition

Jie Wu, Ying Peng, Shengming Zhang, Weigang Qi, Jian Zhang

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments The paper is accepted by the 33rd British Machine Vision Conference (BMVC 2022)

2110.11316 2022-11-08 cs.LG cs.CV 57%

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

Andreas Fürst, Elisabeth Rumetshofer, Johannes Lehner, Viet Tran, Fei Tang, Hubert Ramsauer, David Kreil, Michael Kopp, Günter Klambauer, Angela Bitto-Nemling, Sepp Hochreiter

专题命中图文多模态：multi-modal(abstract)；分类 cs.CV

Comments Published at NeurIPS 2022; Blog: https://ml-jku.github.io/cloob; GitHub: https://github.com/ml-jku/cloob

2207.08455 2022-11-01 cs.CV 57%

Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding

Quande Liu, Youpeng Wen, Jianhua Han, Chunjing Xu, Hang Xu, Xiaodan Liang

专题命中图文多模态：cross-modal(abstract)；分类 cs.CV

Comments Accepted to ECCV 2022 (revise acknowledgement)

2210.12826 2022-10-25 cs.CV cs.LG 57%

Towards Real-Time Text2Video via CLIP-Guided, Pixel-Level Optimization

Peter Schaldenbrand, Zhixuan Liu, Jean Oh

专题命中图文多模态：image-text(abstract)；分类 cs.CV

2209.09407 2022-10-18 cs.CV 57%

DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection

Lewei Yao, Jianhua Han, Youpeng Wen, Xiaodan Liang, Dan Xu, Wei Zhang, Zhenguo Li, Chunjing Xu, Hang Xu

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted to NeurIPS 2022

2209.09502 2022-10-18 cs.CV 57%

GAMA: Generative Adversarial Multi-Object Scene Attacks

Abhishek Aich, Calvin-Khang Ta, Akash Gupta, Chengyu Song, Srikanth V. Krishnamurthy, M. Salman Asif, Amit K. Roy-Chowdhury

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2022; First two authors contributed equally; Includes Supplementary Material

2209.12343 2022-09-27 cs.CV cs.LG 57%

Paraphrasing Is All You Need for Novel Object Captioning

Cheng-Fu Yang, Yao-Hung Hubert Tsai, Wan-Cyuan Fan, Ruslan Salakhutdinov, Louis-Philippe Morency, Yu-Chiang Frank Wang

专题命中图文多模态：image-text(abstract)；分类 cs.CV

Comments Accepted at NeurIPS 2022

1. 图文多模态 4657 篇

RILS: Masked Visual Reconstruction in Language Semantic Space

TIER: Text-Image Entropy Regularization for CLIP-style models

Aligning Bag of Regions for Open-Vocabulary Object Detection

Teaching CLIP to Count to Ten

SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation

Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP

Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP

ITstyler: Image-optimized Text-based Style Transfer

CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labels

Texts as Images in Prompt Tuning for Multi-Label Image Recognition

GIT: A Generative Image-to-text Transformer for Vision and Language

UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding

Finetune like you pretrain: Improved finetuning of zero-shot vision models

MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding

Learning Object-Language Alignments for Open-Vocabulary Object Detection

VLT: Vision-Language Transformer and Query Generation for Referring Segmentation

On the Transferability of Visual Features in Generalized Zero-Shot Learning

Aligning Source Visual and Target Language Domains for Unpaired Video Captioning

AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities

Zero-shot Image Captioning by Anchor-augmented Vision-Language Space Alignment

Levenshtein OCR

Language Matters: A Weakly Supervised Vision-Language Pre-training Approach for Scene Text Detection and Spotting

Zero-shot Visual Commonsense Immorality Prediction

Masked Vision-Language Transformers for Scene Text Recognition

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding

Towards Real-Time Text2Video via CLIP-Guided, Pixel-Level Optimization

DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection

GAMA: Generative Adversarial Multi-Object Scene Attacks

Paraphrasing Is All You Need for Novel Object Captioning