多模态大模型

Pre-training image-language transformers for open-vocabulary tasks

AJ Piergiovanni, Weicheng Kuo, Anelia Angelova

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Zero-shot Object Detection Through Vision-Language Embedding Alignment

Johnathan Xie, Shuai Zheng

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Code: https://github.com/Johnathan-Xie/ZSD-YOLO

PIC 4th Challenge: Semantic-Assisted Multi-Feature Encoding and Multi-Head Decoding for Dense Video Captioning

Yifan Lu, Ziqi Zhang, Yuxin Chen, Chunfeng Yuan, Bing Li, Weiming Hu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 5 pages, 2 figures, report of PIC 4th Challenge Accepted by PIC'22 workshop

GRIT-VLP: Grouped Mini-batch Sampling for Efficient Vision and Language Pre-training

Jaeseok Byun, Taebaek Hwang, Jianlong Fu, Taesup Moon

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Frozen CLIP Models are Efficient Video Learners

Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard de Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, Hongsheng Li

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments ECCV 2022

IDEA: Increasing Text Diversity via Online Multi-Label Recognition for Vision-Language Pre-training

Xinyu Huang, Youcai Zhang, Ying Cheng, Weiwei Tian, Ruiwei Zhao, Rui Feng, Yuejie Zhang, Yaqian Li, Yandong Guo, Xiaobo Zhang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by the 30th ACM International Conference on Multimedia (ACM MM 2022)

Zero-Shot Video Captioning with Evolving Pseudo-Tokens

Yoad Tewel, Yoav Shalev, Roy Nadler, Idan Schwartz, Lior Wolf

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments preprint

Don't Stop Learning: Towards Continual Learning for the CLIP Model

Yuxuan Ding, Lingqiao Liu, Chunna Tian, Jingyuan Yang, Haoxuan Ding

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 12 pages, 5 figures

Simple Open-Vocabulary Object Detection with Vision Transformers

Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, Xiao Wang, Xiaohua Zhai, Thomas Kipf, Neil Houlsby

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments ECCV 2022 camera-ready version

Progress and limitations of deep networks to recognize objects in unusual poses

Amro Abbas, Stéphane Deny

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddings

Gabriel Skantze, Bram Willemsen

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

Comments Published in Journal of Artificial Intelligence Research (JAIR)

Journal ref Journal of Artificial Intelligence Research (JAIR), 2022, vol. 74, pp. 1201-1223

Divert More Attention to Vision-Language Tracking

Mingzhe Guo, Zhipeng Zhang, Heng Fan, Liping Jing

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments 18 pages, 7 figures

DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations

Ximeng Sun, Ping Hu, Kate Saenko

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformer

Jiajun Deng, Zhengyuan Yang, Daqing Liu, Tianlang Chen, Wengang Zhou, Yanyong Zhang, Houqiang Li, Wanli Ouyang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments arXiv admin note: text overlap with arXiv:2104.08541

INDIGO: Intrinsic Multimodality for Domain Generalization

Puneet Mangla, Shivam Chandhok, Milan Aggarwal, Vineeth N Balasubramanian, Balaji Krishnamurthy

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Under Submission

Dual-Level Decoupled Transformer for Video Captioning

Yiqi Gao, Xinglin Hou, Wei Suo, Mengyang Sun, Tiezheng Ge, Yuning Jiang, Peng Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Generating Radiology Reports via Memory-driven Transformer

Zhihong Chen, Yan Song, Tsung-Hui Chang, Xiang Wan

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CL

Comments Natural Language Processing. 11 pages, 6 figures. EMNLP-2020

CapOnImage: Context-driven Dense-Captioning on Image

Yiqi Gao, Xinglin Hou, Yuanmeng Zhang, Tiezheng Ge, Yuning Jiang, Peng Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 13pages, 10figures

Decoupling Zero-Shot Semantic Segmentation

Jian Ding, Nan Xue, Gui-Song Xia, Dengxin Dai

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by CVPR 2022

End-to-End Transformer Based Model for Image Captioning

Yiyu Wang, Jungang Xu, Yingfei Sun

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments AAAI 2022

CRIS: CLIP-Driven Referring Image Segmentation

Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, Tongliang Liu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 5 figures, Accepted by CVPR2022

MVP: Multimodality-guided Visual Pre-training

Longhui Wei, Lingxi Xie, Wengang Zhou, Houqiang Li, Qi Tian

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 14 pages

FlexIT: Towards Flexible Semantic Image Translation

Guillaume Couairon, Asya Grechka, Jakob Verbeek, Holger Schwenk, Matthieu Cord

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments accepted at CVPR 2022

CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields

Can Wang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments To Appear at CVPR 2022

StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation

Peter Schaldenbrand, Zhixuan Liu, Jean Oh

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

Junnan Li, Dongxu Li, Caiming Xiong, Steven Hoi

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

UNISON: Unpaired Cross-lingual Image Captioning

Jiahui Gao, Yi Zhou, Philip L. H. Yu, Shafiq Joty, Jiuxiang Gu

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CL

Comments Accepted by AAAI-2022

On the Efficacy of Co-Attention Transformer Layers in Visual Question Answering

Ankur Sikarwar, Gabriel Kreiman

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

When CNNs Meet Random RNNs: Towards Multi-Level Analysis for RGB-D Object and Scene Recognition

Ali Caglayan, Nevrez Imamoglu, Ahmet Burak Can, Ryosuke Nakamura

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 15 pages, 9 figures, 5 tables (To appear in Computer Vision and Image Understanding, Elsevier)

Gaudí: Conversational Interactions with Deep Representations to Generate Image Collections

Victor S. Bursztyn, Jennifer Healey, Vishwa Vinay

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.AI

Comments Accepted at the NeurIPS 2021 Workshop on Machine Learning for Creativity and Design