多模态大模型

Dual-Level Decoupled Transformer for Video Captioning

Yiqi Gao, Xinglin Hou, Wei Suo, Mengyang Sun, Tiezheng Ge, Yuning Jiang, Peng Wang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Generating Radiology Reports via Memory-driven Transformer

Zhihong Chen, Yan Song, Tsung-Hui Chang, Xiang Wan

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CL

Comments Natural Language Processing. 11 pages, 6 figures. EMNLP-2020

CapOnImage: Context-driven Dense-Captioning on Image

Yiqi Gao, Xinglin Hou, Yuanmeng Zhang, Tiezheng Ge, Yuning Jiang, Peng Wang

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 13pages, 10figures

Decoupling Zero-Shot Semantic Segmentation

Jian Ding, Nan Xue, Gui-Song Xia, Dengxin Dai

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted by CVPR 2022

End-to-End Transformer Based Model for Image Captioning

Yiyu Wang, Jungang Xu, Yingfei Sun

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments AAAI 2022

CRIS: CLIP-Driven Referring Image Segmentation

Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, Tongliang Liu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 5 figures, Accepted by CVPR2022

MVP: Multimodality-guided Visual Pre-training

Longhui Wei, Lingxi Xie, Wengang Zhou, Houqiang Li, Qi Tian

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 14 pages

FlexIT: Towards Flexible Semantic Image Translation

Guillaume Couairon, Asya Grechka, Jakob Verbeek, Holger Schwenk, Matthieu Cord

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments accepted at CVPR 2022

CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields

Can Wang, Menglei Chai, Mingming He, Dongdong Chen, Jing Liao

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments To Appear at CVPR 2022

StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Translation

Peter Schaldenbrand, Zhixuan Liu, Jean Oh

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation

Junnan Li, Dongxu Li, Caiming Xiong, Steven Hoi

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

UNISON: Unpaired Cross-lingual Image Captioning

Jiahui Gao, Yi Zhou, Philip L. H. Yu, Shafiq Joty, Jiuxiang Gu

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CL

Comments Accepted by AAAI-2022

On the Efficacy of Co-Attention Transformer Layers in Visual Question Answering

Ankur Sikarwar, Gabriel Kreiman

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

When CNNs Meet Random RNNs: Towards Multi-Level Analysis for RGB-D Object and Scene Recognition

Ali Caglayan, Nevrez Imamoglu, Ahmet Burak Can, Ryosuke Nakamura

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 15 pages, 9 figures, 5 tables (To appear in Computer Vision and Image Understanding, Elsevier)

Gaudí: Conversational Interactions with Deep Representations to Generate Image Collections

Victor S. Bursztyn, Jennifer Healey, Vishwa Vinay

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.AI

Comments Accepted at the NeurIPS 2021 Workshop on Machine Learning for Creativity and Design

An implementation of the "Guess who?" game using CLIP

Arnau Martí Sarri, Victor Rodriguez-Fernandez

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Code available at https://github.com/ArnauDIMAI/CLIP-GuessWho

Journal ref Intelligent Data Engineering and Automated Learning (IDEAL 2021). Lecture Notes in Computer Science, vol 13113

Mutltimodal AI Companion for Interactive Fairytale Co-creation

Ruiyang Liu, Predrag K. Nikolic

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.MM

A Simple Long-Tailed Recognition Baseline via Vision-Language Model

Teli Ma, Shijie Geng, Mengmeng Wang, Jing Shao, Jiasen Lu, Hongsheng Li, Peng Gao, Yu Qiao

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

TRIE: End-to-End Text Reading and Information Extraction for Document Understanding

Peng Zhang, Yunlu Xu, Zhanzhan Cheng, Shiliang Pu, Jing Lu, Liang Qiao, Yi Niu, Fei Wu

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted to ACM MM2020. Code is available at https://davar-lab.github.io/publication.html or https://github.com/hikopensource/DAVAR-Lab-OCR

Visual News: Benchmark and Challenges in News Image Captioning

Fuxiao Liu, Yinghan Wang, Tianlu Wang, Vicente Ordonez

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 9 pages, 5 figures, accepted to EMNLP2021

Sensor-Augmented Egocentric-Video Captioning with Dynamic Modal Attention

Katsuyuki Nakamura, Hiroki Ohashi, Mitsuhiro Okada

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted to ACM Multimedia (ACMMM) 2021

MiniVLM: A Smaller and Faster Vision-Language Model

Jianfeng Wang, Xiaowei Hu, Pengchuan Zhang, Xiujun Li, Lijuan Wang, Lei Zhang, Jianfeng Gao, Zicheng Liu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

LIIR at SemEval-2021 task 6: Detection of Persuasion Techniques In Texts and Images using CLIP features

Erfan Ghadery, Damien Sileo, Marie-Francine Moens

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

Empirical Analysis of Image Caption Generation using Deep Learning

Aditya Bhattacharya, Eshwar Shamanna Girishekar, Padmakar Anil Deshpande

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Withdrawing for further updates to the work

Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation

Ruizhe Cheng, Bichen Wu, Peizhao Zhang, Peter Vajda, Joseph E. Gonzalez

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 4 pages, 1 figure

Kaleido-BERT: Vision-Language Pre-training on Fashion Domain

Mingchen Zhuge, Dehong Gao, Deng-Ping Fan, Linbo Jin, Ben Chen, Haoming Zhou, Minghui Qiu, Ling Shao

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments CVPR2021 Accepted. Code: https://github.com/mczhuge/Kaleido-BERT

Improved Bengali Image Captioning via deep convolutional neural network based encoder-decoder model

Mohammad Faiyaz Khan, S. M. Sadiq-Ur-Rahman Shifath, Md. Saiful Islam

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted in "IJCACI 2020: International Joint Conference on Advances in Computational Intelligence"

Iconographic Image Captioning for Artworks

Eva Cetinic

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted at Workshop on Fine Art Pattern Extraction and Recognition (FAPER), ICPR, 2020

Macroscopic Control of Text Generation for Image Captioning

Zhangzi Zhu, Tianlei Wang, Hong Qu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

TIME: Text and Image Mutual-Translation Adversarial Networks

Bingchen Liu, Kunpeng Song, Yizhe Zhu, Gerard de Melo, Ahmed Elgammal

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Comments AAAI-2021