多模态大模型

Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network

Jiayi Ji, Yunpeng Luo, Xiaoshuai Sun, Fuhai Chen, Gen Luo, Yongjian Wu, Yue Gao, Rongrong Ji

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Accepted at AAAI 2021 (preprint version)

TAP: Text-Aware Pre-training for Text-VQA and Text-Caption

Zhengyuan Yang, Yijuan Lu, Jianfeng Wang, Xi Yin, Dinei Florencio, Lijuan Wang, Cha Zhang, Lei Zhang, Jiebo Luo

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

In-Home Daily-Life Captioning Using Radio Signals

Lijie Fan, Tianhong Li, Yuan Yuan, Dina Katabi

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments ECCV 2020. The first two authors contributed equally to this paper

Dense-Captioning Events in Videos: SYSU Submission to ActivityNet Challenge 2020

Teng Wang, Huicheng Zheng, Mingjing Yu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Second-place solution to TASK 2 (Dense video captioning) in ActivityNet Challenge 2020. Code is available at https://github.com/ttengwang/dense-video-captioning-pytorch

Connecting Vision and Language with Localized Narratives

Jordi Pont-Tuset, Jasper Uijlings, Soravit Changpinyo, Radu Soricut, Vittorio Ferrari

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments ECCV 2020 Camera Ready

Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks

Fengda Zhu, Yi Zhu, Xiaojun Chang, Xiaodan Liang

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

X-Linear Attention Networks for Image Captioning

Yingwei Pan, Ting Yao, Yehao Li, Tao Mei

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments CVPR 2020; The source code and model are publicly available at: https://github.com/Panda-Peter/image-captioning

Image to Language Understanding: Captioning approach

Madhavan Seshadri, Malavika Srikanth, Mikhail Belov

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 8 pages

Going Beneath the Surface: Evaluating Image Captioning for Grammaticality, Truthfulness and Diversity

Huiyuan Xie, Tom Sherborne, Alexander Kuhnle, Ann Copestake

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog

Feilong Chen, Fandong Meng, Jiaming Xu, Peng Li, Bo Xu, Jie Zhou

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

Comments Accepted at AAAI 2020

Visual-Textual Association with Hardest and Semi-Hard Negative Pairs Mining for Person Search

Jing Ge, Guangyu Gao, Zhen Liu

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

A Novel Technique for Evidence based Conditional Inference in Deep Neural Networks via Latent Feature Perturbation

Dinesh Khandelwal, Suyash Agrawal, Parag Singla, Chetan Arora

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Unified Vision-Language Pre-Training for Image Captioning and VQA

Luowei Zhou, Hamid Palangi, Lei Zhang, Houdong Hu, Jason J. Corso, Jianfeng Gao

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments AAAI 2020 camera-ready version. The code and the pre-trained models are available at https://github.com/LuoweiZhou/VLP

Exploring Overall Contextual Information for Image Captioning in Human-Like Cognitive Style

Hongwei Ge, Zehang Yan, Kai Zhang, Mingde Zhao, Liang Sun

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Comments ICCV 2019

Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions

Yulei Niu, Hanwang Zhang, Zhiwu Lu, Shih-Fu Chang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted as regular paper in IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Substantial text overlap with arXiv:1712.01892

Grounding Referring Expressions in Images by Variational Context

Hanwang Zhang, Yulei Niu, Shih-Fu Chang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments in 2018 Conference on Computer Vision and Pattern Recognition (CVPR'18)

Generating Diverse and Informative Natural Language Fashion Feedback

Gil Sadeh, Lior Fritz, Gabi Shalev, Eduard Oks

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Harvesting Information from Captions for Weakly Supervised Semantic Segmentation

Johann Sawatzky, Debayan Banerjee, Juergen Gall

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

3G structure for image caption generation

Aihong Yuan, Xuelong Li, Xiaoqiang Lu

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 35 pages, 7 figures, magazine

Journal ref Neurocomputing 330: 17-28 (2019)

ExCL: Extractive Clip Localization Using Natural Language Descriptions

Soham Ghosh, Anuva Agarwal, Zarana Parekh, Alexander Hauptmann

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CL

Comments Accepted at NAACL 2019, Short Paper

Show, Translate and Tell

Dheeraj Peri, Shagan Sah, Raymond Ptucha

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Visual Entailment: A Novel Task for Fine-Grained Image Understanding

Ning Xie, Farley Lai, Derek Doran, Asim Kadav

专题命中 图文多模态 ：cross-modal(abstract)；分类 cs.CV

Learning Style Compatibility for Furniture

Divyansh Aggarwal, Elchin Valiyev, Fadime Sener, Angela Yao

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments German Conference on Pattern Recognition(GCPR)

PIRC Net : Using Proposal Indexing, Relationships and Context for Phrase Grounding

Rama Kovvuri, Ram Nevatia

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments Accepted in ACCV 2018

Bringing back simplicity and lightliness into neural image captioning

Jean-Benoit Delbrouck, Stéphane Dupont

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CL

Equal But Not The Same: Understanding the Implicit Relationship Between Persuasive Images and Text

Mingda Zhang, Rebecca Hwa, Adriana Kovashka

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments To appear in BMVC2018

Unpaired Image Captioning by Language Pivoting

Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Gang Wang

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV

Comments 17 pages, 4 figures, Accepted at ECCV 2018

Weakly Supervised Attention Learning for Textual Phrases Grounding

Zhiyuan Fang, Shu Kong, Tianshu Yu, Yezhou Yang

专题命中 图文多模态 ：image-text(abstract)；分类 cs.CV

Comments 4 pages, 3 figures

Deep Matching Autoencoders

Tanmoy Mukherjee, Makoto Yamada, Timothy M. Hospedales

专题命中 图文多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 10 pages

Video Captioning with Multi-Faceted Attention

Xiang Long, Chuang Gan, Gerard de Melo

专题命中 图文多模态 ：multimodal(abstract)；分类 cs.CV