多模态大模型 - arXivDaily 专题

2212.09297 2022-12-20 cs.CV 79%

Transferring General Multimodal Pretrained Models to Text Recognition

Junyang Lin, Xuancheng Ren, Yichang Zhang, Gao Liu, Peng Wang, An Yang, Chang Zhou

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

2209.13763 2022-12-01 cs.AI 79%

Clustering-Induced Generative Incomplete Image-Text Clustering (CIGIT-C)

Dongjin Guo, Xiaoming Su, Jiatai Wang, Limin Liu, Zhiyong Pei, Zhiwei Xu

专题命中图文多模态：image-text(title,abstract)；分类 cs.AI

Comments 13 pages,12 figures

2209.11223 2022-09-23 cs.CV cs.GR 79%

UniColor: A Unified Framework for Multi-Modal Colorization with Transformer

Zhitong Huang, Nanxuan Zhao, Jing Liao

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by SIGGRAPH Asia 2022. Project page: https://luckyhzt.github.io/unicolor

2105.11333 2022-09-22 cs.CV 79%

Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training

Jong Hak Moon, Hyungyung Lee, Woncheol Shin, Young-Hak Kim, Edward Choi

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE Journal of Biomedical and Health Informatics

Journal ref IEEE Journal of Biomedical and Health Informatics 2022

2108.06458 2022-09-07 cs.CV 79%

Cross-Modal Graph with Meta Concepts for Video Captioning

Hao Wang, Guosheng Lin, Steven C. H. Hoi, Chunyan Miao

专题命中图文多模态：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at IEEE Transactions on Image Processing

2208.04361 2022-08-10 cs.CV 79%

Semi-Supervised Cross-Modal Salient Object Detection with U-Structure Networks

Yunqing Bao, Hang Dai, Abdulmotaleb Elsaddik

专题命中图文多模态：cross-modal(title,abstract)；分类 cs.CV

2205.09307 2022-05-20 cs.CV 79%

Support-set based Multi-modal Representation Enhancement for Video Captioning

Xiaoya Chen, Jingkuan Song, Pengpeng Zeng, Lianli Gao, Heng Tao Shen

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

2203.00843 2022-04-07 cs.CV 79%

X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning

Zhihao Yuan, Xu Yan, Yinghong Liao, Yao Guo, Guanbin Li, Zhen Li, Shuguang Cui

专题命中图文多模态：cross-modal(title,abstract)；分类 cs.CV

Comments To appear in CVPR2022

2203.09173 2022-03-18 cs.CL 79%

On Vision Features in Multimodal Machine Translation

Bei Li, Chuanhao Lv, Zefan Zhou, Tao Zhou, Tong Xiao, Anxiang Ma, JingBo Zhu

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Long paper accepted by ACL2022 main conference

2203.07519 2022-03-18 cs.CL 79%

Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer

Woojeong Jin, Dong-Ho Lee, Chenguang Zhu, Jay Pujara, Xiang Ren

专题命中图文多模态：cross-modal(title,abstract)；分类 cs.CL

Comments Accepted to ACL 2022, 13 pages, 4 figures

2109.05812 2022-02-16 cs.CL 79%

UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation

Zhengkun Zhang, Xiaojun Meng, Yasheng Wang, Xin Jiang, Qun Liu, Zhenglu Yang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at AAAI2022

2101.01881 2021-10-25 cs.CV cs.LG 79%

MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding

Woojeong Jin, Maziar Sanjabi, Shaoliang Nie, Liang Tan, Xiang Ren, Hamed Firooz

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted to EMNLP 2021 Findings

2108.05863 2021-08-13 cs.CV 79%

Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision

Xiaoshi Wu, Hadar Averbuch-Elor, Jin Sun, Noah Snavely

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments Published in ICCV 2021; Project webpage: https://www.cs.cornell.edu/projects/babel/

2106.02400 2021-06-07 cs.CV cs.IR cs.LG 79%

A Deep Local and Global Scene-Graph Matching for Image-Text Retrieval

Manh-Duy Nguyen, Binh T. Nguyen, Cathal Gurrin

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

2103.16022 2021-03-31 cs.CV 79%

Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays

Xiaosong Wang, Ziyue Xu, Leo Tam, Dong Yang, Daguang Xu

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

1601.03916 2021-02-03 cs.CL 79%

Multimodal Pivots for Image Caption Translation

Julian Hitschler, Shigehiko Schamoni, Stefan Riezler

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Final version, accepted at ACL 2016. New section on Human Evaluation

2007.08883 2021-02-02 cs.CV 79%

Consensus-Aware Visual-Semantic Embedding for Image-Text Matching

Haoran Wang, Ying Zhang, Zhong Ji, Yanwei Pang, Lin Ma

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2020, Code is publicly available at: https://github.com/BruceW91/CVSE

2003.03669 2020-07-23 cs.CV 79%

Adaptive Offline Quintuplet Loss for Image-Text Matching

Tianlang Chen, Jiajun Deng, Jiebo Luo

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by ECCV 2020. Code is available at https://github.com/sunnychencool/AOQ

1806.08251 2020-07-08 cs.CV 79%

Learning Multimodal Representations for Unseen Activities

AJ Piergiovanni, Michael S. Ryoo

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Journal ref WACV 2020

2004.01894 2020-04-07 cs.CL 79%

Evaluating Multimodal Representations on Visual Semantic Textual Similarity

Oier Lopez de Lacalle, Ander Salaberria, Aitor Soroa, Gorka Azkune, Eneko Agirre

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted in ECAI-2020, 8 pages, 6 tables, 6 figures

2004.00277 2020-04-02 cs.CV 79%

Graph Structured Network for Image-Text Matching

Chunxiao Liu, Zhendong Mao, Tianzhu Zhang, Hongtao Xie, Bin Wang, Yongdong Zhang

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by CVPR2020

2002.08510 2020-02-21 cs.CV 79%

Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching

Tianlang Chen, Jiebo Luo

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted by AAAI-20

1911.09042 2019-11-26 cs.CV 79%

Learning Cross-modal Context Graph for Visual Grounding

Yongfei Liu, Bo Wan, Xiaodan Zhu, Xuming He

专题命中图文多模态：cross-modal(title,abstract)；分类 cs.CV

Comments AAAI-2020

1910.05752 2019-10-15 cs.CL 79%

VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning

Ziqi Zhang, Yaya Shi, Jiutong Wei, Chunfeng Yuan, Bing Li, Weiming Hu

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CL

1909.02701 2019-09-09 cs.CV 79%

Visual Semantic Reasoning for Image-Text Matching

Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, Yun Fu

专题命中图文多模态：image-text(title,abstract)；分类 cs.CV

Comments Accepted to ICCV 2019 (Oral)

1908.07519 2019-08-22 cs.CV cs.HC cs.LG eess.IV eess.SP 79%

Multi-Modal Recognition of Worker Activity for Human-Centered Intelligent Manufacturing

Wenjin Tao, Ming C. Leu, Zhaozheng Yin

专题命中图文多模态：multi-modal(title,abstract)；分类 cs.CV

Comments 17 pages, 8 figures, 6 tables

1907.09748 2019-07-24 cs.CL cs.IR cs.LG 79%

Position Focused Attention Network for Image-Text Matching

Yaxiong Wang, Hao Yang, Xueming Qian, Lin Ma, Jing Lu, Biao Li, Xin Fan

专题命中图文多模态：image-text(title,abstract)；分类 cs.CL

Comments Accepted by IJCAI 2019

1905.08181 2019-05-31 cs.CL 79%

A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks

Álvaro Peris, Francisco Casacuberta

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CL

Comments ACL 2019 - System demonstrations

1905.07841 2019-05-21 cs.CV 79%

Multimodal Transformer with Multi-View Visual Representation for Image Captioning

Jun Yu, Jing Li, Zhou Yu, Qingming Huang

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments submitted to a journal

1905.02963 2019-05-09 cs.CV 79%

Multimodal Semantic Attention Network for Video Captioning

Liang Sun, Bing Li, Chunfeng Yuan, Zhengjun Zha, Weiming Hu

专题命中图文多模态：multimodal(title,abstract)；分类 cs.CV

Comments 6 pages, 4 figures, accepted by IEEE International Conference on Multimedia and Expo (ICME) 2019

1. 图文多模态 4676 篇

Transferring General Multimodal Pretrained Models to Text Recognition

Clustering-Induced Generative Incomplete Image-Text Clustering (CIGIT-C)

UniColor: A Unified Framework for Multi-Modal Colorization with Transformer

Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training

Cross-Modal Graph with Meta Concepts for Video Captioning

Semi-Supervised Cross-Modal Salient Object Detection with U-Structure Networks

Support-set based Multi-modal Representation Enhancement for Video Captioning

X-Trans2Cap: Cross-Modal Knowledge Transfer using Transformer for 3D Dense Captioning

On Vision Features in Multimodal Machine Translation

Leveraging Visual Knowledge in Language Tasks: An Empirical Study on Intermediate Pre-training for Cross-modal Knowledge Transfer

UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation

MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding

Towers of Babel: Combining Images, Language, and 3D Geometry for Learning Multimodal Vision

A Deep Local and Global Scene-Graph Matching for Image-Text Retrieval

Self-supervised Image-text Pre-training With Mixed Data In Chest X-rays

Multimodal Pivots for Image Caption Translation

Consensus-Aware Visual-Semantic Embedding for Image-Text Matching

Adaptive Offline Quintuplet Loss for Image-Text Matching

Learning Multimodal Representations for Unseen Activities

Evaluating Multimodal Representations on Visual Semantic Textual Similarity

Graph Structured Network for Image-Text Matching

Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching

Learning Cross-modal Context Graph for Visual Grounding

VATEX Captioning Challenge 2019: Multi-modal Information Fusion and Multi-stage Training Strategy for Video Captioning

Visual Semantic Reasoning for Image-Text Matching

Multi-Modal Recognition of Worker Activity for Human-Centered Intelligent Manufacturing

Position Focused Attention Network for Image-Text Matching

A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks

Multimodal Transformer with Multi-View Visual Representation for Image Captioning

Multimodal Semantic Attention Network for Video Captioning