多模态大模型

AFNet-M: Adaptive Fusion Network with Masks for 2D+3D Facial Expression Recognition

Mingzhe Sui, Hanting Li, Zhaoqing Zhu, Feng Zhao

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 6 pages, 6 figures, 4 tables

Classification of Astronomical Bodies by Efficient Layer Fine-Tuning of Deep Neural Networks

Sabeesh Ethiraj, Bharath Kumar Bolla

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at 5th Conference on Information and Communication Technology (CICT), 2021

SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, Yuan Cao

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Published at ICLR 2022

Unified Pretraining Framework for Document Understanding

Jiuxiang Gu, Jason Kuen, Vlad I. Morariu, Handong Zhao, Nikolaos Barmpalios, Rajiv Jain, Ani Nenkova, Tong Sun

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 12 pages, 4 figures, NeurIPS 2021 (Updated Camera Ready)

SeqDialN: Sequential Visual Dialog Networks in Joint Visual-Linguistic Representation Space

Liu Yang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments This submission has been withdrawn by arXiv administrators due to an unresolvable authorship dispute

General Facial Representation Learning in a Visual-Linguistic Manner

Yinglin Zheng, Hao Yang, Ting Zhang, Jianmin Bao, Dongdong Chen, Yangyu Huang, Lu Yuan, Dong Chen, Ming Zeng, Fang Wen

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV、cs.CL

Comments CVPR2022 Oral; 16 pages, 6 figures, 14 tables

Omnivore: A Single Model for Many Visual Modalities

Rohit Girdhar, Mannat Singh, Nikhila Ravi, Laurens van der Maaten, Armand Joulin, Ishan Misra

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at CVPR 2022 (Oral Presentation)

Shifting More Attention to Visual Backbone: Query-modulated Refinement Networks for End-to-End Visual Grounding

Jiabo Ye, Junfeng Tian, Ming Yan, Xiaoshan Yang, Xuwu Wang, Ji Zhang, Liang He, Xin Lin

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.MM

DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection

Jinhyung Park, Chenfeng Xu, Yiyang Zhou, Masayoshi Tomizuka, Wei Zhan

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments https://github.com/Divadi/DetMatch

Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations

Robert Wolfe, Aylin Caliskan

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CL、cs.AI

Comments To be published in ACL 2022

Grounding Commands for Autonomous Vehicles via Layer Fusion with Region-specific Dynamic Layer Attention

Hou Pong Chan, Mingxi Guo, Cheng-Zhong Xu

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Submitted to IROS 2022

Fast Road Segmentation via Uncertainty-aware Symmetric Network

Yicong Chang, Feng Xue, Fei Sheng, Wenteng Liang, Anlong Ming

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICRA 2022

Visual-Language Navigation Pretraining via Prompt-based Environmental Self-exploration

Xiwen Liang, Fengda Zhu, Lingling Li, Hang Xu, Xiaodan Liang

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACL 2022

Multi-View Fusion Transformer for Sensor-Based Human Activity Recognition

Yimu Wang, Kun Yu, Yan Wang, Hui Xue

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 5 pages, 1 figure

Self-attention fusion for audiovisual emotion recognition with incomplete data

Kateryna Chumachenko, Alexandros Iosifidis, Moncef Gabbouj

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、eess.AS

RegionCLIP: Region-based Language-Image Pretraining

Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, Jianfeng Gao

专题命中 多模态训练与对齐 ：image-text(abstract)；分类 cs.CV、cs.AI

Comments Technical report

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 15 pages

Driving Behavior Explanation with Multi-level Fusion

Hédi Ben-Younes, Éloi Zablocki, Patrick Pérez, Matthieu Cord

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted at NeurIPS Workshop ML4AD 2020

Journal ref Pattern Recognition, Volume 123, March 2022, 108421

Achieving Human Parity on Visual Question Answering

Ming Yan, Haiyang Xu, Chenliang Li, Junfeng Tian, Bin Bi, Wei Wang, Weihua Chen, Xianzhe Xu, Fan Wang, Zheng Cao, Zhicheng Zhang, Qiyu Zhang, Ji Zhang, Songfang Huang, Fei Huang, Luo Si, Rong Jin

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Emotion Recognition from Multiple Modalities: Fundamentals and Methodologies

Sicheng Zhao, Guoli Jia, Jufeng Yang, Guiguang Ding, Kurt Keutzer

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.AI、cs.MM

Comments Accepted by IEEE Signal Processing Magazine (SPM)

A Prior Guided Adversarial Representation Learning and Hypergraph Perceptual Network for Predicting Abnormal Connections of Alzheimer's Disease

Qiankun Zuo, Baiying Lei, Shuqiang Wang, Yong Liu, Bingchuan Wang, Yanyan Shen

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Dense Contrastive Visual-Linguistic Pretraining

Lei Shi, Kai Shuang, Shijie Geng, Peng Gao, Zuohui Fu, Gerard de Melo, Yunpeng Chen, Sen Su

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted by ACM Multimedia 2021. arXiv admin note: text overlap with arXiv:2007.13135

Fusion with Hierarchical Graphs for Mulitmodal Emotion Recognition

Shuyun Tang, Zhaojie Luo, Guoshun Nan, Yuichiro Yoshikawa, Ishiguro Hiroshi

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.AI、cs.MM

Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search

Jialu Wang, Yang Liu, Xin Eric Wang

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments 14 pages, EMNLP 2021

TxT: Crossmodal End-to-End Learning with Transformers

Jan-Martin O. Steitz, Jonas Pfeiffer, Iryna Gurevych, Stefan Roth

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments To appear at the 43rd DAGM German Conference on Pattern Recognition (GCPR) 2021

Data Efficient Masked Language Modeling for Vision and Language

Yonatan Bitton, Gabriel Stanovsky, Michael Elhadad, Roy Schwartz

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to Findings of EMNLP 2021

A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data

Muhammad Mobaidul Islam, Abdullah Al Redwan Newaz, Ali Karimoddini

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Episodic Transformer for Vision-and-Language Navigation

Alexander Pashevich, Cordelia Schmid, Chen Sun

专题命中 多模态训练与对齐 ：multimodal(abstract)；分类 cs.CV、cs.AI

Comments ICCV 2021; 19 pages; Code available at https://github.com/alexpashevich/E.T

TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment

Jianwei Yang, Yonatan Bisk, Jianfeng Gao

专题命中 多模态训练与对齐 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICCV 2021

OVIS: Open-Vocabulary Visual Instance Search via Visual-Semantic Aligned Representation Learning

Sheng Liu, Kevin Lin, Lijuan Wang, Junsong Yuan, Zicheng Liu

专题命中 多模态训练与对齐 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 10 pages