多模态大模型

Open-Vocabulary Instance Segmentation via Robust Cross-Modal Pseudo-Labeling

Dat Huynh, Jason Kuen, Zhe Lin, Jiuxiang Gu, Ehsan Elhamifar

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Transformer-based Multimodal Information Fusion for Facial Expression Analysis

Wei Zhang, Feng Qiu, Suzhen Wang, Hao Zeng, Zhimeng Zhang, Rudong An, Bowen Ma, Yu Ding

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Colored Point Cloud to Image Alignment

Noam Rotstein, Amit Bracha, Ron Kimmel

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Network

Kai Xuan, Lei Xiang, Xiaoqian Huang, Lichi Zhang, Shu Liao, Dinggang Shen, Qian Wang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments Final version, IEEE Transactions on Medical Imaging, code available at \url{https://github.com/woxuankai/SpatialAlignmentNetwork}

M2TS: Multi-Scale Multi-Modal Approach Based on Transformer for Source Code Summarization

Yuexiu Gao, Chen Lyu

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted by ICPC 2022

Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers

Arthur Bucker, Luis Figueredo, Sami Haddadin, Ashish Kapoor, Shuang Ma, Rogerio Bonatti

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.AI

Multi-Modal Learning for AU Detection Based on Multi-Head Fused Transformers

Xiang Zhang, Lijun Yin

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref FG 2021

DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection

Yingwei Li, Adams Wei Yu, Tianjian Meng, Ben Caine, Jiquan Ngiam, Daiyi Peng, Junyang Shen, Bo Wu, Yifeng Lu, Denny Zhou, Quoc V. Le, Alan Yuille, Mingxing Tan

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments CVPR 2022. 1st rank 3D detection method on Waymo Challenge Leaderboard: https://waymo.com/open/challenges/entry/?timestamp=1647356360224524&challenge=DETECTION_3D&emailId=5451f123-a0ea

Modular and Parameter-Efficient Multimodal Fusion with Prompting

Sheng Liang, Mengjie Zhao, Hinrich Schütze

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to Findings of ACL 2022

Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion

Ziqi Huang, Li Lin, Pujin Cheng, Linkai Peng, Xiaoying Tang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments 6 pages, 5 figures, submitted to ICPR 2022

Multi-Modal Recurrent Fusion for Indoor Localization

Jianyuan Yu, Pu, Wang, Toshiaki Koike-Akino, Philip V. Orlik

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Comments 5 pages, 4 figures, 1 table

Deep multi-modal aggregation network for MR image reconstruction with auxiliary modality

Chun-Mei Feng, Huazhu Fu, Tianfei Zhou, Yong Xu, Ling Shao, David Zhang

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Cross-Modal Distillation for RGB-Depth Person Re-Identification

Frank Hafner, Amran Bhuiyan, Julian F. P. Kooij, Eric Granger

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Journal ref Computer Vision and Image Understanding, 103352 (2022)

Anchor Graph Structure Fusion Hashing for Cross-Modal Similarity Search

Lu Wang, Jie Yang, Masoumeh Zareapoor, Zhonglong Zheng

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Multi-channel Attentive Graph Convolutional Network With Sentiment Fusion For Multimodal Sentiment Analysis

Luwei Xiao, Xingjiao Wu, Wen Wu, Jing Yang, Liang He

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CL

CM3: A Causal Masked Multimodal Model of the Internet

Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer

专题命中 多模态训练与对齐 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL

Hardware Implementation of Multimodal Biometric using Fingerprint and Iris

Tariq M Khan

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions

Anil Rahate, Rahee Walambe, Sheela Ramanna, Ketan Kotecha

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments This is published in Information Fusion Journal and published copy can be downloaded from https://authors.elsevier.com/c/1eIVS5a7-Gls0Y for 50 days. https://www.sciencedirect.com/science/article/pii/S1566253521002530

Journal ref Information Fusion 81(2022) 203-239

LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding

Yang Xu, Yiheng Xu, Tengchao Lv, Lei Cui, Furu Wei, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Wanxiang Che, Min Zhang, Lidong Zhou

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CL

Comments ACL 2021 main conference

Multi-modal Visual Place Recognition in Dynamics-Invariant Perception Space

Lin Wu, Teng Wang, Changyin Sun

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Journal ref IEEE Signal Processing Letters 2021

BM-NAS: Bilevel Multimodal Neural Architecture Search

Yihang Yin, Siyu Huang, Xiang Zhang

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by AAAI 2022

Understanding and Measuring Robustness of Multimodal Learning

Nishant Vishwamitra, Hongxin Hu, Ziming Zhao, Long Cheng, Feng Luo

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.MM

Learning Aligned Cross-Modal Representation for Generalized Zero-Shot Classification

Zhiyu Fang, Xiaobin Zhu, Chun Yang, Zheng Han, Jingyan Qin, Xu-Cheng Yin

专题命中 多模态训练与对齐 ：cross-modal(title,abstract)；分类 cs.CV

Multimodal Representation Learning via Maximization of Local Mutual Information

Ruizhi Liao, Daniel Moyer, Miriam Cha, Keegan Quigley, Seth Berkowitz, Steven Horng, Polina Golland, William M. Wells

专题命中 多模态训练与对齐 ：multimodal(title)；image-text(abstract)；分类 cs.CV

Comments In Proceedings of International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2021

Journal ref In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 273-283. Springer, Cham, 2021

High-resolution Depth Maps Imaging via Attention-based Hierarchical Multi-modal Fusion

Zhiwei Zhong, Xianming Liu, Junjun Jiang, Debin Zhao, Zhiwen Chen, Xiangyang Ji

专题命中 多模态训练与对齐 ：multi-modal(title,abstract)；分类 cs.CV

Routing with Self-Attention for Multimodal Capsule Networks

Kevin Duarte, Brian Chen, Nina Shvetsova, Andrew Rouditchenko, Samuel Thomas, Alexander Liu, David Harwath, James Glass, Hilde Kuehne, Mubarak Shah

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Geometric Multimodal Deep Learning with Multi-Scaled Graph Wavelet Convolutional Network

Maysam Behmanesh, Peyman Adibi, Mohammad Saeed Ehsani, Jocelyn Chanussot

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.AI

Comments 13 pages

Sparse Fusion for Multimodal Transformers

Yi Ding, Alex Rich, Mason Wang, Noah Stier, Matthew Turk, Pradeep Sen, Tobias Höllerer

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments 11 pages, 4 figures, 5 tables, Yi Ding and Alex Rich contributed equally

IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration

Xiaoshui Huang, Wentao Qu, Yifan Zuo, Yuming Fang, Xiaowei Zhao

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Technical report

Benchmarking Quality-Dependent and Cost-Sensitive Score-Level Multimodal Biometric Fusion Algorithms

Norman Poh, Thirimachos Bourlai, Josef Kittler, Lorene Allano, Fernando Alonso-Fernandez, Onkar Ambekar, John Baker, Bernadette Dorizzi, Omolara Fatukasi, Julian Fierrez, Harald Ganster, Javier Ortega-Garcia, Donald Maurer, Albert Ali Salah, Tobias Scheidat, Claus Vielhauer

专题命中 多模态训练与对齐 ：multimodal(title,abstract)；分类 cs.CV

Comments Published at IEEE Transactions on Information Forensics and Security journal