多模态大模型 - arXivDaily 专题

2104.02862 2022-12-27 cs.CV 79%

Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification

Xudong Tian, Zhizhong Zhang, Shaohui Lin, Yanyun Qu, Yuan Xie, Lizhuang Ma

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted by CVPR 2022 as Oral presentation

2212.10030 2022-12-21 cs.AI 79%

InterMulti:Multi-view Multimodal Interactions with Text-dominated Hierarchical High-order Fusion for Emotion Analysis

Feng Qiu, Wanzeng Kong, Yu Ding

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

Comments 9 pages, 3 figures. arXiv admin note: text overlap with arXiv:2212.08661

2112.11088 2022-12-21 cs.CV 79%

EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection

Zhe Liu, Tengteng Huang, Bingling Li, Xiwu Chen, Xi Wang, Xiang Bai

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by TPAMI-2022

2211.16509 2022-12-20 q-bio.GN cs.AI cs.LG q-bio.BM stat.ML 79%

Multimodal Learning for Multi-Omics: A Survey

Sina Tabakhi, Mohammod Naimul Islam Suvon, Pegah Ahadian, Haiping Lu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

Comments 52 pages, 3 figures; Revised matrix factorization fusion section

2212.07039 2022-12-15 cs.CV 79%

Multi-Modal Domain Fusion for Multi-modal Aerial View Object Classification

Sumanth Udupa, Aniruddh Sikdar, Suresh Sundaram

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 7 pages,2 figures

2212.04661 2022-12-12 cs.CV 79%

An Attention-based Multi-Scale Feature Learning Network for Multimodal Medical Image Fusion

Meng Zhou, Xiaolan Xu, Yuxuan Zhang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 8 pages, 8 figures, 3 tables

2212.03943 2022-12-09 cs.CV 79%

Learning Polysemantic Spoof Trace: A Multi-Modal Disentanglement Network for Face Anti-spoofing

Kaicheng Li, Hongyu Yang, Binghui Chen, Pengyu Li, Biao Wang, Di Huang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted by AAAI 2023

1807.02444 2022-12-06 cs.CV physics.optics 79%

Multi-modal Non-line-of-sight Passive Imaging

Andre Beckus, Alexandru Tamasan, George K. Atia

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Journal ref IEEE Transactions on Image Processing, vol. 28, no. 7, pp. 3372-3382, July 2019

2211.14042 2022-11-28 cs.LG cs.AI q-bio.BM 79%

Molecular Joint Representation Learning via Multi-modal Information

Tianyu Wu, Yang Tang, Qiyu Sun, Luolin Xiong

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.AI

2211.13309 2022-11-28 cs.CV cs.LG 79%

How do Cross-View and Cross-Modal Alignment Affect Representations in Contrastive Learning?

Thomas M. Hehn, Julian F. P. Kooij, Dariu M. Gavrila

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

2204.11515 2022-11-24 cs.CY cs.CV 79%

Multimodal Dual Emotion with Fusion of Visual Sentiment for Rumor Detection

Ge Wang, Li Tan, Ziliang Shang, He Liu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments There is an error in the experimental recording process, and the results reported in the article need to be re-checked

2005.06331 2022-11-24 cs.IR cs.CV cs.LG 79%

Multi-modal Embedding Fusion-based Recommender

Anna Wroblewska, Jacek Dabrowski, Michal Pastuszak, Andrzej Michalowski, Michal Daniluk, Barbara Rychalska, Mikolaj Wieczorek, Sylwia Sysko-Romanczuk

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 7 pages, 8 figures

Journal ref revised and improved version: Electronics MDPI - https://www.mdpi.com/2079-9292/11/9/1391

2211.12021 2022-11-23 cs.CV 79%

ViFi-Loc: Multi-modal Pedestrian Localization using GAN with Camera-Phone Correspondences

Hansi Liu, Kristin Dana, Marco Gruteser, Hongsheng Lu

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2211.06168 2022-11-23 cs.CL 79%

Unimodal and Multimodal Representation Training for Relation Extraction

Ciaran Cooney, Rachel Heyburn, Liam Madigan, Mairead O'Cuinn, Chloe Thompson, Joana Cavadas

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments 11 pages, 3 figures, 30th Irish Conference on Artificial Intelligence and Cognitive Science

2211.09807 2022-11-22 cs.CV 79%

Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information

Weijie Su, Xizhou Zhu, Chenxin Tao, Lewei Lu, Bin Li, Gao Huang, Yu Qiao, Xiaogang Wang, Jie Zhou, Jifeng Dai

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2209.00302 2022-11-22 cs.LG cs.MM 79%

Progressive Fusion for Multimodal Integration

Shiv Shankar, Laure Thompson, Madalina Fiterau

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.MM

2202.03390 2022-11-21 cs.LG cs.AI 79%

Geometric Multimodal Contrastive Representation Learning

Petra Poklukar, Miguel Vasco, Hang Yin, Francisco S. Melo, Ana Paiva, Danica Kragic

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

Comments ICML 2022 Camera ready version (update)

2211.06080 2022-11-14 cs.RO cs.CV 79%

Multi-modal Fusion Technology based on Vehicle Information: A Survey

Yan Gong, Jianli Lu, Jiayi Wu, Wenzhuo Liu

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

2211.02269 2022-11-07 cs.CL 79%

Late Fusion with Triplet Margin Objective for Multimodal Ideology Prediction and Analysis

Changyuan Qiu, Winston Wu, Xinliang Frederick Zhang, Lu Wang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL

Comments EMNLP 2022

2211.01353 2022-11-03 eess.IV cs.CV cs.LG 79%

Fourier Disentangled Multimodal Prior Knowledge Fusion for Red Nucleus Segmentation in Brain MRI

Guanghui Fu, Gabriel Jimenez, Sophie Loizillon, Rosana El Jurdi, Lydia Chougar, Didier Dormont, Romain Valabregue, Ninon Burgos, Stéphane Lehéricy, Daniel Racoceanu, Olivier Colliot, the ICEBERG Study Group

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2211.00207 2022-11-02 cs.CV 79%

GMF: General Multimodal Fusion Framework for Correspondence Outlier Rejection

Xiaoshui Huang, Wentao Qu, Yifan Zuo, Yuming Fang, Xiaowei Zhao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by IEEE RAL

2208.12079 2022-11-01 cs.CV 79%

Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection

Taohua Zhou, Yining Shi, Junjie Chen, Kun Jiang, Mengmeng Yang, Diange Yang

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments 12 pages,6 figures

2206.12078 2022-10-27 cs.LG cs.AI eess.SP 79%

Multimodal sensor data fusion for in-situ classification of animal behavior using accelerometry and GNSS data

Reza Arablouei, Ziwei Wang, Greg J. Bishop-Hurley, Jiajun Liu

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.AI

2203.05406 2022-10-26 cs.IR cs.MM 79%

Disentangled Multimodal Representation Learning for Recommendation

Fan Liu, Huilin Chen, Zhiyong Cheng, Anan Liu, Liqiang Nie, Mohan Kankanhalli

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.MM

Comments IEEE Transactions on Multimedia (TMM)

2203.13530 2022-10-25 cs.CV 79%

Multimodal Pre-training Based on Graph Attention Network for Document Understanding

Zhenrong Zhang, Jiefeng Ma, Jun Du, Licheng Wang, Jianshu Zhang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

2210.09550 2022-10-19 cs.CL 79%

Probing Cross-modal Semantics Alignment Capability from the Textual Perspective

Zheng Ma, Shi Zong, Mianzhi Pan, Jianbing Zhang, Shujian Huang, Xinyu Dai, Jiajun Chen

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CL

Comments Findings of EMNLP2022

2210.04843 2022-10-11 cs.LG cs.CV 79%

Multi-Modal Fusion by Meta-Initialization

Matthew T. Jackson, Shreshth A. Malik, Michael T. Matthews, Yousuf Mohamed-Ahmed

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV

Comments The first two authors contributed equally

2106.05124 2022-10-11 cs.CV 79%

PCNet: A Structure Similarity Enhancement Method for Multispectral and Multimodal Image Registration

Si-Yuan Cao, Beinan Yu, Lun Luo, Shu-Jie Chen, Chunguang Li, Hui-Liang Shen

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments 33 pages, 16 figures

2112.02252 2022-10-05 cs.CV 79%

Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction

Yikai Wang, Fuchun Sun, Wenbing Huang, Fengxiang He, Dacheng Tao

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV

Comments Accepted by TPAMI 2022. Code is available at https://github.com/yikaiw/CEN. arXiv admin note: text overlap with arXiv:2011.05005

2209.13822 2022-10-04 cs.CV 79%

TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval

Xiaohan Zou, Changqiao Wu, Lele Cheng, Zhongyuan Wang

专题命中多模态训练与对齐：cross-modal(title,abstract)；分类 cs.CV

1. 多模态训练与对齐 6918 篇

Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification

InterMulti:Multi-view Multimodal Interactions with Text-dominated Hierarchical High-order Fusion for Emotion Analysis

EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection

Multimodal Learning for Multi-Omics: A Survey

Multi-Modal Domain Fusion for Multi-modal Aerial View Object Classification

An Attention-based Multi-Scale Feature Learning Network for Multimodal Medical Image Fusion

Learning Polysemantic Spoof Trace: A Multi-Modal Disentanglement Network for Face Anti-spoofing

Multi-modal Non-line-of-sight Passive Imaging

Molecular Joint Representation Learning via Multi-modal Information

How do Cross-View and Cross-Modal Alignment Affect Representations in Contrastive Learning?

Multimodal Dual Emotion with Fusion of Visual Sentiment for Rumor Detection

Multi-modal Embedding Fusion-based Recommender

ViFi-Loc: Multi-modal Pedestrian Localization using GAN with Camera-Phone Correspondences

Unimodal and Multimodal Representation Training for Relation Extraction

Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information

Progressive Fusion for Multimodal Integration

Geometric Multimodal Contrastive Representation Learning

Multi-modal Fusion Technology based on Vehicle Information: A Survey

Late Fusion with Triplet Margin Objective for Multimodal Ideology Prediction and Analysis

Fourier Disentangled Multimodal Prior Knowledge Fusion for Red Nucleus Segmentation in Brain MRI

GMF: General Multimodal Fusion Framework for Correspondence Outlier Rejection

Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection

Multimodal sensor data fusion for in-situ classification of animal behavior using accelerometry and GNSS data

Disentangled Multimodal Representation Learning for Recommendation

Multimodal Pre-training Based on Graph Attention Network for Document Understanding

Probing Cross-modal Semantics Alignment Capability from the Textual Perspective

Multi-Modal Fusion by Meta-Initialization

PCNet: A Structure Similarity Enhancement Method for Multispectral and Multimodal Image Registration

Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction

TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval