多模态大模型 - arXivDaily 专题

2207.07278 2023-04-07 cs.CV cs.CL cs.MM 82%

Boosting Multi-Modal E-commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization

Mengyin Liu, Chao Zhu, Hongyu Gao, Weibo Gu, Hongfa Wang, Wei Liu, Xu-cheng Yin

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.MM

2211.15028 2023-02-21 cs.CL cs.AI cs.MM 82%

Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging

Li Yuan, Yi Cai, Jin Wang, Qing Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

Comments accepted in AAAI-2023

2302.07740 2023-02-16 cs.CL cs.AI cs.CV cs.LG 82%

Team Triple-Check at Factify 2: Parameter-Efficient Large Foundation Models with Feature Representations for Multi-Modal Fact Verification

Wei-Wei Du, Hong-Wei Wu, Wei-Yao Wang, Wen-Chih Peng

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments AAAI-23 DeFactify 2 Workshop (1st Prize)

2302.05289 2023-02-13 cs.CV cs.AI cs.CL cs.SI 82%

Rumor Classification through a Multimodal Fusion Framework and Ensemble Learning

Abderrazek Azri, Cécile Favre, Nouria Harbi, Jérôme Darmont, Camille Noûs

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Information Systems Frontiers, 2022

2301.00254 2023-01-03 cs.MM cs.AI cs.CV 82%

Depression Diagnosis and Analysis via Multimodal Multi-order Factor Fusion

Chengbo Yuan, Qianhui Xu, Yong Luo

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.AI、cs.MM

2111.10974 2022-12-29 cs.CV cs.AI cs.CL 82%

Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture

Daria Bakshandaeva, Denis Dimitrov, Vladimir Arkhipkin, Alex Shonenkov, Mark Potanin, Denis Karachev, Andrey Kuznetsov, Anton Voronov, Vera Davydova, Elena Tutubalina, Aleksandr Petiushko

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

2211.14739 2022-11-29 cs.CV cs.AI cs.CL 82%

MNER-QG: An End-to-End MRC framework for Multimodal Named Entity Recognition with Query Grounding

Meihuizi Jia, Lei Shen, Xin Shen, Lejian Liao, Meng Chen, Xiaodong He, Zhendong Chen, Jiaqi Li

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments 13 pages, 6 figures, published to AAAI

2208.11064 2022-10-18 cs.LG 82%

Multi-Modal Representation Learning with Self-Adaptive Threshold for Commodity Verification

Chenchen Han, Heng Jia

专题命中多模态训练与对齐：multi-modal(title,abstract)；multimodal(abstract)

2209.11949 2022-09-27 cs.LG cs.AI cs.CL cs.MM 82%

Hybrid Multimodal Fusion for Humor Detection

Haojie Xu, Weifeng Liu, Jingwei Liu, Mingzheng Li, Yu Feng, Yasi Peng, Yunwei Shi, Xiao Sun, Meng Wang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

Comments 7 pages, 1 figure, to appear in MuSe 2022 (ACM MM2022 co-located workshop)

2209.00891 2022-09-05 cs.CL cs.AI cs.LG cs.MM 82%

Multi-modal Contrastive Representation Learning for Entity Alignment

Zhenxi Lin, Ziheng Zhang, Meng Wang, Yinghui Shi, Xian Wu, Yefeng Zheng

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CL、cs.AI、cs.MM

Comments Accepted by COLING 2022

2208.03051 2022-08-15 cs.CV cs.CL cs.SD eess.AS eess.IV 82%

Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysis

Jia Li, Ziyang Zhang, Junjie Lang, Yueqi Jiang, Liuwei An, Peng Zou, Yangyang Xu, Sheng Gao, Jie Lin, Chunxiao Fan, Xiao Sun, Meng Wang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、eess.AS

Comments 8 pages, 2 figures, to appear in MuSe 2022 (ACM MM2022 co-located workshop)

2204.04777 2022-04-12 cs.LG 82%

Multimodal Machine Learning in Precision Health

Adrienne Kline, Hanyin Wang, Yikuan Li, Saya Dennis, Meghan Hutch, Zhenxing Xu, Fei Wang, Feixiong Cheng, Yuan Luo

专题命中多模态训练与对齐：multimodal(title,abstract)；multi-modal(abstract)

2112.12072 2021-12-23 cs.CV cs.AI cs.CL 82%

Hierarchical Cross-Modality Semantic Correlation Learning Model for Multimodal Summarization

Litian Zhang, Xiaoming Zhang, Junshu Pan, Feiran Huang

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted by AAAI2022

2106.03821 2021-09-16 cs.SD cs.CL cs.CV eess.AS 82%

Active Speaker Detection as a Multi-Objective Optimization with Uncertainty-based Multimodal Fusion

Baptiste Pouthier, Laurent Pilati, Leela K. Gudupudi, Charles Bouveyron, Frederic Precioso

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、eess.AS

Comments In INTERSPEECH 2021

Journal ref Proc. Interspeech 2021, 2381-2385

2109.00895 2021-09-03 cs.CV cs.AI cs.CL 82%

Knowledge Perceived Multi-modal Pretraining in E-commerce

Yushan Zhu, Huaixiao Tou, Wen Zhang, Ganqiang Ye, Hui Chen, Ningyu Zhang, Huajun Chen

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to ACM MM 2021

2108.00809 2021-08-03 eess.IV 82%

Cross-Modal Knowledge Transfer via Inter-Modal Translation and Alignment for Affect Recognition

Vandana Rajan, Alessio Brutti, Andrea Cavallaro

专题命中多模态训练与对齐：cross-modal(title,abstract)；multi-modal(abstract)

Comments Under review

2103.05597 2021-03-10 cs.LG 82%

A Discriminative Vectorial Framework for Multi-modal Feature Representation

Lei Gao, Ling Guan

专题命中多模态训练与对齐：multi-modal(title,abstract)；cross-modal(abstract)

Comments Accepted

Journal ref IEEE Transactions on Multimedia, 2021

1911.03821 2021-01-27 cs.CL cs.CV cs.LG eess.AS 82%

Adaptive Fusion Techniques for Multimodal Data

Gaurav Sahu, Olga Vechtomova

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、eess.AS

Comments Camera-ready version for EACL 2021

1906.04115 2020-08-27 cs.LG stat.ML 82%

Robust Multi-Modal Sensor Fusion: An Adversarial Approach

Siddharth Roheda, Hamid Krim, Benjamin S. Riggan

专题命中多模态训练与对齐：multi-modal(title,abstract)；multimodal(abstract)

1911.03977 2020-07-15 cs.AI cs.CL cs.CV cs.LG 82%

Multimodal Intelligence: Representation Learning, Information Fusion, and Applications

Chao Zhang, Zichao Yang, Xiaodong He, Li Deng

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

1904.11171 2020-04-02 cs.IR 82%

Fusion-supervised Deep Cross-modal Hashing

Li Wang, Lei Zhu, En Yu, Jiande Sun, Huaxiang Zhang

专题命中多模态训练与对齐：cross-modal(title,abstract)；multi-modal(abstract)

2003.12383 2020-03-30 cs.AI cs.CL cs.CV cs.LG 82%

End-to-End Entity Classification on Multimodal Knowledge Graphs

W. X. Wilcke, P. Bloem, V. de Boer, R. H. van t Veer, F. A. H. van Harmelen

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Submitted to the 17th International Conference on Principles of Knowledge Representation and Reasoning (2020)

1904.09078 2019-04-22 cs.LG stat.ML 82%

EmbraceNet: A robust deep learning architecture for multimodal classification

Jun-Ho Choi, Jong-Seok Lee

专题命中多模态训练与对齐：multimodal(title,abstract)；cross-modal(abstract)

Comments Code available at https://github.com/idearibosome/embracenet

Journal ref Information Fusion 51 (2019) 259-270

1902.09487 2019-02-26 cs.CV cs.AI cs.CL cs.LG 82%

MUREL: Multimodal Relational Reasoning for Visual Question Answering

Remi Cadene, Hedi Ben-younes, Matthieu Cord, Nicolas Thome

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments CVPR2019 accepted paper

1802.05335 2018-11-13 cs.LG stat.ML 82%

Multimodal Generative Models for Scalable Weakly-Supervised Learning

Mike Wu, Noah Goodman

专题命中多模态训练与对齐：multimodal(title,abstract)；multi-modal(abstract)

Comments To appear at NIPS 2018; 9 pages with supplement

1807.01332 2018-07-05 cs.LG stat.ML 82%

Multi-Level Feature Abstraction from Convolutional Neural Networks for Multimodal Biometric Identification

Sobhan Soleymani, Ali Dabouei, Hadi Kazemi, Jeremy Dawson, Nasser M. Nasrabadi

专题命中多模态训练与对齐：multimodal(title,abstract)；multi-modal(abstract)

Comments Accepted in "2018 International Conference on Pattern Recognition"

1802.02892 2018-02-09 cs.CL cs.AI cs.CV 82%

Efficient Large-Scale Multi-Modal Classification

D. Kiela, E. Grave, A. Joulin, T. Mikolov

专题命中多模态训练与对齐：multi-modal(title,abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Published at AAAI-18, 7 pages

1711.03483 2017-11-10 cs.CL cs.AI cs.CV 82%

Learning Multi-Modal Word Representation Grounded in Visual Context

Éloi Zablocki, Benjamin Piwowarski, Laure Soulier, Patrick Gallinari

专题命中多模态训练与对齐：multi-modal(title)；multimodal(abstract)；分类 cs.CV、cs.CL、cs.AI

2607.28590 2026-07-31 cs.CV cs.CL 新提交 82%

VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

VAD：为多模态在线策略蒸馏中的目标重建归因视觉证据

Kangning Zhang, Yixing Li, Shuai Shao, Qingyao Li, Zhengxi Lu, Zhiyuan Yao, Jianghao Lin, Wenxiang Jiao, Yuan Lu, Weiwen Liu, Weinan Zhang, Yong Yu

机构 * Shanghai Jiao Tong University（上海交通大学）； Xiaohongshu Inc.（小红书公司）； The Chinese University of Hong Kong（香港中文大学）； Zhejiang University（浙江大学）； Southeast University（东南大学）

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CV、cs.CL

AI总结该研究提出视觉归因蒸馏（VAD）算法，通过反事实目标重建分离教师校正中的视觉证据分量，在6个4B/9B规模的细粒度视觉基准上，性能优于现有蒸馏方法。

Comments The project is accessible at https://github.com/DeepExperience/VAD_Multimodal_OPD

2607.14682 2026-07-17 cs.AI cs.CL cs.LG 新提交 82%

Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment

停止思考，开始观察：通过无推理对齐实现多模态文档问答的高效训练后优化

Harikrishnan P M, Goutham Vignesh, Ganesh Parab, Saisubramaniam Gopalakrishnan, Vishal Vaddina, Varun V, Rohit Agrawal

专题命中多模态训练与对齐：multimodal(title,abstract)；分类 cs.CL、cs.AI

AI总结研究多模态文档问答中高效训练后优化问题，提出感知 - RFT 框架，用组相对策略优化绕过推理令牌直接对齐视觉与基础输出，通过构建变体评估推理必要性，发现启用推理模型有优势，还识别基础差异，表明早期转换可减少训练数据并保持精度。

Comments Accepted at ICML 2026, Workshop on Efficient Multimodal Question Answering (EMM-QA)

1. 多模态训练与对齐 6903 篇

Boosting Multi-Modal E-commerce Attribute Value Extraction via Unified Learning Scheme and Dynamic Range Minimization

Joint Multimodal Entity-Relation Extraction Based on Edge-enhanced Graph Alignment Network and Word-pair Relation Tagging

Team Triple-Check at Factify 2: Parameter-Efficient Large Foundation Models with Feature Representations for Multi-Modal Fact Verification

Rumor Classification through a Multimodal Fusion Framework and Ensemble Learning

Depression Diagnosis and Analysis via Multimodal Multi-order Factor Fusion

Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture

MNER-QG: An End-to-End MRC framework for Multimodal Named Entity Recognition with Query Grounding

Multi-Modal Representation Learning with Self-Adaptive Threshold for Commodity Verification

Hybrid Multimodal Fusion for Humor Detection

Multi-modal Contrastive Representation Learning for Entity Alignment

Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysis

Multimodal Machine Learning in Precision Health

Hierarchical Cross-Modality Semantic Correlation Learning Model for Multimodal Summarization

Active Speaker Detection as a Multi-Objective Optimization with Uncertainty-based Multimodal Fusion

Knowledge Perceived Multi-modal Pretraining in E-commerce

Cross-Modal Knowledge Transfer via Inter-Modal Translation and Alignment for Affect Recognition

A Discriminative Vectorial Framework for Multi-modal Feature Representation

Adaptive Fusion Techniques for Multimodal Data

Robust Multi-Modal Sensor Fusion: An Adversarial Approach

Multimodal Intelligence: Representation Learning, Information Fusion, and Applications

Fusion-supervised Deep Cross-modal Hashing

End-to-End Entity Classification on Multimodal Knowledge Graphs

EmbraceNet: A robust deep learning architecture for multimodal classification

MUREL: Multimodal Relational Reasoning for Visual Question Answering

Multimodal Generative Models for Scalable Weakly-Supervised Learning

Multi-Level Feature Abstraction from Convolutional Neural Networks for Multimodal Biometric Identification

Efficient Large-Scale Multi-Modal Classification

Learning Multi-Modal Word Representation Grounded in Visual Context

VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment