多模态信息融合 - arXivDaily 专题

2505.07895 2025-06-23 cs.LG cs.AI 50%

Representation Learning with Mutual Influence of Modalities for Node Classification in Multi-Modal Heterogeneous Networks

Jiafan Li, Jiaqi Zhu, Liang Chang, Yilin Li, Miaomiao Li, Yang Wang, Hongan Wang

机构 * Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）； University of Chinese Academy of Sciences（中国科学院大学）； School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）； Binzhou Institute of Technology（滨州职业技术学院）

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

2505.10176 2025-05-16 cs.NE 50%

Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence

Xiang He, Dongcheng Zhao, Yang Li, Qingqun Kong, Xin Yang, Yi Zeng

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments The manuscript is under review and the code is available at https://github.com/Brain-Cog-Lab/IEMF

2505.00422 2025-05-02 cs.LG cs.CL 50%

Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training

Yu Han, Aaron Ceross, Jeroen H. M. Bergmann

机构 * Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford（生物医学工程研究所，工程科学系，牛津大学）； Birmingham Law School, University of Birmingham（伯明翰法学院，伯明翰大学）

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2504.20447 2025-04-30 cs.SD cs.AI eess.AS 50%

APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech

Zhicheng Lian, Lizhi Wang, Hua Huang

机构 * Beijing Normal University（北京师范大学）

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2504.04789 2025-04-08 cs.AI 50%

Multimodal Agricultural Agent Architecture (MA3): A New Paradigm for Intelligent Agricultural Decision-Making

Zhuoning Xu, Jian Xu, Mingqing Zhang, Peijie Wang, Chao Deng, Cheng-Lin Liu

专题命中音视频/视觉语言融合：information fusion(abstract)

2503.06405 2025-04-02 cs.SD cs.AI eess.AS 50%

Heterogeneous bimodal attention fusion for speech emotion recognition

Jiachen Luo, Huy Phan, Lin Wang, Joshua Reiss

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

2503.23721 2025-04-01 cs.LG cs.AI 50%

Unimodal-driven Distillation in Multimodal Emotion Recognition with Dynamic Fusion

Jiagen Li, Rui Yu, Huihao Huang, Huaicheng Yan

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2503.22687 2025-04-01 eess.AS cs.AI 50%

Qieemo: Speech Is All You Need in the Emotion Recognition in Conversations

Jinming Chen, Jingyi Fang, Yuanzhong Zheng, Yaoxuan Wang, Haojun Fei

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2503.06108 2025-03-11 cs.SD cs.AI 50%

Multi-modal expressive personality recognition in data non-ideal audiovisual based on multi-scale feature enhancement and modal augment

Weixuan Kong, Jinpeng Yu, Zijun Li, Hanwei Liu, Jiqing Qu, Hui Xiao, Xuefeng Li

专题命中音视频/视觉语言融合：feature-level fusion(abstract)

2502.18481 2025-02-27 cs.IR cs.AI 50%

MDE: Modality Discrimination Enhancement for Multi-modal Recommendation

Hang Zhou, Yucheng Wang, Huijing Zhan

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

2412.09928 2025-02-19 cs.SD eess.AS 50%

Leveraging Multimodal Methods and Spontaneous Speech for Alzheimer's Disease Identification

Yifan Gao, Long Guo, Hong Liu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments ICASSP 2025

2501.00778 2025-01-03 cs.CL cs.CY 50%

Decoding the Flow: CauseMotion for Emotional Causality Analysis in Long-form Conversations

Yuxuan Zhang, Yulong Li, Zichen Yu, Feilong Tang, Zhixiang Lu, Chong Li, Kang Dang, Jionglong Su

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments 7pages

2412.16444 2024-12-24 cs.CL cs.LG 50%

Effective Context Modeling Framework for Emotion Recognition in Conversations

Cuong Tran Van, Thanh V. T. Tran, Van Nguyen, Truong Son Hy

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2412.16243 2024-12-24 cs.LG 50%

Bag of Tricks for Multimodal AutoML with Image, Text, and Tabular Data

Zhiqiang Tang, Zihan Zhong, Tong He, Gerald Friedland

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2410.06543 2024-10-10 cs.CR cs.SD eess.AS 50%

Gumbel Rao Monte Carlo based Bi-Modal Neural Architecture Search for Audio-Visual Deepfake Detection

Aravinda Reddy PN, Raghavendra Ramachandra, Krothapalli Sreenivasa Rao, Pabitra Mitra Vinod Rathod

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

2408.14964 2024-08-28 cs.LG 50%

Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning

Sakhinana Sagar Srinivas, Venkataramana Runkana

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

Comments Paper Accepted at Workshop on Robustness of Few-shot and Zero-shot Learning in Foundation Models at NeurIPS 2023

2407.03026 2024-07-04 cs.SD cs.AI eess.AS 50%

Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition

Jinming Chen, Jingyi Fang, Yuanzhong Zheng, Yaoxuan Wang, Haojun Fei

专题命中音视频/视觉语言融合：information fusion(abstract)

Comments accpeted by interspeech 2014, 5 pages, 1 figure

2407.02005 2024-07-03 cs.CL cs.SD eess.AS 50%

An End-to-End Speech Summarization Using Large Language Model

Hengchao Shang, Zongyao Li, Jiaxin Guo, Shaojun Li, Zhiqiang Rao, Yuanchang Luo, Daimeng Wei, Hao Yang

专题命中音视频/视觉语言融合：information fusion(abstract)

Comments InterSpeech 2024

2404.01958 2024-04-03 cs.LG 50%

MESEN: Exploit Multimodal Data to Design Unimodal Human Activity Recognition with Few Labels

Lilin Xu, Chaojie Gu, Rui Tan, Shibo He, Jiming Chen

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments Accepted to the 21th ACM Conference on Embedded Networked Sensor Systems (SenSys 2023)

2312.11974 2023-12-29 cs.SD cs.HC eess.AS 50%

Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks

Mengbo Li, Yuanzhong Zheng, Dichucheng Li, Yulun Wu, Yaoxuan Wang, Haojun Fei

专题命中音视频/视觉语言融合：information fusion(abstract)

2312.01568 2023-12-05 cs.HC cs.SD eess.AS 50%

Multimodal Speech Emotion Recognition Using Modality-specific Self-Supervised Frameworks

Rutherford Agbeshi Patamia, Paulo E. Santos, Kingsley Nketia Acheampong, Favour Ekong, Kwabena Sarpong, She Kun

专题命中音视频/视觉语言融合：feature-level fusion(abstract)

2302.05040 2023-06-22 cs.CL cs.LG cs.SD eess.AS 50%

PATCorrect: Non-autoregressive Phoneme-augmented Transformer for ASR Error Correction

Ziji Zhang, Zhehui Wang, Rajesh Kamma, Sharanya Eswaran, Narayanan Sadagopan

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

Comments Accepted camera-ready version for INTERSPEECH 2023

2306.07115 2023-06-13 cs.CL cs.SD eess.AS 50%

Exploring Attention Mechanisms for Multimodal Emotion Recognition in an Emergency Call Center Corpus

Théo Deschamps-Berger, Lori Lamel, Laurence Devillers

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments 5 pages, 2 figures, 4 tables

Journal ref Published in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

2303.06341 2023-03-14 eess.AS 50%

The NPU-ASLP System for Audio-Visual Speech Recognition in MISP 2022 Challenge

Pengcheng Guo, He Wang, Bingshen Mu, Ao Zhang, Peikun Chen

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

Comments 2 pages, accepted by ICASSP 2023

2303.03177 2023-03-07 eess.AS cs.CL cs.LG cs.SD 50%

Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis

Vikramjit Mitra, Vasudha Kowtha, Hsiang-Yun Sherry Chien, Erdrin Azemi, Carlos Avendano

专题命中音视频/视觉语言融合：multi-modal fusion(abstract)

Comments 5 pages, conference

2209.04355 2023-02-09 cs.AI 50%

MIntRec: A New Dataset for Multimodal Intent Recognition

Hanlei Zhang, Hua Xu, Xin Wang, Qianrui Zhou, Shaojie Zhao, Jiayan Teng

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments Accepted by ACM MM 2022 (Main Track, Long Paper)

2204.10196 2022-12-22 cs.CL cs.AI 50%

Multimodal Hate Speech Detection from Bengali Memes and Texts

Md. Rezaul Karim, Sumon Kanti Dey, Tanhim Islam, Md. Shajalal, Bharathi Raja Chakravarthi

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments arXiv admin note: text overlap with arXiv:2107.00648 by other authors

Journal ref Pre-print for our paper at International Conference on Speech & Language Technology for Low-resource Languages (SPELLL'2022)

2210.17444 2022-12-06 cs.LG 50%

Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations

Sijie Mai, Ying Zeng, Haifeng Hu

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Comments This paper is accepted by IEEE Transactions on Multimedia. This version addresses some mistakes and typos in the original paper. The appendix is available at https://github.com/TmacMai/Multimodal-Information-Bottleneck/blob/main/appendix.pdf

2211.03539 2022-11-08 cs.HC cs.AI 50%

Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces

Amr Gomaa

专题命中音视频/视觉语言融合：multimodal fusion(abstract)

Journal ref In Proceedings of the 2022 International Conference on Multimodal Interaction, pp. 690-695. 2022

2210.05254 2022-10-12 cs.SD cs.AI eess.AS 50%

Deep Spectro-temporal Artifacts for Detecting Synthesized Speech

Xiaohui Liu, Meng Liu, Lin Zhang, Linjuan Zhang, Chang Zeng, Kai Li, Nan Li, Kong Aik Lee, Longbiao Wang, Jianwu Dang

专题命中音视频/视觉语言融合：information fusion(abstract)

Comments 7 pages, 1 figures, Accecpted by Proceedings of the 1st International Workshop on Deepfake Detection for Audio Multimedia

1. 音视频/视觉语言融合 498 篇

Representation Learning with Mutual Influence of Modalities for Node Classification in Multi-Modal Heterogeneous Networks

Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence

Toward Automated Regulatory Decision-Making: Trustworthy Medical Device Risk Classification with Multimodal Transformers and Self-Training

APG-MOS: Auditory Perception Guided-MOS Predictor for Synthetic Speech

Multimodal Agricultural Agent Architecture (MA3): A New Paradigm for Intelligent Agricultural Decision-Making

Heterogeneous bimodal attention fusion for speech emotion recognition

Unimodal-driven Distillation in Multimodal Emotion Recognition with Dynamic Fusion

Qieemo: Speech Is All You Need in the Emotion Recognition in Conversations

Multi-modal expressive personality recognition in data non-ideal audiovisual based on multi-scale feature enhancement and modal augment

MDE: Modality Discrimination Enhancement for Multi-modal Recommendation

Leveraging Multimodal Methods and Spontaneous Speech for Alzheimer's Disease Identification

Decoding the Flow: CauseMotion for Emotional Causality Analysis in Long-form Conversations

Effective Context Modeling Framework for Emotion Recognition in Conversations

Bag of Tricks for Multimodal AutoML with Image, Text, and Tabular Data

Gumbel Rao Monte Carlo based Bi-Modal Neural Architecture Search for Audio-Visual Deepfake Detection

Cross-Modal Learning for Chemistry Property Prediction: Large Language Models Meet Graph Machine Learning

Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition

An End-to-End Speech Summarization Using Large Language Model

MESEN: Exploit Multimodal Data to Design Unimodal Human Activity Recognition with Few Labels

Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks

Multimodal Speech Emotion Recognition Using Modality-specific Self-Supervised Frameworks

PATCorrect: Non-autoregressive Phoneme-augmented Transformer for ASR Error Correction

Exploring Attention Mechanisms for Multimodal Emotion Recognition in an Emergency Call Center Corpus

The NPU-ASLP System for Audio-Visual Speech Recognition in MISP 2022 Challenge

Pre-trained Model Representations and their Robustness against Noise for Speech Emotion Analysis

MIntRec: A New Dataset for Multimodal Intent Recognition

Multimodal Hate Speech Detection from Bengali Memes and Texts

Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations

Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive Interfaces

Deep Spectro-temporal Artifacts for Detecting Synthesized Speech