多模态大模型

Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection

Jiehui Jia, Huan Zhang, Jinhua Liang

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Grounding Task Assistance with Multimodal Cues from a Single Demonstration

Gabriel Sarch, Balasaravanan Thoravi Kumaravel, Sahithya Ravi, Vibhav Vineet, Andrew D. Wilson

机构 * Microsoft Research（微软研究院） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

DGFNet: End-to-End Audio-Visual Source Separation Based on Dynamic Gating Fusion

Yinfeng Yu, Shiyu Sun

机构 * School of Computer Science and Technology（计算机科学与技术学院） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI

Comments Main paper (9 pages). Accepted for publication by ICMR(International Conference on Multimedia Retrieval) 2025

Uncovering the Visual Contribution in Audio-Visual Speech Recognition

Zhaofeng Lin, Naomi Harte

机构 * Sigmedia Group, School of Engineering Trinity College Dublin（Sigmedia集团，工程学院，三一学院都柏林） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments 5 pages, 2 figures. Accepted to ICASSP 2025

Exploring Modality Disruption in Multimodal Fake News Detection

Moyang Liu, Kaiying Yan, Yukun Liu, Ruibo Fu, Zhengqi Wen, Xuefei Liu, Chenxing Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation

Zhihua Xu, Tianshui Chen, Zhijing Yang, Siyuan Peng, Keze Wang, Liang Lin

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted at TMM 2025

Generating Diverse Audio-Visual 360 Soundscapes for Sound Event Localization and Detection

Adrian S. Roman, Aiden Chang, Gerardo Meza, Iran R. Roman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model

Xinyuan Qian, Jiaran Gao, Yaodan Zhang, Qiquan Zhang, Hexin Liu, Leibny Paola Garcia, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV、cs.AI、cs.MM

Comments accepted by IEEE Journal of Selected Topics in Signal Processing

Beyond Text-to-Text: An Overview of Multimodal and Generative Artificial Intelligence for Education Using Topic Modeling

Ville Heilala, Roberto Araya, Raija Hämäläinen

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Journal ref Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing (SAC'25), March 31--April 4, 2025, Catania, Italy

Extremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning

Yang Liu, Xun Zhang, Jiale Du, Xinbo Gao, Jungong Han

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Feature-Based Dual Visual Feature Extraction Model for Compound Multimodal Emotion Recognition

Ran Liu, Fengyu Zhang, Cong Yu, Longjiang Yang, Zhuofan Wen, Siyuan Zhang, Hailiang Yao, Shun Chen, Zheng Lian, Bin Liu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, Hao Fei

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Survey, working under progress; 12 figures, 4 tables, 44 pages; Resource at https://github.com/yaotingwangofficial/Awesome-MCoT

An Audio-Visual Fusion Emotion Generation Model Based on Neuroanatomical Alignment

Haidong Wang, Qia Shan, JianHua Zhang, PengFei Xiao, Ao Liu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI

Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation

Henghui Du, Guangyao Li, Chang Zhou, Chunjie Zhang, Alan Zhao, Di Hu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

cantnlp@DravidianLangTech2025: A Bag-of-Sounds Approach to Multimodal Hate Speech Detection

Sidney Wong, Andrew Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted Fifth Workshop on Speech and Language Technologies for Dravidian Languages

TRACE: Real-Time Multimodal Common Ground Tracking in Situated Collaborative Dialogues

Hannah VanderHoeven, Brady Bhalla, Ibrahim Khebour, Austin Youngren, Videep Venkatesha, Mariah Bradford, Jack Fitzgerald, Carlos Mabrey, Jingxuan Tu, Yifan Zhu, Kenneth Lai, Changsoo Jung, James Pustejovsky, Nikhil Krishnaswamy

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 11 pages, 4 tables, 4 figures, to appear at NAACL 2025 Demos program, Albuquerque, NM, USA

Prompt2LVideos: Exploring Prompts for Understanding Long-Form Multimodal Videos

Soumya Shamarao Jahagirdar, Jayasree Saha, C V Jawahar

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments CVIP 2024

R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning

Jiaxing Zhao, Xihan Wei, Liefeng Bo

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Machine Learning Can Predict Videoconference Fluidity and Enjoyment

Andrew Chang, Viswadruth Akkaraju, Ray McFadden Cogliano, David Poeppel, Dustin Freeman

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments ICASSP 2025

Enhancing Video Music Recommendation with Transformer-Driven Audio-Visual Embeddings

Shimiao Liu, Alexander Lerch

专题命中 音频语音多模态 ：audio-visual(title)；cross-modal(abstract)；分类 cs.MM

Comments 2024 IEEE 5th International Symposium on the Internet of Sounds (IS2), Erlangen, Germany, 2024, pp. 1-6

Audio-Visual Target Speaker Extraction with Reverse Selective Auditory Attention

Ruijie Tao, Xinyuan Qian, Yidi Jiang, Junjie Li, Jiadong Wang, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

A Survey on Multimodal Recommender Systems: Recent Advances and Future Directions

Jinfeng Xu, Zheyu Chen, Shuo Yang, Jinze Li, Wei Wang, Xiping Hu, Steven Hoi, Edith Ngai

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

AV-Flow: Transforming Text to Audio-Visual Human-like Interactions

Aggelina Chatziagapi, Louis-Philippe Morency, Hongyu Gong, Michael Zollhoefer, Dimitris Samaras, Alexander Richard

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Mind the Gap: Aligning the Brain with Language Models Requires a Nonlinear and Multimodal Approach

Danny Dongyeop Han, Yunju Cho, Jiook Cha, Jay-Yoon Lee

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Leveraging Multimodal Methods and Spontaneous Speech for Alzheimer's Disease Identification

Yifan Gao, Long Guo, Hong Liu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments ICASSP 2025

Predicting Cognitive Decline: A Multimodal AI Approach to Dementia Screening from Speech

Lei Chi, Arav Sharma, Ari Gebhardt, Joseph T. Colonel

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Submitted to IEEE ICAD 2025

Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education

Arne Bewersdorff, Christian Hartmann, Marie Hornberger, Kathrin Seßler, Maria Bannert, Enkelejda Kasneci, Gjergji Kasneci, Xiaoming Zhai, Claudia Nerdel

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments revised version 2. September 2024

M3PT: A Transformer for Multimodal, Multi-Party Social Signal Prediction with Person-aware Blockwise Attention

Yiming Tang, Abrar Anwar, Jesse Thomason

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

EmoTech: A Multi-modal Speech Emotion Recognition Using Multi-source Low-level Information with Hybrid Recurrent Network

Shamin Bin Habib Avro, Taieba Taher, Nursadul Mamun

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 eess.AS

LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations

Soumya Dutta, Sriram Ganapathy

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments ICASSP 2025; 5 pages, 4 figures, 2 tables