多模态大模型

Exploring Modality Disruption in Multimodal Fake News Detection

Moyang Liu, Kaiying Yan, Yukun Liu, Ruibo Fu, Zhengqi Wen, Xuefei Liu, Chenxing Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation

Zhihua Xu, Tianshui Chen, Zhijing Yang, Siyuan Peng, Keze Wang, Liang Lin

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted at TMM 2025

Generating Diverse Audio-Visual 360 Soundscapes for Sound Event Localization and Detection

Adrian S. Roman, Aiden Chang, Gerardo Meza, Iran R. Roman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

SAV-SE: Scene-aware Audio-Visual Speech Enhancement with Selective State Space Model

Xinyuan Qian, Jiaran Gao, Yaodan Zhang, Qiquan Zhang, Hexin Liu, Leibny Paola Garcia, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV、cs.AI、cs.MM

Comments accepted by IEEE Journal of Selected Topics in Signal Processing

Beyond Text-to-Text: An Overview of Multimodal and Generative Artificial Intelligence for Education Using Topic Modeling

Ville Heilala, Roberto Araya, Raija Hämäläinen

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Journal ref Proceedings of the 40th ACM/SIGAPP Symposium on Applied Computing (SAC'25), March 31--April 4, 2025, Catania, Italy

Extremely Simple Out-of-distribution Detection for Audio-visual Generalized Zero-shot Learning

Yang Liu, Xun Zhang, Jiale Du, Xinbo Gao, Jungong Han

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Feature-Based Dual Visual Feature Extraction Model for Compound Multimodal Emotion Recognition

Ran Liu, Fengyu Zhang, Cong Yu, Longjiang Yang, Zhuofan Wen, Siyuan Zhang, Hailiang Yao, Shun Chen, Zheng Lian, Bin Liu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, Hao Fei

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments Survey, working under progress; 12 figures, 4 tables, 44 pages; Resource at https://github.com/yaotingwangofficial/Awesome-MCoT

An Audio-Visual Fusion Emotion Generation Model Based on Neuroanatomical Alignment

Haidong Wang, Qia Shan, JianHua Zhang, PengFei Xiao, Ao Liu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI

Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation

Henghui Du, Guangyao Li, Chang Zhou, Chunjie Zhang, Alan Zhao, Di Hu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

cantnlp@DravidianLangTech2025: A Bag-of-Sounds Approach to Multimodal Hate Speech Detection

Sidney Wong, Andrew Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted Fifth Workshop on Speech and Language Technologies for Dravidian Languages

TRACE: Real-Time Multimodal Common Ground Tracking in Situated Collaborative Dialogues

Hannah VanderHoeven, Brady Bhalla, Ibrahim Khebour, Austin Youngren, Videep Venkatesha, Mariah Bradford, Jack Fitzgerald, Carlos Mabrey, Jingxuan Tu, Yifan Zhu, Kenneth Lai, Changsoo Jung, James Pustejovsky, Nikhil Krishnaswamy

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 11 pages, 4 tables, 4 figures, to appear at NAACL 2025 Demos program, Albuquerque, NM, USA

Prompt2LVideos: Exploring Prompts for Understanding Long-Form Multimodal Videos

Soumya Shamarao Jahagirdar, Jayasree Saha, C V Jawahar

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments CVIP 2024

R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning

Jiaxing Zhao, Xihan Wei, Liefeng Bo

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multimodal Machine Learning Can Predict Videoconference Fluidity and Enjoyment

Andrew Chang, Viswadruth Akkaraju, Ray McFadden Cogliano, David Poeppel, Dustin Freeman

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments ICASSP 2025

Enhancing Video Music Recommendation with Transformer-Driven Audio-Visual Embeddings

Shimiao Liu, Alexander Lerch

专题命中 音频语音多模态 ：audio-visual(title)；cross-modal(abstract)；分类 cs.MM

Comments 2024 IEEE 5th International Symposium on the Internet of Sounds (IS2), Erlangen, Germany, 2024, pp. 1-6

Audio-Visual Target Speaker Extraction with Reverse Selective Auditory Attention

Ruijie Tao, Xinyuan Qian, Yidi Jiang, Junjie Li, Jiadong Wang, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

A Survey on Multimodal Recommender Systems: Recent Advances and Future Directions

Jinfeng Xu, Zheyu Chen, Shuo Yang, Jinze Li, Wei Wang, Xiping Hu, Steven Hoi, Edith Ngai

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

AV-Flow: Transforming Text to Audio-Visual Human-like Interactions

Aggelina Chatziagapi, Louis-Philippe Morency, Hongyu Gong, Michael Zollhoefer, Dimitris Samaras, Alexander Richard

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Mind the Gap: Aligning the Brain with Language Models Requires a Nonlinear and Multimodal Approach

Danny Dongyeop Han, Yunju Cho, Jiook Cha, Jay-Yoon Lee

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Leveraging Multimodal Methods and Spontaneous Speech for Alzheimer's Disease Identification

Yifan Gao, Long Guo, Hong Liu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments ICASSP 2025

Predicting Cognitive Decline: A Multimodal AI Approach to Dementia Screening from Speech

Lei Chi, Arav Sharma, Ari Gebhardt, Joseph T. Colonel

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Submitted to IEEE ICAD 2025

Taking the Next Step with Generative Artificial Intelligence: The Transformative Role of Multimodal Large Language Models in Science Education

Arne Bewersdorff, Christian Hartmann, Marie Hornberger, Kathrin Seßler, Maria Bannert, Enkelejda Kasneci, Gjergji Kasneci, Xiaoming Zhai, Claudia Nerdel

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments revised version 2. September 2024

M3PT: A Transformer for Multimodal, Multi-Party Social Signal Prediction with Person-aware Blockwise Attention

Yiming Tang, Abrar Anwar, Jesse Thomason

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

EmoTech: A Multi-modal Speech Emotion Recognition Using Multi-source Low-level Information with Hybrid Recurrent Network

Shamin Bin Habib Avro, Taieba Taher, Nursadul Mamun

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 eess.AS

LLM supervised Pre-training for Multimodal Emotion Recognition in Conversations

Soumya Dutta, Sriram Ganapathy

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments ICASSP 2025; 5 pages, 4 figures, 2 tables

Multimodal Marvels of Deep Learning in Medical Diagnosis: A Comprehensive Review of COVID-19 Detection

Md Shofiqul Islam, Khondokar Fida Hasan, Hasibul Hossain Shajeeb, Humayan Kabir Rana, Md Saifur Rahmand, Md Munirul Hasan, AKM Azad, Ibrahim Abdullah, Mohammad Ali Moni

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments 43 pages

Neural Speech Tracking in a Virtual Acoustic Environment: Audio-Visual Benefit for Unscripted Continuous Speech

Mareike Daeglau, Juergen Otten, Giso Grimm, Bojana Mirkovic, Volker Hohmann, Stefan Debener

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Multi-modal Speech Enhancement with Limited Electromyography Channels

Fuyuan Feng, Longting Xu, Rohan Kumar Das

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments Accepted by ICASSP 2025

DCIM-AVSR : Efficient Audio-Visual Speech Recognition via Dual Conformer Interaction Module

Xinyu Wang, Haotian Jiang, Haolin Huang, Yu Fang, Mengjie Xu, Qian Wang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted to ICASSP 2025