多模态大模型

DeepStroke: An Efficient Stroke Screening Framework for Emergency Rooms with Multimodal Adversarial Deep Learning

Tongan Cai, Haomiao Ni, Mingli Yu, Xiaolei Huang, Kelvin Wong, John Volpi, James Z. Wang, Stephen T. C. Wong

专题命中 音频语音多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

CI-AVSR: A Cantonese Audio-Visual Speech Dataset for In-car Command Recognition

Wenliang Dai, Samuel Cahyawijaya, Tiezheng Yu, Elham J. Barezi, Peng Xu, Cheuk Tung Shadow Yiu, Rita Frieske, Holy Lovenia, Genta Indra Winata, Qifeng Chen, Xiaojuan Ma, Bertram E. Shi, Pascale Fung

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CL、cs.AI

Comments 6 pages

Visual Sound Localization in the Wild by Cross-Modal Interference Erasing

Xian Liu, Rui Qian, Hang Zhou, Di Hu, Weiyao Lin, Ziwei Liu, Bolei Zhou, Xiaowei Zhou

专题命中 音频语音多模态 ：cross-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV、eess.AS

Comments Accepted by AAAI Conference on Artificial Intelligence (AAAI) 2022. 16 pages

Dynamical Audio-Visual Navigation: Catching Unheard Moving Sound Sources in Unmapped 3D Environments

Abdelrahman Younes

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV、eess.AS

Information Fusion in Attention Networks Using Adaptive and Multi-level Factorized Bilinear Pooling for Audio-visual Emotion Recognition

Hengshun Zhou, Jun Du, Yuanyuan Zhang, Qing Wang, Qing-Feng Liu, Chin-Hui Lee

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.MM、eess.AS

The MuSe 2021 Multimodal Sentiment Analysis Challenge: Sentiment, Emotion, Physiological-Emotion, and Stress

Lukas Stappen, Alice Baird, Lukas Christ, Lea Schumann, Benjamin Sertolli, Eva-Maria Messner, Erik Cambria, Guoying Zhao, Björn W. Schuller

专题命中 音频语音多模态 ：multimodal(title,abstract)；audio-visual(abstract)；分类 cs.CL、cs.AI

3D-MOV: Audio-Visual LSTM Autoencoder for 3D Reconstruction of Multiple Objects from Video

Justin Wilson, Ming C. Lin

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、eess.AS

Leveraging Recent Advances in Deep Learning for Audio-Visual Emotion Recognition

Liam Schoneveld, Alice Othmani, Hazem Abdelkawy

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV、eess.AS

Comments 8 pages, 3 figures, Pattern Recognition Letters

Multi-modal Residual Perceptron Network for Audio-Video Emotion Recognition

Xin Chang, Władysław Skarbek

专题命中 音频语音多模态 ：multi-modal(title,abstract)；audio-visual(abstract)；分类 cs.AI、eess.AS

Multi-level Attention Fusion Network for Audio-visual Event Recognition

Mathilde Brousmiche, Jean Rouat, Stéphane Dupont

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.MM

Comments Preprint submitted to the Information Fusion journal in August 2020

Where and When: Space-Time Attention for Audio-Visual Explanations

Yanbei Chen, Thomas Hummel, A. Sophia Koepke, Zeynep Akata

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.AI

Tie Your Embeddings Down: Cross-Modal Latent Spaces for End-to-end Spoken Language Understanding

Bhuvan Agrawal, Markus Müller, Martin Radfar, Samridhi Choudhary, Athanasios Mouchtaris, Siegfried Kunzmann

专题命中 音频语音多模态 ：cross-modal(title,abstract)；multi-modal(abstract)；分类 cs.CL、eess.AS

Comments 7 pages, 6 figures

Seeing voices and hearing voices: learning discriminative embeddings using cross-modal self-supervision

Soo-Whan Chung, Hong Goo Kang, Joon Son Chung

专题命中 音频语音多模态 ：cross-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV、eess.AS

Comments Under submission as a conference paper

Rule-embedded network for audio-visual voice activity detection in live musical video streams

Yuanbo Hou, Yi Deng, Bilei Zhu, Zejun Ma, Dick Botteldooren

专题命中 音频语音多模态 ：audio-visual(title,abstract)；cross-modal(abstract)；分类 cs.MM、eess.AS

Comments Submitted to ICASSP 2021

Themes Informed Audio-visual Correspondence Learning

Runze Su, Fei Tao, Xudong Liu, Haoran Wei, Xiaorong Mei, Zhiyao Duan, Lei Yuan, Ji Liu, Yuying Xie

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.AI、cs.MM

Comments Submitting to ICASSP 2021

SoundSpaces: Audio-Visual Navigation in 3D Environments

Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted to ECCV 2020 (Spotlight). Project page: http://vision.cs.utexas.edu/projects/audio_visual_navigation/

Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention

Bin Duan, Hao Tang, Wei Wang, Ziliang Zong, Guowei Yang, Yan Yan

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、eess.AS

Automatic Quality Assessment for Audio-Visual Verification Systems. The LOVe submission to NIST SRE Challenge 2019

Grigory Antipov, Nicolas Gengembre, Olivier Le Blouch, Gaël Le Lan

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.MM、eess.AS

Comments 5 pages, 1 figure, accepted at INTERSPEECH 2020. Corrected the reference [20]

Temporal aggregation of audio-visual modalities for emotion recognition

Andreea Birhala, Catalin Nicolae Ristea, Anamaria Radoi, Liviu Cristian Dutu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.AI

Disentangled Speech Embeddings using Cross-modal Self-supervision

Arsha Nagrani, Joon Son Chung, Samuel Albanie, Andrew Zisserman

专题命中 音频语音多模态 ：cross-modal(title,abstract)；audio-visual(abstract)；分类 cs.CV、eess.AS

Comments ICASSP 2020. The first three authors contributed equally to this work

Look, Listen, and Act: Towards Audio-Visual Embodied Navigation

Chuang Gan, Yiwei Zhang, Jiajun Wu, Boqing Gong, Joshua B. Tenenbaum

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted by ICRA 2020. Project page: http://avn.csail.mit.edu

Audio-Visual Model Distillation Using Acoustic Images

Andrés F. Pérez, Valentina Sanguineti, Pietro Morerio, Vittorio Murino

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、eess.AS

Comments Accepted at WACV 2020; supplementary material at page 11; code available at https://github.com/afperezm/acoustic-images-distillation

Listening while Speaking and Visualizing: Improving ASR through Multimodal Chain

Johanes Effendi, Andros Tjandra, Sakriani Sakti, Satoshi Nakamura

专题命中 音频语音多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted in IEEE ASRU 2019

Contextual Audio-Visual Switching For Speech Enhancement in Real-World Environments

Ahsan Adeel, Mandar Gogate, Amir Hussain

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、eess.AS

Comments 16 pages, 7 figures. arXiv admin note: substantial text overlap with arXiv:1808.00046

Journal ref Information Fusion, 2019

Weakly Supervised Representation Learning for Unsynchronized Audio-Visual Events

Sanjeel Parekh, Slim Essid, Alexey Ozerov, Ngoc Q. K. Duong, Patrick Pérez, Gaël Richard

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、eess.AS

Multimodal Utterance-level Affect Analysis using Visual, Audio and Text Features

Didan Deng, Yuqian Zhou, Jimin Pi, Bertram E. Shi

专题命中 音频语音多模态 ：multimodal(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 5 pages, 1 figure, subject to the 2018 IJCNN challenge on One-Minute Gradual-Emotion Recognition

Do Visual Features Improve Other-Initiated Repair Detection? A Dyadic Multimodal Approach

视觉特征能否改善他人发起的修复检测？一种二元多模态方法

Anh Ngo, Nicolas Rollet, Catherine Pelachaud, Chloé Clavel

机构 * INRIA Paris（法国国家信息与自动化研究所巴黎分院） ； ISIR, Sorbonne University Paris（巴黎索邦大学信息学、系统与机器人研究所） ； Télécom Paris, SES, Institut Polytechnique de Paris, I3-CNRS Paris（巴黎电信学院、巴黎高等理工学院SES系、巴黎综合理工学院I3 - 法国国家科学研究中心） ； CNRS（法国国家科学研究中心） ； LTCI, Institut Polytechnique de Paris（巴黎综合理工学院LTCI实验室） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；cross-modal(abstract)；分类 cs.AI

AI总结 研究他人发起的修复检测问题，提出结合视觉特征的二元多模态模型，通过在两个语料库上评估，证明视觉信息能提升检测性能，提供跨模态特征贡献见解。

Comments Accepted to ICMI 2026 (International Conference on Multimodal Interaction), October 5-9, 2026, Napoli, Italy

Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs

Jingfei Xia, Mingchen Zhuge, Tiantian Geng, Shun Fan, Yuantai Wei, Zhenyu He, Feng Zheng

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV

Comments Our code is available at https://github.com/AndyFrancesco29/Audio-Visual-Figure-Skating

Audio Visual Language Maps for Robot Navigation

Chenguang Huang, Oier Mees, Andy Zeng, Wolfram Burgard

专题命中 音频语音多模态 ：multimodal(abstract)；cross-modal(abstract)；audio-visual(abstract)；multimodal foundation model(abstract)

Comments Project page: https://avlmaps.github.io/

Generalizing Video DeepFake Detection by Self-generated Audio-Visual Pseudo-Fakes

通过自动生成的音频视觉伪假来泛化视频深度伪造检测

Zihe Wei, Yuezun Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；cross-modal(abstract)；分类 cs.MM

AI总结 本文提出AVPF方法，通过自动生成多样化的音频视觉伪假样本提升模型泛化能力，实验显示在多个标准数据集上平均性能提升达7.4%。