多模态大模型

Deep Variational Generative Models for Audio-visual Speech Separation

Viet-Nhat Nguyen, Mostafa Sadeghi, Elisa Ricci, Xavier Alameda-Pineda

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted to the 31st IEEE International Workshop on Machine Learning for Signal Processing (MLSP), Oct. 25-28, 2021, Gold Coast, Queensland, Australia

Audio-visual Multi-channel Integration and Recognition of Overlapped Speech

Jianwei Yu, Shi-Xiong Zhang, Bo Wu, Shansong Liu, Shoukang Hu, Mengzhe Geng, Xunying Liu, Helen Meng, Dong Yu

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments TASLP 2021

ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learning

Sangho Lee, Jiwan Chung, Youngjae Yu, Gunhee Kim, Thomas Breuel, Gal Chechik, Yale Song

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Published to ICCV2021

Towards a Generic Multimodal Architecture for Batch and Streaming Big Data Integration

Siham Yousfi, Maryem Rhanoui, Dalila Chiadmi

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Journal ref Journal of Computer Science, Volume 15 No. 1, 2019, 207-220

ViNet: Pushing the limits of Visual Modality for Audio-Visual Saliency Prediction

Samyak Jain, Pradeep Yarlagadda, Shreyank Jyoti, Shyamgopal Karthik, Ramanathan Subramanian, Vineet Gandhi

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Appearing in the proceedings of the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) (camera-ready version)

Deep Music Retrieval for Fine-Grained Videos by Exploiting Cross-Modal-Encoded Voice-Overs

Tingtian Li, Zixun Sun, Haoruo Zhang, Jin Li, Ziming Wu, Hui Zhan, Yipeng Yu, Hengcan Shi

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.MM

Comments accepted by ACM SIGIR '21

Multi-Modal Detection of Alzheimer's Disease from Speech and Text

Amish Mittal, Sourav Sahoo, Arnhav Datar, Juned Kadiwala, Hrithwik Shalu, Jimson Mathew

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CL

Comments 9 pages, 3 figures, Accepted in BIOKDD 2021

Is Someone Speaking? Exploring Long-term Temporal Features for Audio-visual Active Speaker Detection

Ruijie Tao, Zexu Pan, Rohan Kumar Das, Xinyuan Qian, Mike Zheng Shou, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments ACM Multimedia 2021

A Biologically Plausible Audio-Visual Integration Model for Continual Learning

Wenjie Chen, Fengtong Du, Ye Wang, Lihong Cao

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI

Comments Accepted by 2021 International Joint Conference on Neural Networks

Software Defined Media: Virtualization of Audio-Visual Services

Manabu Tsukada, Keiko Ogawa, Masahiro Ikeda, Takuro Sone, Kenta Niwa, Shoichiro Saito, Takashi Kasuya, Hideki Sunahara, Hiroshi Esaki

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.MM

Comments IEEE International Conference on Communications (ICC2017), Paris, France, 21-25 May 2017

Audio-Oriented Multimodal Machine Comprehension: Task, Dataset and Model

Zhiqi Huang, Fenglin Liu, Xian Wu, Shen Ge, Helin Wang, Wei Fan, Yuexian Zou

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments AAAI 2021

Cross-Modal Transformer-Based Neural Correction Models for Automatic Speech Recognition

Tomohiro Tanaka, Ryo Masumura, Mana Ihori, Akihiko Takashima, Takafumi Moriya, Takanori Ashihara, Shota Orihashi, Naoki Makishima

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CL

Comments Accepted to Interspeech 2021

Multi-Modal Chorus Recognition for Improving Song Search

Jiaan Wang, Zhixu Li, Binbin Gu, Tingyi Zhang, Qingsheng Liu, Zhigang Chen

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments Accepted at the 30th International Conference on Artificial Neural Networks (ICANN 2021)

Listen to Your Favorite Melodies with img2Mxml, Producing MusicXML from Sheet Music Image by Measure-based Multimodal Deep Learning-driven Assembly

Tomoyuki Shishido, Fehmiju Fati, Daisuke Tokushige, Yasuhiro Ono

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 19 pages, 7 figures

Watching Too Much Television is Good: Self-Supervised Audio-Visual Representation Learning from Movies and TV Shows

Mahdi M. Kalayeh, Nagendra Kamath, Lingyi Liu, Ashok Chandrashekar

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Deep Learning Frameworks Applied For Audio-Visual Scene Classification

Lam Pham, Alexander Schindler, Mina Schütz, Jasmin Lampert, Sven Schlarb, Ross King

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments 6 pages

Cross-Modal Attention Consistency for Video-Audio Unsupervised Learning

Shaobo Min, Qi Dai, Hongtao Xie, Chuang Gan, Yongdong Zhang, Jingdong Wang

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV

S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations

Jheng-hao Lin, Yist Y. Lin, Chung-Ming Chien, Hung-yi Lee

专题命中 音频语音多模态 ：any-to-any(title,abstract)；分类 eess.AS

Comments Accepted by INTERSPEECH 2021

EMA2S: An End-to-End Multimodal Articulatory-to-Speech System

Yu-Wen Chen, Kuo-Hsuan Hung, Shang-Yi Chuang, Jonathan Sherman, Wen-Chin Huang, Xugang Lu, Yu Tsao

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Neural dSCA: demixing multimodal interaction among brain areas during naturalistic experiments

Yu Takagi, Laurence T. Hunt, Ryu Ohata, Hiroshi Imamizu, Jun-ichiro Hirayama

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Multi-modal Point-of-Care Diagnostics for COVID-19 Based On Acoustics and Symptoms

Srikanth Raj Chetupalli, Prashant Krishnan, Neeraj Sharma, Ananya Muguli, Rohit Kumar, Viral Nanda, Lancelot Mark Pinto, Prasanta Kumar Ghosh, Sriram Ganapathy

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments The Manuscript is submitted to IEEE-EMBS Journal of Biomedical and Health Informatics on June 1, 2021

Multi-Modal Semantic Inconsistency Detection in Social Media News Posts

Scott McCrae, Kehan Wang, Avideh Zakhor

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Exploiting Temporal Dependencies for Cross-Modal Music Piece Identification

Luis Carvalho, Gerhard Widmer

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 eess.AS

Comments 5 pages, 3 figures

Journal ref Proceedings of the 29th European Signal Processing Conference (EUSIPCO 2021), Dublin, Ireland

Recent Advances and Trends in Multimodal Deep Learning: A Review

Jabeen Summaira, Xi Li, Amin Muhammad Shoib, Songyuan Li, Jabbar Abdul

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Multi-target DoA Estimation with an Audio-visual Fusion Mechanism

Xinyuan Qian, Maulik Madhavi, Zexu Pan, Jiadong Wang, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments ICASSP 2021 accepted

FragmentVC: Any-to-Any Voice Conversion by End-to-End Extracting and Fusing Fine-Grained Voice Fragments With Attention

Yist Y. Lin, Chung-Ming Chien, Jheng-Hao Lin, Hung-yi Lee, Lin-shan Lee

专题命中 音频语音多模态 ：any-to-any(title,abstract)；分类 eess.AS

Comments To appear in the proceedings of ICASSP 2021, equal contribution from first two authors

Cross-Modal Music-Video Recommendation: A Study of Design Choices

Laure Pretet, Gael Richard, Geoffroy Peeters

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.MM

Multimodal Self-Supervised Learning of General Audio Representations

Luyu Wang, Pauline Luc, Adria Recasens, Jean-Baptiste Alayrac, Aaron van den Oord

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Uncertainty-Aware Boosted Ensembling in Multi-Modal Settings

Utkarsh Sarawgi, Rishab Khincha, Wazeer Zulfikar, Satrajit Ghosh, Pattie Maes

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted at IJCNN 2021, to appear in IEEE proceedings. Equal contributions from US, RK and WZ

Fusing information streams in end-to-end audio-visual speech recognition

Wentao Yu, Steffen Zeiler, Dorothea Kolossa

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments 5 pages

Journal ref Published in International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021