多模态大模型 - arXivDaily 专题

2306.16862 2023-06-30 cs.CV cs.AI 76%

Sustainable Palm Tree Farming: Leveraging IoT and Multi-Modal Data for Early Detection and Mapping of Red Palm Weevil

Yosra Hajjaji, Ayyub Alzahem, Wadii Boulila, Imed Riadh Farah, Anis Koubaa

专题命中音频语音多模态：multi-modal(title)；分类 cs.CV、cs.AI

2211.01317 2023-05-04 cs.SD cs.AI cs.LG cs.NE eess.AS 76%

Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogramming

Yun-Ning Hung, Chao-Han Huck Yang, Pin-Yu Chen, Alexander Lerch

专题命中音频语音多模态：cross-modal(title)；分类 cs.AI、eess.AS

Comments Accepted to IEEE ICASSP 2023. The implementation is available at https://github.com/biboamy/music-repro

2304.07958 2023-04-18 cs.CV cs.SD eess.AS 76%

Recursive Joint Attention for Audio-Visual Fusion in Regression based Emotion Recognition

R Gnana Praveen, Eric Granger, Patrick Cardinal

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、eess.AS

2303.05309 2023-03-10 cs.CV cs.CL 76%

MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition

Xize Cheng, Linjun Li, Tao Jin, Rongjie Huang, Wang Lin, Zehan Wang, Huangdai Liu, Ye Wang, Aoxiong Yin, Zhou Zhao

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.CL

Comments https://github.com/Exgc/AVMuST-TED

2112.12180 2023-01-13 cs.CV cs.AI 76%

Multimodal Personality Recognition using Cross-Attention Transformer and Behaviour Encoding

Tanay Agrawal, Dhruv Agarwal, Michal Balazia, Neelabh Sinha, Francois Bremond

专题命中音频语音多模态：multimodal(title)；分类 cs.CV、cs.AI

Comments Preprint. Final paper accepted at the 17th International Conference on Computer Vision Theory and Applications (VISAPP), virtual, February, 2022. 8 pages

2211.00988 2022-11-03 cs.CV cs.LG cs.SD eess.AS eess.SP 76%

Audio-visual speech enhancement with a deep Kalman filter generative model

Ali Golmakani, Mostafa Sadeghi, Romain Serizel

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、eess.AS

2205.05206 2022-05-12 eess.AS cs.CV cs.LG cs.SD 76%

Best of Both Worlds: Multi-task Audio-Visual Automatic Speech Recognition and Active Speaker Detection

Otavio Braga, Olivier Siohan

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、eess.AS

2110.07592 2022-04-05 cs.CL cs.SD eess.AS 76%

DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances

Sreyan Ghosh, Samden Lepcha, S Sakshi, Rajiv Ratn Shah, S. Umesh

专题命中音频语音多模态：multimodal(title)；分类 cs.CL、eess.AS

Comments Submitted to Interspeech 2022

2110.02411 2021-10-07 cs.SD cs.CL cs.LG eess.AS 76%

Voice Aging with Audio-Visual Style Transfer

Justin Wilson, Sunyeong Park, Seunghye J. Wilson, Ming C. Lin

专题命中音频语音多模态：audio-visual(title)；分类 cs.CL、eess.AS

2110.01367 2021-10-05 cs.SD cs.MM eess.AS 76%

Audio-Visual Evaluation of Oratory Skills

Tzvi Michelson, Shmuel Peleg

专题命中音频语音多模态：audio-visual(title)；分类 cs.MM、eess.AS

Comments TransAI 2021

2110.01001 2021-10-05 cs.MM cs.SD eess.AS 76%

Multimodal Fusion Based Attentive Networks for Sequential Music Recommendation

Kunal Vaswani, Yudhik Agrawal, Vinoo Alluri

专题命中音频语音多模态：multimodal(title)；分类 cs.MM、eess.AS

2103.14201 2021-08-17 cs.SD cs.CV cs.LG eess.AS 76%

Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis

Nikhil Singh, Jeff Mentch, Jerry Ng, Matthew Beveridge, Iddo Drori

专题命中音频语音多模态：cross-modal(title)；分类 cs.CV、eess.AS

Comments ICCV 2021. Project page: https://web.media.mit.edu/~nsingh1/image2reverb/

2108.04906 2021-08-12 cs.SD cs.CV cs.MM 76%

Depth Infused Binaural Audio Generation using Hierarchical Cross-Modal Attention

Kranti Kumar Parida, Siddharth Srivastava, Neeraj Matiyali, Gaurav Sharma

专题命中音频语音多模态：cross-modal(title)；分类 cs.CV、cs.MM

Comments Presented at Sight and Sound Workshop, CVPR 2021

2104.01978 2021-06-08 cs.SD cs.AI eess.AS 76%

Acted vs. Improvised: Domain Adaptation for Elicitation Approaches in Audio-Visual Emotion Recognition

Haoqi Li, Yelin Kim, Cheng-Hao Kuo, Shrikanth Narayanan

专题命中音频语音多模态：audio-visual(title)；分类 cs.AI、eess.AS

Comments paper accepted by INTERSPEECH2021

2005.14405 2021-03-23 cs.CV cs.MM 76%

Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization

Komal Chugh, Parul Gupta, Abhinav Dhall, Ramanathan Subramanian

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM

2012.13235 2020-12-25 cs.LG cs.CL cs.CV 76%

Detecting Hateful Memes Using a Multimodal Deep Ensemble

Vlad Sandulescu

专题命中音频语音多模态：multimodal(title)；分类 cs.CV、cs.CL

Comments 6 pages, NeurIPS 2020, The Hateful Memes Challenge Workshop at NeurIPS 2020

Journal ref The Hateful Memes Challenge Workshop at NeurIPS 2020

2011.11970 2020-11-25 cs.SD cs.IR cs.MM eess.AS 76%

A Novel Multimodal Music Genre Classifier using Hierarchical Attention and Convolutional Neural Network

Manish Agrawal, Abhilash Nandy

专题命中音频语音多模态：multimodal(title)；分类 cs.MM、eess.AS

Comments 7 pages, 4 figures

2009.07879 2020-09-18 cs.CV cs.LG cs.MM cs.NE cs.RO 76%

Using Sensory Time-cue to enable Unsupervised Multimodal Meta-learning

Qiong Liu, Yanxia Zhang

专题命中音频语音多模态：multimodal(title)；分类 cs.CV、cs.MM

2009.01225 2020-09-07 cs.CV eess.AS 76%

Seeing wake words: Audio-visual Keyword Spotting

Liliane Momeni, Triantafyllos Afouras, Themos Stafylakis, Samuel Albanie, Andrew Zisserman

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、eess.AS

2007.01836 2020-08-13 eess.AS cs.CL cs.LG cs.SD 76%

Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning

Pavel Denisov, Ngoc Thang Vu

专题命中音频语音多模态：cross-modal(title)；分类 cs.CL、eess.AS

Comments Interspeech 2020

2007.13976 2020-07-29 cs.SD cs.CV eess.AS 76%

Self-supervised Neural Audio-Visual Sound Source Localization via Probabilistic Spatial Modeling

Yoshiki Masuyama, Yoshiaki Bando, Kohei Yatabe, Yoko Sasaki, Masaki Onishi, Yasuhiro Oikawa

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、eess.AS

Comments Accepted for publication in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

2007.09053 2020-07-20 cs.RO cs.AI cs.CL 76%

Situated Multimodal Control of a Mobile Robot: Navigation through a Virtual Environment

Katherine Krajovic, Nikhil Krishnaswamy, Nathaniel J. Dimick, R. Pito Salas, James Pustejovsky

专题命中音频语音多模态：multimodal(title)；分类 cs.CL、cs.AI

Comments 4 pages, 1 table, 4 figures, proceedings of RoboDIAL special session a SigDIAL 2020

2004.14840 2020-05-01 eess.AS cs.CV cs.LG cs.SD stat.ML 76%

Multiresolution and Multimodal Speech Recognition with Transformers

Georgios Paraskevopoulos, Srinivas Parthasarathy, Aparna Khare, Shiva Sundaram

专题命中音频语音多模态：multimodal(title)；分类 cs.CV、eess.AS

Comments Accepted for ACL 2020

2002.05314 2020-02-14 eess.AS cs.LG cs.MM cs.SD stat.ML 76%

Self-supervised learning for audio-visual speaker diarization

Yifan Ding, Yong Xu, Shi-Xiong Zhang, Yahuan Cong, Liqiang Wang

专题命中音频语音多模态：audio-visual(title)；分类 cs.MM、eess.AS

2001.10832 2020-01-30 eess.AS cs.LG cs.MM cs.SD eess.IV 76%

Audio-Visual Decision Fusion for WFST-based and seq2seq Models

Rohith Aralikatti, Sharad Roy, Abhinav Thanda, Dilip Kumar Margam, Pujitha Appan Kandala, Tanay Sharma, Shankar M Venkatesan

专题命中音频语音多模态：audio-visual(title)；分类 cs.MM、eess.AS

Comments Submitted for review to ICASSP 2020 on October 21st, 2019

1911.07917 2020-01-03 cs.CV cs.LG cs.SD eess.AS 76%

Cross-modal supervised learning for better acoustic representations

Shaoyong Jia, Xin Shu, Yang Yang, Dawei Liang, Qiyue Liu, Junhui Liu

专题命中音频语音多模态：cross-modal(title)；分类 cs.CV、eess.AS

1911.05659 2019-11-25 eess.SP cs.CL cs.LG eess.AS 76%

M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues

Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, Dinesh Manocha

专题命中音频语音多模态：multimodal(title)；分类 cs.CL、eess.AS

1909.11232 2019-09-26 cs.LG cs.CL cs.CV stat.ML 76%

Sign Language Recognition Analysis using Multimodal Data

Al Amin Hosain, Panneer Selvam Santhalingam, Parth Pathak, Jana Kosecka, Huzefa Rangwala

专题命中音频语音多模态：multimodal(title)；分类 cs.CV、cs.CL

Comments conference : IEEE DSAA, 2019, Washington DC

1909.02764 2019-09-10 cs.CL cs.HC eess.AS 76%

Towards Multimodal Emotion Recognition in German Speech Events in Cars using Transfer Learning

Deniz Cevher, Sebastian Zepf, Roman Klinger

专题命中音频语音多模态：multimodal(title)；分类 cs.CL、eess.AS

Comments 12 pages, 2 figures, accepted at KONVENS 2019

1907.04355 2019-07-11 cs.CL cs.LG cs.SD eess.AS 76%

Transfer Learning from Audio-Visual Grounding to Speech Recognition

Wei-Ning Hsu, David Harwath, James Glass

专题命中音频语音多模态：audio-visual(title)；分类 cs.CL、eess.AS

Comments Accepted to Interspeech 2019. 4 pages, 2 figures

1. 音频语音多模态 4587 篇

Sustainable Palm Tree Farming: Leveraging IoT and Multi-Modal Data for Early Detection and Mapping of Red Palm Weevil

Low-Resource Music Genre Classification with Cross-Modal Neural Model Reprogramming

Recursive Joint Attention for Audio-Visual Fusion in Regression based Emotion Recognition

MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition

Multimodal Personality Recognition using Cross-Attention Transformer and Behaviour Encoding

Audio-visual speech enhancement with a deep Kalman filter generative model

Best of Both Worlds: Multi-task Audio-Visual Automatic Speech Recognition and Active Speaker Detection

DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances

Voice Aging with Audio-Visual Style Transfer

Audio-Visual Evaluation of Oratory Skills

Multimodal Fusion Based Attentive Networks for Sequential Music Recommendation

Image2Reverb: Cross-Modal Reverb Impulse Response Synthesis

Depth Infused Binaural Audio Generation using Hierarchical Cross-Modal Attention

Acted vs. Improvised: Domain Adaptation for Elicitation Approaches in Audio-Visual Emotion Recognition

Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization

Detecting Hateful Memes Using a Multimodal Deep Ensemble

A Novel Multimodal Music Genre Classifier using Hierarchical Attention and Convolutional Neural Network

Using Sensory Time-cue to enable Unsupervised Multimodal Meta-learning

Seeing wake words: Audio-visual Keyword Spotting

Pretrained Semantic Speech Embeddings for End-to-End Spoken Language Understanding via Cross-Modal Teacher-Student Learning

Self-supervised Neural Audio-Visual Sound Source Localization via Probabilistic Spatial Modeling

Situated Multimodal Control of a Mobile Robot: Navigation through a Virtual Environment

Multiresolution and Multimodal Speech Recognition with Transformers

Self-supervised learning for audio-visual speaker diarization

Audio-Visual Decision Fusion for WFST-based and seq2seq Models

Cross-modal supervised learning for better acoustic representations

M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues

Sign Language Recognition Analysis using Multimodal Data

Towards Multimodal Emotion Recognition in German Speech Events in Cars using Transfer Learning

Transfer Learning from Audio-Visual Grounding to Speech Recognition