多模态大模型

QuAVF: Quality-aware Audio-Visual Fusion for Ego4D Talking to Me Challenge

Hsi-Che Lin, Chien-Yi Wang, Min-Hung Chen, Szu-Wei Fu, Yu-Chiang Frank Wang

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Comments 1st place at Ego4D Talking to Me (TTM) Challenge 2023

Bayesian Networks for the robust and unbiased prediction of depression and its symptoms utilizing speech and multimodal data

Salvatore Fara, Orlaith Hickey, Alexandra Georgescu, Stefano Goria, Emilia Molimpakis, Nicholas Cummins

专题命中 音频语音多模态 ：multimodal(title)；分类 cs.AI

Comments Accepted for publication at Interspeech 2023

How Good is Automatic Segmentation as a Multimodal Discourse Annotation Aid?

Corbyn Terpstra, Ibrahim Khebour, Mariah Bradford, Brett Wisniewski, Nikhil Krishnaswamy, Nathaniel Blanchard

专题命中 音频语音多模态 ：multimodal(title)；分类 cs.CL

Comments 7 pages, 1 figure, 2 tables, Proceedings of 19th Joint ISO-ACL Workshop on Interoperable Semantic Annotation (ISA 2023)

Synthesizing Photorealistic Virtual Humans Through Cross-modal Disentanglement

Siddarth Ravichandran, Ondřej Texler, Dimitar Dinev, Hyun Jae Kang

专题命中 音频语音多模态 ：cross-modal(title)；分类 cs.CV

A dataset for Audio-Visual Sound Event Detection in Movies

Rajat Hebbar, Digbalay Bose, Krishna Somandepalli, Veena Vijai, Shrikanth Narayanan

专题命中 音频语音多模态 ：audio-visual(title)；分类 eess.AS

Exploring adaptation of VideoMAE for Audio-Visual Diarization & Social @ Ego4d Looking at me Challenge

Yinan He, Guo Chen

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Video-Music Retrieval:A Dual-Path Cross-Modal Network

Xin Gu, Yinghua Shen, Chaohui Lv

专题命中 音频语音多模态 ：cross-modal(title)；分类 cs.MM

Comments 5pages,3figures

Unsupervised Audio-Visual Lecture Segmentation

Darshan Singh S, Anchit Gupta, C. V. Jawahar, Makarand Tapaswi

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Comments 17 pages, 14 figures, 14 tables, Accepted to WACV 2023. Project page: https://cvit.iiit.ac.in/research/projects/cvit-projects/avlectures

Towards Intercultural Affect Recognition: Audio-Visual Affect Recognition in the Wild Across Six Cultures

Leena Mathur, Ralph Adolphs, Maja J Matarić

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Comments Accepted at IEEE International Conference on Automatic Face and Gesture Recognition (FG 2023), publication and presentation at refereed IEEE workshop

Audio-Visual Face Reenactment

Madhav Agarwal, Rudrabha Mukhopadhyay, Vinay Namboodiri, C V Jawahar

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Comments Winter Conference on Applications of Computer Vision (WACV), 2023

Multimodal Speech Emotion Recognition using Cross Attention with Aligned Audio and Text

Yoonhyung Lee, Seunghyun Yoon, Kyomin Jung

专题命中 音频语音多模态 ：multimodal(title)；分类 eess.AS

Comments 5 pages, accepted by INTERSPEECH 2020

Journal ref Proc. Interspeech 2020, 2717-2721

Multimodal Clustering with Role Induced Constraints for Speaker Diarization

Nikolaos Flemotomos, Shrikanth Narayanan

专题命中 音频语音多模态 ：multimodal(title)；分类 eess.AS

Comments To appear at Interspeech 2022

Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning

Maximilian Du, Olivia Y. Lee, Suraj Nair, Chelsea Finn

专题命中 音频语音多模态 ：audio-visual(title)；分类 eess.AS

Journal ref Robotics Science and Systems (RSS) 2022

Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation

Xian Liu, Qianyi Wu, Hang Zhou, Yinghao Xu, Rui Qian, Xinyi Lin, Xiaowei Zhou, Wayne Wu, Bo Dai, Bolei Zhou

专题命中 音频语音多模态 ：cross-modal(title)；分类 cs.CV

Comments Accepted by IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022. Camera-Ready Version, 19 Pages

Enhancing Affective Representations of Music-Induced EEG through Multimodal Supervision and latent Domain Adaptation

Kleanthis Avramidis, Christos Garoufis, Athanasia Zlatintsi, Petros Maragos

专题命中 音频语音多模态 ：multimodal(title)；分类 eess.AS

Comments 5 pages, 3 figures, IEEE ICASSP 2022

Multimodal analysis of the predictability of hand-gesture properties

Taras Kucherenko, Rajmund Nagy, Michael Neff, Hedvig Kjellström, Gustav Eje Henter

专题命中 音频语音多模态 ：multimodal(title)；分类 cs.MM

Comments Accepted at the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) 2022

Disentanglement Learning for Variational Autoencoders Applied to Audio-Visual Speech Enhancement

Guillaume Carbajal, Julius Richter, Timo Gerkmann

专题命中 音频语音多模态 ：audio-visual(title)；分类 eess.AS

Comments arXiv admin note: text overlap with arXiv:2102.06454

Journal ref 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA)

Masking Modalities for Cross-modal Video Retrieval

Valentin Gabeur, Arsha Nagrani, Chen Sun, Karteek Alahari, Cordelia Schmid

专题命中 音频语音多模态 ：cross-modal(title)；分类 cs.CV

Comments Accepted at WACV 2022

Families In Wild Multimedia: A Multimodal Database for Recognizing Kinship

Joseph P. Robinson, Zaid Khan, Yu Yin, Ming Shao, Yun Fu

专题命中 音频语音多模态 ：multimodal(title)；分类 cs.CV

Journal ref IEEE Transactions on Multimedia (2021)

Audio-Visual Transformer Based Crowd Counting

Usman Sajid, Xiangyu Chen, Hasan Sajid, Taejoon Kim, Guanghui Wang

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Cross-modal Spectrum Transformation Network For Acoustic Scene classification

Yang Liu, Alexandros Neophytou, Sunando Sengupta, Eric Sommerlade

专题命中 音频语音多模态 ：cross-modal(title)；分类 eess.AS

Journal ref ICASSP 2021

End-to-end Speech Translation via Cross-modal Progressive Training

Rong Ye, Mingxuan Wang, Lei Li

专题命中 音频语音多模态 ：cross-modal(title)；分类 cs.CL

Comments Accepted at InterSpeech2021

Multi-modal Conditional Bounding Box Regression for Music Score Following

Florian Henkel, Gerhard Widmer

专题命中 音频语音多模态 ：multi-modal(title)；分类 eess.AS

Comments Accepted for publication in the Proceedings of the 29th European Signal Processing Conference (EUSIPCO), Dublin, Ireland, 2021

Active Contrastive Learning of Audio-Visual Video Representations

Shuang Ma, Zhaoyang Zeng, Daniel McDuff, Yale Song

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Labelling unlabelled videos from scratch with multi-modal self-supervision

Yuki M. Asano, Mandela Patrick, Christian Rupprecht, Andrea Vedaldi

专题命中 音频语音多模态 ：multi-modal(title)；分类 cs.CV

Comments Accepted to NeurIPS 2020. Project page: https://www.robots.ox.ac.uk/~vgg/research/selavi, code: https://github.com/facebookresearch/selavi

An Audio-Video Deep and Transfer Learning Framework for Multimodal Emotion Recognition in the wild

Denis Dresvyanskiy, Elena Ryumina, Heysem Kaya, Maxim Markitantov, Alexey Karpov, Wolfgang Minker

专题命中 音频语音多模态 ：multimodal(title)；分类 eess.AS

Comments Results on test dataset and acknowledgements were added

Learning Contextual Tag Embeddings for Cross-Modal Alignment of Audio and Tags

Xavier Favory, Konstantinos Drossos, Tuomas Virtanen, Xavier Serra

专题命中 音频语音多模态 ：cross-modal(title)；分类 eess.AS

Comments 5 pages, 1 figure

Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues

Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, Dinesh Manocha

专题命中 音频语音多模态 ：audio-visual(title)；分类 cs.CV

Comments Accepted to ACMMM-2020

Audio-Visual Calibration with Polynomial Regression for 2-D Projection Using SVD-PHAT

Francois Grondin, Hao Tang, James Glass

专题命中 音频语音多模态 ：audio-visual(title)；分类 eess.AS

Multimodal Learning For Classroom Activity Detection

Hang Li, Yu Kang, Wenbiao Ding, Song Yang, Songfan Yang, Gale Yan Huang, Zitao Liu

专题命中 音频语音多模态 ：multimodal(title)；分类 eess.AS

Comments The 45th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020)