多模态大模型

Jointly Learning Visual and Auditory Speech Representations from Raw Data

Alexandros Haliassos, Pingchuan Ma, Rodrigo Mira, Stavros Petridis, Maja Pantic

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments ICLR 2023. Code: https://github.com/ahaliassos/raven

Robust Dancer: Long-term 3D Dance Synthesis Using Unpaired Data

Bin Feng, Tenglong Ao, Zequn Liu, Wei Ju, Libin Liu, Ming Zhang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Preliminary video demo: https://youtu.be/gJbxG9QlcUU

VMCML: Video and Music Matching via Cross-Modality Lifting

Yi-Shan Lee, Wei-Cheng Tseng, Fu-En Wang, Min Sun

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Audio-Text Models Do Not Yet Leverage Natural Language

Ho-Hsiang Wu, Oriol Nieto, Juan Pablo Bello, Justin Salamon

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Copyright 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Human in the Loop for Machine Creativity

Neo Christopher Chung

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments 9th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2021), Blue Sky Ideas track

VE-KWS: Visual Modality Enhanced End-to-End Keyword Spotting

Ao Zhang, He Wang, Pengcheng Guo, Yihui Fu, Lei Xie, Yingying Gao, Shilei Zhang, Junlan Feng

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments 5 pages. Accepted at ICASSP2023

CASP-Net: Rethinking Video Saliency Prediction from an Audio-VisualConsistency Perceptual Perspective

Junwen Xiong, Ganglai Wang, Peng Zhang, Wei Huang, Yufei Zha, Guangtao Zhai

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments 10 pages, 7 figures, CVPR2023 accepted

A trained humanoid robot can perform human-like crossmodal social attention and conflict resolution

Di Fu, Fares Abawi, Hugo Carneiro, Matthias Kerzel, Ziwei Chen, Erik Strahl, Xun Liu, Stefan Wermter

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.AI

Comments accepted for publication in the International Journal of Social Robotics

ImagineNET: Target Speaker Extraction with Intermittent Visual Cue through Embedding Inpainting

Zexu Pan, Wupeng Wang, Marvin Borsdorf, Haizhou Li

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Accepted by ICASSP2023

Disentangled Feature Learning for Real-Time Neural Speech Coding

Xue Jiang, Xiulian Peng, Yuan Zhang, Yan Lu

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Comments ICASSP 2023 (Accepted)

Explainable Human-centered Traits from Head Motion and Facial Expression Dynamics

Surbhi Madan, Monika Gahalawat, Tanaya Guha, Roland Goecke, Ramanathan Subramanian

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representations

Shehzeen Hussain, Paarth Neekhara, Jocelyn Huang, Jason Li, Boris Ginsburg

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Comments Published as a conference paper at ICASSP 2023

Real-Time Acoustic Perception for Automotive Applications

Jun Yin, Stefano Damiano, Marian Verhelst, Toon van Waterschoot, Andre Guntoro

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Proceedings of the 1st International Workshop on Reading Music Systems

Jorge Calvo-Zaragoza, Jan Hajič, Alexander Pacha

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Proceedings edited by Jorge Calvo-Zaragoza, Jan Hajič jr. and Alexander Pacha

Audio2Gestures: Generating Diverse Gestures from Audio

Jing Li, Di Kang, Wenjie Pei, Xuefei Zhe, Ying Zhang, Linchao Bao, Zhenyu He

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2108.06720

Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Models

Tadahiro Taniguchi, Tomoaki Nakamura, Masahiro Suzuki, Ryo Kuniyasu, Kaede Hayashi, Akira Taniguchi, Takato Horii, Takayuki Nagai

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments New Gener. Comput. (2020)

Journal ref New Generation Computing, 2020, volume 38, 23--48

The RPM3D project: 3D Kinematics for Remote Patient Monitoring

Alicia Fornés, Asma Bensalah, Cristina Carmona-Duarte, Jialuo Chen, Miguel A. Ferrer, Andreas Fischer, Josep Lladós, Cristina Martín, Eloy Opisso, Réjean Plamondon, Anna Scius-Bertrand, Josep Maria Tormos

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Proceedings of the 2nd International Workshop on Reading Music Systems

Jorge Calvo-Zaragoza, Alexander Pacha

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Proceedings edited by Jorge Calvo-Zaragoza and Alexander Pacha

Proceedings of the 3rd International Workshop on Reading Music Systems

Jorge Calvo-Zaragoza, Alexander Pacha

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Proceedings edited by Jorge Calvo-Zaragoza and Alexander Pacha

A Case for Business Process-Specific Foundation Models

Yara Rizk, Praveen Venkateswaran, Vatche Isahagian, Vinod Muthusamy

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Mix and Localize: Localizing Sound Sources in Mixtures

Xixi Hu, Ziyang Chen, Andrew Owens

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments CVPR 2022

Autonomous Racing using a Hybrid Imitation-Reinforcement Learning Architecture

Chinmay Vilas Samak, Tanmay Vilas Samak, Sivanathan Kandhasamy

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.AI

A Comparative Study of Self-supervised Speech Representation Based Voice Conversion

Wen-Chin Huang, Shu-Wen Yang, Tomoki Hayashi, Tomoki Toda

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Comments Accepted to IEEE Journal of Selected Topics in Signal Processing. arXiv admin note: substantial text overlap with arXiv:2110.06280

Balanced Deep CCA for Bird Vocalization Detection

Sumit Kumar, B. Anshuman, Linus Ruettimann, Richard H. R. Hahnloser, Vipul Arora

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora

Sara Papi, Alina Karakanta, Matteo Negri, Marco Turchi

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Journal ref AACL 2022

Climate and Weather: Inspecting Depression Detection via Emotion Recognition

Wen Wu, Mengyue Wu, Kai Yu

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Journal ref ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6262-6266

Privacy-Utility Balanced Voice De-Identification Using Adversarial Examples

Meng Chen, Li Lu, Jiadi Yu, Yingying Chen, Zhongjie Ba, Feng Lin, Kui Ren

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics

Eda Okur, Saurav Sahay, Roddy Fuentes Alba, Lama Nachman

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments Proceedings of the 1st Workshop on Mathematical Natural Language Processing (MathNLP) at EMNLP 2022

Hear The Flow: Optical Flow-Based Self-Supervised Visual Sound Source Localization

Dennis Fedorishin, Deen Dayal Mohan, Bhavin Jawade, Srirangaraj Setlur, Venu Govindaraju

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments Accepted to WACV 2023

Improving Speech Prosody of Audiobook Text-to-Speech Synthesis with Acoustic and Textual Contexts

Detai Xin, Sharath Adavanne, Federico Ang, Ashish Kulkarni, Shinnosuke Takamichi, Hiroshi Saruwatari

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS