多模态大模型

Towards Ultrasound Tongue Image prediction from EEG during speech production

Tamás Gábor Csapó, Frigyes Viktor Arthur, Péter Nagy, Ádám Boncz

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments accepted at Interspeech 2023

Journal ref Proceedings of Interspeech 2023

CorrTalk: Correlation Between Hierarchical Speech and Facial Activity Variances for 3D Animation

Zhaojie Chu, Kailing Guo, Xiaofen Xing, Yilin Lan, Bolun Cai, Xiangmin Xu

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference

Dejan Porjazovski, Yaroslav Getman, Tamás Grósz, Mikko Kurimo

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments Accepted at ACMM 2023

Utilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reports

Qingqing Zhu, Tejas Sudharshan Mathai, Pritam Mukherjee, Yifan Peng, Ronald M. Summers, Zhiyong Lu

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL

GestSync: Determining who is speaking without a talking head

Sindhu B Hegde, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments Accepted in BMVC 2023, 10 pages paper, 7 pages supplementary, 7 Figures

Robust One-Shot Singing Voice Conversion

Naoya Takahashi, Mayank Kumar Singh, Yuki Mitsufuji

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration

Kehong Gong, Dongze Lian, Heng Chang, Chuan Guo, Zihang Jiang, Xinxin Zuo, Michael Bi Mi, Xinchao Wang

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by ICCV2023

Passage Summarization with Recurrent Models for Audio-Sheet Music Retrieval

Luis Carvalho, Gerhard Widmer

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments In Proceedings of the 24th Conference of the International Society for Music Information Retrieval (ISMIR 2023), Milan, Italy

Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach

Tae Jin Park, Kunal Dhawan, Nithin Koluguri, Jagadeesh Balam

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments 4 pages 1 reference page, ICASSP format

Dawn of the transformer era in speech emotion recognition: closing the valence gap

Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, Björn W. Schuller

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Journal ref in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 9, pp. 10745-10759, 1 Sept. 2023

Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition

Weiwei Zhou, Jiada Lu, Zhaolong Xiong, Weifeng Wang

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis

Linsen Song, Wayne Wu, Chaoyou Fu, Chen Change Loy, Ran He

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments TCSVT 2022

Journal ref IEEE Transactions on Circuits and Systems for Video Technology (Volume: 33, Issue: 3, March 2023)

An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation

Pengzhi Gao, Ruiqing Zhang, Zhongjun He, Hua Wu, Haifeng Wang

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL

Voice Conversion with Denoising Diffusion Probabilistic GAN Models

Xulong Zhang, Jianzong Wang, Ning Cheng, Jing Xiao

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments Accepted by 19th International Conference on Advanced Data Mining and Applications. (ADMA 2023)

Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight

Yunhua Zhang, Hazel Doughty, Cees G. M. Snoek

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments Under review

Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis

Jiahe Li, Jiawei Zhang, Xiao Bai, Jun Zhou, Lin Gu

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by ICCV 2023. Project page: https://fictionarry.github.io/ER-NeRF/

Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse

Ozge Mercanoglu Sincan, Necati Cihan Camgoz, Richard Bowden

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Cuing Without Sharing: A Federated Cued Speech Recognition Framework via Mutual Knowledge Distillation

Yuxuan Zhang, Lei Liu, Li Liu

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM

Let's Give a Voice to Conversational Agents in Virtual Reality

Michele Yin, Gabriel Roccabruna, Abhinav Azad, Giuseppe Riccardi

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Intelligent Sight and Sound: A Chronic Cancer Pain Dataset

Catherine Ordun, Alexandra N. Cha, Edward Raff, Byron Gaskin, Alex Hanson, Mason Rule, Sanjay Purushotham, James L. Gulley

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Published as conference paper at the 35th Conference on Neural Information Processing Systems (NeurIPS 2021) Track on Datasets and Benchmarks

Journal ref 2021, Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

Computer Vision Estimation of Emotion Reaction Intensity in the Wild

Yang Qian, Ali Kargarandehkordi, Onur Cezmi Mutlu, Saimourya Surabhi, Mohammadmahdi Honarmand, Dennis Paul Wall, Peter Washington

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Audio Inputs for Active Speaker Detection and Localization via Microphone Array

Davide Berghi, Philip J. B. Jackson

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions

Yunfei Liu, Lijian Lin, Fei Yu, Changyin Zhou, Yu Li

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by ICCV 2023

OxfordVGG Submission to the EGO4D AV Transcription Challenge

Jaesung Huh, Max Bain, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Technical Report

TencentPretrain: A Scalable and Flexible Toolkit for Pre-training Models of Different Modalities

Zhe Zhao, Yudong Li, Cheng Hou, Jing Zhao, Rong Tian, Weijie Liu, Yiren Chen, Ningyuan Sun, Haoyan Liu, Weiquan Mao, Han Guo, Weigang Guo, Taiqiang Wu, Tao Zhu, Wenhang Shi, Chen Chen, Shan Huang, Sihong Chen, Liqun Liu, Feifei Li, Xiaoshuai Chen, Xingwu Sun, Zhanhui Kang, Xiaoyong Du, Linlin Shen, Kimmo Yan

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL

UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data

Heeseung Kim, Sungwon Kim, Jiheum Yeom, Sungroh Yoon

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Comments INTERSPEECH 2023, Oral

InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Prompt

Dongchao Yang, Songxiang Liu, Rongjie Huang, Chao Weng, Helen Meng

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments Submit to TASLP

Learning an Interpretable End-to-End Network for Real-Time Acoustic Beamforming

Hao Liang, Guanxing Zhou, Xiaotong Tu, Andreas Jakobsson, Xinghao Ding, Yue Huang

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments 12 pages, 9 figures

NFTs to MARS: Multi-Attention Recommender System for NFTs

Seonmi Kim, Youngbin Lee, Yejin Kim, Joohwan Hong, Yongjae Lee

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Contrastive Learning-Based Audio to Lyrics Alignment for Multiple Languages

Simon Durand, Daniel Stoller, Sebastian Ewert

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments 5 pages, accepted at the International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2023

Journal ref ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023, pp. 1-5