多模态大模型

Generalized zero-shot audio-to-intent classification

Veera Raghavendra Elluru, Devang Kulshreshtha, Rohit Paturi, Sravan Bodapati, Srikanth Ronanki

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI、eess.AS

CSLNSpeech: solving extended speech separation problem with the help of Chinese sign language

Jiasong Wu, Xuan Li, Taotao Li, Fanman Meng, Youyong Kong, Guanyu Yang, Lotfi Senhadji, Huazhong Shu

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments 13 pages, 6 figures, 5 tables

Automatic Disfluency Detection from Untranscribed Speech

Amrit Romana, Kazuhito Koishida, Emily Mower Provost

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer

Kunkun Pang, Dafei Qin, Yingruo Fan, Julian Habekost, Takaaki Shiratori, Junichi Yamagishi, Taku Komura

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 13 figures

Vision-based Analysis of Driver Activity and Driving Performance Under the Influence of Alcohol

Ross Greer, Akshay Gopalkrishnan, Sumega Mandadi, Pujitha Gunaratne, Mohan M. Trivedi, Thomas D. Marcotte

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Withdrawn at the request of industry research collaborators, per contract agreement

A Comparative Study of Voice Conversion Models with Large-Scale Speech and Singing Data: The T13 Systems for the Singing Voice Conversion Challenge 2023

Ryuichi Yamamoto, Reo Yoneyama, Lester Phillip Violeta, Wen-Chin Huang, Tomoki Toda

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 cs.CL、eess.AS

Comments Accepted to ASRU 2023

Audio Visual Speaker Localization from EgoCentric Views

Jinzheng Zhao, Yong Xu, Xinyuan Qian, Wenwu Wang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.MM、eess.AS

AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model

Seungwhan Moon, Andrea Madotto, Zhaojiang Lin, Tushar Nagarajan, Matt Smith, Shashank Jain, Chun-Fu Yeh, Prakash Murugesan, Peyman Heidari, Yue Liu, Kavya Srinet, Babak Damavandi, Anuj Kumar

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

My Science Tutor (MyST) -- A Large Corpus of Children's Conversational Speech

Sameer S. Pradhan, Ronald A. Cole, Wayne H. Ward

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

FoleyGen: Visually-Guided Audio Generation

Xinhao Mei, Varun Nagaraja, Gael Le Lan, Zhaoheng Ni, Ernie Chang, Yangyang Shi, Vikas Chandra

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.MM、eess.AS

Addressing the Blind Spots in Spoken Language Processing

Amit Moryossef

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Comments 5 pages

UniBriVL: Robust Universal Representation and Generation of Audio Driven Diffusion Models

Sen Fang, Bowen Gao, Yangjian Wu, Teik Toe Teoh

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI、eess.AS

Comments Voice-Text fusion input; The first work of audio driven diffusion model. arXiv admin note: text overlap with arXiv:2303.04585

Comparative Analysis of Deep-Fake Algorithms

Nikhil Sontakke, Sejal Utekar, Shivansh Rastogi, Shriraj Sonawane

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 4 figures, 2 tables, Published with International Journal of Computer Science Trends and Technology (IJCST)

Journal ref International Journal of Computer Science Trends and Technology (IJCST) V11(4): Page(109-115) Jul - Aug 2023. ISSN: 2347-8578

A Comprehensive Survey for Evaluation Methodologies of AI-Generated Music

Zeyu Xiong, Weitao Wang, Jing Yu, Yue Lin, Ziyan Wang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI、eess.AS

Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation

Ziyang Chen, Shengyi Qian, Andrew Owens

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments ICCV 2023. Project site: https://ificl.github.io/SLfM/

Radio2Text: Streaming Speech Recognition Using mmWave Radio Signals

Running Zhao, Jiangtao Yu, Hang Zhao, Edith C. H. Ngai

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted by Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (ACM IMWUT/UbiComp 2023)

Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge

Hugo Carneiro, Cornelius Weber, Stefan Wermter

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV、eess.AS

Comments 17 pages, 8 figures, 7 tables, Published in Neurocomputing

Journal ref Neurocomputing (2023); Volume 545; 126271

Capturing Spectral and Long-term Contextual Information for Speech Emotion Recognition Using Deep Learning Techniques

Samiul Islam, Md. Maksudul Haque, Abu Jobayer Md. Sadat

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments the research paper is still in progress

Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis

Zhenhui Ye, Ziyue Jiang, Yi Ren, Jinglin Liu, Chen Zhang, Xiang Yin, Zejun Ma, Zhou Zhao

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments Accepted by ICML 2023 Workshop, 6 pages, 3 figures

The Hidden Dance of Phonemes and Visage: Unveiling the Enigmatic Link between Phonemes and Facial Features

Liao Qu, Xianwei Zou, Xiang Li, Yandong Wen, Rita Singh, Bhiksha Raj

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV、eess.AS

Comments Interspeech 2023

CLIPSonic: Text-to-Audio Synthesis with Unlabeled Videos and Pretrained Language-Vision Models

Hao-Wen Dong, Xiaoyu Liu, Jordi Pons, Gautam Bhattacharya, Santiago Pascual, Joan Serrà, Taylor Berg-Kirkpatrick, Julian McAuley

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.MM、eess.AS

Comments Accepted by WASPAA 2023. Demo: https://salu133445.github.io/clipsonic/

Topic Identification For Spontaneous Speech: Enriching Audio Features With Embedded Linguistic Information

Dejan Porjazovski, Tamás Grósz, Mikko Kurimo

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to EUSIPCO 2023

Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models

Simian Luo, Chuanhao Yan, Chenxu Hu, Hang Zhao

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Classification of Infant Sleep/Wake States: Cross-Attention among Large Scale Pretrained Transformer Networks using Audio, ECG, and IMU Data

Kai Chieh Chang, Mark Hasegawa-Johnson, Nancy L. McElwain, Bashima Islam

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.MM、eess.AS

Comments Preprint for APSIPA2023

Mu$^{2}$SLAM: Multitask, Multilingual Speech and Language Models

Yong Cheng, Yu Zhang, Melvin Johnson, Wolfgang Macherey, Ankur Bapna

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments ICML 2023

Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognition

Shujie Hu, Xurong Xie, Zengrui Jin, Mengzhe Geng, Yi Wang, Mingyu Cui, Jiajun Deng, Xunying Liu, Helen Meng

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI、eess.AS

Comments accepted by ICASSP 2023

PATCorrect: Non-autoregressive Phoneme-augmented Transformer for ASR Error Correction

Ziji Zhang, Zhehui Wang, Rajesh Kamma, Sharanya Eswaran, Narayanan Sadagopan

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted camera-ready version for INTERSPEECH 2023

Visual-Aware Text-to-Speech

Mohan Zhou, Yalong Bai, Wei Zhang, Ting Yao, Tiejun Zhao, Tao Mei

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments accepted as oral and top 3% paper by ICASSP 2023

Journal ref ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023, 1-5

RealImpact: A Dataset of Impact Sound Fields for Real Objects

Samuel Clarke, Ruohan Gao, Mason Wang, Mark Rau, Julia Xu, Jui-Hsien Wang, Doug L. James, Jiajun Wu

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments CVPR 2023 (Highlight). Project page: https://samuelpclarke.com/realimpact/

OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment

Xize Cheng, Tao Jin, Linjun Li, Wang Lin, Xinyu Duan, Zhou Zhao

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACL2023 (Oral)