多模态大模型

TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation

Haiyang Liu, Xingchao Yang, Tomoya Akiyama, Yuantian Huang, Qiaoge Li, Shigeru Kuriyama, Takafumi Taketomi

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments 16 pages, 8 figures

Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach

Siqi Li, Danni Liu, Jan Niehues

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL

Comments EMNLP 2024

You Only Speak Once to See

Wenhao Yang, Jianguo Wei, Wenhuan Lu, Lei Li

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 7 pages, 4 figures

Efficient learning-based sound propagation for virtual and real-world audio processing applications

Anton Jeran Ratnarajah

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments PhD thesis

AggregHate: An Efficient Aggregative Approach for the Detection of Hatemongers on Social Platforms

Tom Marzea, Abraham Israeli, Oren Tsur

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Seed-Music: A Unified Framework for High Quality and Controlled Music Generation

Ye Bai, Haonan Chen, Jitong Chen, Zhuo Chen, Yi Deng, Xiaohong Dong, Lamtharn Hantrakul, Weituo Hao, Qingqing Huang, Zhongyi Huang, Dongya Jia, Feihu La, Duc Le, Bochen Li, Chumin Li, Hui Li, Xingxing Li, Shouda Liu, Wei-Tsung Lu, Yiqing Lu, Andrew Shaw, Janne Spijkervet, Yakun Sun, Bo Wang, Ju-Chiang Wang, Yuping Wang, Yuxuan Wang, Ling Xu, Yifeng Yang, Chao Yao, Shuo Zhang, Yang Zhang, Yilin Zhang, Hang Zhao, Ziyi Zhao, Dejian Zhong, Shicen Zhou, Pei Zou

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Seed-Music technical report, 20 pages, 5 figures

StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion

Yinghao Aaron Li, Xilin Jiang, Cong Han, Nima Mesgarani

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Faster Speech-LLaMA Inference with Multi-token Prediction

Desh Raj, Gil Keren, Junteng Jia, Jay Mahadeokar, Ozlem Kalinli

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Submitted to IEEE ICASSP 2025

1M-Deepfakes Detection Challenge

Zhixi Cai, Abhinav Dhall, Shreya Ghosh, Munawar Hayat, Dimitrios Kollias, Kalin Stefanov, Usman Tariq

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments ACM MM 2024. Challenge webpage: https://deepfakes1m.github.io/

A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities

Jungpil Shin, Abu Saleh Musa Miah, Md. Humaun Kabir, Md. Abdur Rahim, Abdullah Al Shiam

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Journal ref IEEE Access-09 September 2024

Zero-shot Audio Topic Reranking using Large Language Models

Mengjie Qian, Rao Ma, Adian Liusie, Erfan Loweimi, Kate M. Knill, Mark J. F. Gales

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

DeWinder: Single-Channel Wind Noise Reduction using Ultrasound Sensing

Kuang Yuan, Shuo Han, Swarun Kumar, Bhiksha Raj

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Look Hear: Gaze Prediction for Speech-directed Human Attention

Sounak Mondal, Seoyoung Ahn, Zhibo Yang, Niranjan Balasubramanian, Dimitris Samaras, Gregory Zelinsky, Minh Hoai

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted for ECCV 2024

Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations

Xinran Li, Xiaomao Fan, Qingyang Wu, Xiaojiang Peng, Ye Li

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Data standardization for robust lip sync

Chun Wang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Unsupervised Welding Defect Detection Using Audio And Video

Georg Stemmer, Jose A. Lopez, Juan A. Del Hoyo Ontiveros, Arvind Raju, Tara Thimmanaik, Sovan Biswas

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 21 pages

Dynamic Motion Synthesis: Masked Audio-Text Conditioned Spatio-Temporal Transformers

Sohan Anisetty, James Hays

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Watermarking Techniques for Large Language Models: A Survey

Yuqing Liang, Jiancheng Xiao, Wensheng Gan, Philip S. Yu

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments Preprint. 19 figures, 7 tables

Enhancing Sound Source Localization via False Negative Elimination

Zengjie Song, Jiangshe Zhang, Yuxi Wang, Junsong Fan, Zhaoxiang Zhang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2203.13412

MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion

Chencan Fu, Yabiao Wang, Jiangning Zhang, Zhengkai Jiang, Xiaofeng Mao, Jiafu Wu, Weijian Cao, Chengjie Wang, Yanhao Ge, Yong Liu

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.MM

Comments Accepted by ACM MM 2024

Combined assessment of auditory distance perception and externalization

Henning Hoppe, Steven van de Par, Virginia Flanagin, Stephan D. Ewert

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments This work has been submitted to The Journal of the Acoustical Society of America of the for possible publication

LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation

Shihao Chen, Yu Gu, Jianwei Cui, Jie Zhang, Rilin Chen, Lirong Dai

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Comments Accepted to ISCSLP 2024. arXiv admin note: text overlap with arXiv:2406.05325

KunquDB: An Attempt for Speaker Verification in the Chinese Opera Scenario

Huali Zhou, Yuke Lin, Dong Liu, Ming Li

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Accepted by ICPR 2024

An Overlooked Role of Context-Sensitive Dendrites

Mohsin Raza, Ahsan Adeel

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.AI

SNR-Progressive Model with Harmonic Compensation for Low-SNR Speech Enhancement

Zhongshu Hou, Tong Lei, Qinwen Hu, Zhanzhong Cao, Ming Tang, Jing Lu

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

HSDreport: Heart Sound Diagnosis with Echocardiography Reports

Zihan Zhao, Pingjie Wang, Liudan Zhao, Yuchen Yang, Ya Zhang, Kun Sun, Xin Sun, Xin Zhou, Yu Wang, Yanfeng Wang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

NeuralMultiling: A Novel Neural Architecture Search for Smartphone based Multilingual Speaker Verification

Aravinda Reddy PN, Raghavendra Ramachandra, K. Sreenivasa Rao, Pabitra Mitra

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Identity-Driven Multimedia Forgery Detection via Reference Assistance

Junhao Xu, Jingjing Chen, Xue Song, Feng Han, Haijun Shan, Yugang Jiang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.MM

Evaluating and Enhancing Trustworthiness of LLMs in Perception Tasks

Malsha Ashani Mahawatta Dona, Beatriz Cabrero-Daniel, Yinan Yu, Christian Berger

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted in 27th IEEE International Conference on Intelligent Transportation Systems (ITSC) 2024

EmoFace: Audio-driven Emotional 3D Face Animation

Chang Liu, Qunfen Lin, Zijiao Zeng, Ye Pan

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 2024

Journal ref 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR). IEEE, 2024: 387-397