多模态大模型

Voice-Enabled AI Agents can Perform Common Scams

Richard Fang, Dylan Bowman, Daniel Kang

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

PAT: Parameter-Free Audio-Text Aligner to Boost Zero-Shot Audio Classification

Ashish Seth, Ramaneswaran Selvakumar, Sonal Kumar, Sreyan Ghosh, Dinesh Manocha

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments 18 pages

CLIP-VAD: Exploiting Vision-Language Models for Voice Activity Detection

Andrea Appiani, Cigdem Beyan

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotation

Sen Fang, Sizhou Chen, Yalin Feng, Xiaofeng Zhang, Teik Toe Teoh

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Zero-Shot Sing Voice Conversion: built upon clustering-based phoneme representations

Wangjin Zhou, Fengrun Zhang, Yiming Liu, Wenhao Guan, Yi Zhao, Tatsuya Kawahara

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Latent CLAP Loss for Better Foley Sound Synthesis

Tornike Karchkhadze, Hassan Salami Kavaki, Mohammad Rasool Izadi, Bryce Irvin, Mikolaj Kegler, Ari Hertz, Shuo Zhang, Marko Stamenovic

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Journal ref EUSIPCO 2024 Proceedings, ISBN: 978-9-4645-9361-7

The USTC-NERCSLIP Systems for the CHiME-8 MMCSG Challenge

Ya Jiang, Hongbo Lan, Jun Du, Qing Wang, Shutong Niu

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions

Leena Mathur, Paul Pu Liang, Louis-Philippe Morency

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments EMNLP 2024 Main Conference

TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation

Haiyang Liu, Xingchao Yang, Tomoya Akiyama, Yuantian Huang, Qiaoge Li, Shigeru Kuriyama, Takafumi Taketomi

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments 16 pages, 8 figures

Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach

Siqi Li, Danni Liu, Jan Niehues

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL

Comments EMNLP 2024

You Only Speak Once to See

Wenhao Yang, Jianguo Wei, Wenhuan Lu, Lei Li

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 7 pages, 4 figures

Efficient learning-based sound propagation for virtual and real-world audio processing applications

Anton Jeran Ratnarajah

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments PhD thesis

AggregHate: An Efficient Aggregative Approach for the Detection of Hatemongers on Social Platforms

Tom Marzea, Abraham Israeli, Oren Tsur

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Seed-Music: A Unified Framework for High Quality and Controlled Music Generation

Ye Bai, Haonan Chen, Jitong Chen, Zhuo Chen, Yi Deng, Xiaohong Dong, Lamtharn Hantrakul, Weituo Hao, Qingqing Huang, Zhongyi Huang, Dongya Jia, Feihu La, Duc Le, Bochen Li, Chumin Li, Hui Li, Xingxing Li, Shouda Liu, Wei-Tsung Lu, Yiqing Lu, Andrew Shaw, Janne Spijkervet, Yakun Sun, Bo Wang, Ju-Chiang Wang, Yuping Wang, Yuxuan Wang, Ling Xu, Yifeng Yang, Chao Yao, Shuo Zhang, Yang Zhang, Yilin Zhang, Hang Zhao, Ziyi Zhao, Dejian Zhong, Shicen Zhou, Pei Zou

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Seed-Music technical report, 20 pages, 5 figures

StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion

Yinghao Aaron Li, Xilin Jiang, Cong Han, Nima Mesgarani

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Faster Speech-LLaMA Inference with Multi-token Prediction

Desh Raj, Gil Keren, Junteng Jia, Jay Mahadeokar, Ozlem Kalinli

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Submitted to IEEE ICASSP 2025

1M-Deepfakes Detection Challenge

Zhixi Cai, Abhinav Dhall, Shreya Ghosh, Munawar Hayat, Dimitrios Kollias, Kalin Stefanov, Usman Tariq

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments ACM MM 2024. Challenge webpage: https://deepfakes1m.github.io/

A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities

Jungpil Shin, Abu Saleh Musa Miah, Md. Humaun Kabir, Md. Abdur Rahim, Abdullah Al Shiam

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Journal ref IEEE Access-09 September 2024

Zero-shot Audio Topic Reranking using Large Language Models

Mengjie Qian, Rao Ma, Adian Liusie, Erfan Loweimi, Kate M. Knill, Mark J. F. Gales

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

DeWinder: Single-Channel Wind Noise Reduction using Ultrasound Sensing

Kuang Yuan, Shuo Han, Swarun Kumar, Bhiksha Raj

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Look Hear: Gaze Prediction for Speech-directed Human Attention

Sounak Mondal, Seoyoung Ahn, Zhibo Yang, Niranjan Balasubramanian, Dimitris Samaras, Gregory Zelinsky, Minh Hoai

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted for ECCV 2024

Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations

Xinran Li, Xiaomao Fan, Qingyang Wu, Xiaojiang Peng, Ye Li

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Data standardization for robust lip sync

Chun Wang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Unsupervised Welding Defect Detection Using Audio And Video

Georg Stemmer, Jose A. Lopez, Juan A. Del Hoyo Ontiveros, Arvind Raju, Tara Thimmanaik, Sovan Biswas

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 21 pages

Dynamic Motion Synthesis: Masked Audio-Text Conditioned Spatio-Temporal Transformers

Sohan Anisetty, James Hays

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Watermarking Techniques for Large Language Models: A Survey

Yuqing Liang, Jiancheng Xiao, Wensheng Gan, Philip S. Yu

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments Preprint. 19 figures, 7 tables

Enhancing Sound Source Localization via False Negative Elimination

Zengjie Song, Jiangshe Zhang, Yuxi Wang, Junsong Fan, Zhaoxiang Zhang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2203.13412

MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion

Chencan Fu, Yabiao Wang, Jiangning Zhang, Zhengkai Jiang, Xiaofeng Mao, Jiafu Wu, Weijian Cao, Chengjie Wang, Yanhao Ge, Yong Liu

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.MM

Comments Accepted by ACM MM 2024

Combined assessment of auditory distance perception and externalization

Henning Hoppe, Steven van de Par, Virginia Flanagin, Stephan D. Ewert

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments This work has been submitted to The Journal of the Acoustical Society of America of the for possible publication

LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation

Shihao Chen, Yu Gu, Jianwei Cui, Jie Zhang, Rilin Chen, Lirong Dai

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Comments Accepted to ISCSLP 2024. arXiv admin note: text overlap with arXiv:2406.05325