多模态大模型

VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesis

Jaemin Jung, Junseok Ahn, Chaeyoung Jung, Tan Dat Nguyen, Youngjoon Jang, Joon Son Chung

机构 * Korea Advanced Institute of Science and Technology（韩国科学技术院） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments Accepted to ICASSP 2025

WMCodec: End-to-End Neural Speech Codec with Deep Watermarking for Authenticity Verification

Junzuo Zhou, Jiangyan Yi, Yong Ren, Jianhua Tao, Tao Wang, Chu Yuan Zhang

机构 * Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所） ； Department of Automation Tsinghua University（清华大学自动化系） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion

Changan Chen, Juze Zhang, Shrinidhi K. Lakshmikanth, Yusu Fang, Ruizhi Shao, Gordon Wetzstein, Li Fei-Fei, Ehsan Adeli

机构 * Stanford University（斯坦福大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Project page: languageofmotion.github.io

wav2pos: Sound Source Localization using Masked Autoencoders

Axel Berg, Jens Gulin, Mark O'Connor, Chuteng Zhou, Karl Åström, Magnus Oskarsson

机构 * Lund University（隆德大学） ； Arm（安谋科技） ； Sony（索尼） ； Tenstorrent 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments IPIN 2024

From Audio Deepfake Detection to AI-Generated Music Detection -- A Pathway and Overview

Yupei Li, Manuel Milling, Lucia Specia, Björn W. Schuller

机构 * Imperial College London（帝国理工学院） ； Technical University of Munich（慕尼黑工业大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Explainable Artificial Intelligence for Medical Applications: A Review

Qiyang Sun, Alican Akman, Björn W. Schuller

机构 * Imperial College London（帝国理工学院） ； Technical University of Munich（慕尼黑工业大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

STEVE-Audio: Expanding the Goal Conditioning Modalities of Embodied Agents in Minecraft

Nicholas Lenzen, Amogh Raut, Andrew Melnik

机构 * Bielefeld University（比勒费尔德大学） ； Indian Institute of Technology BHU（印度理工学院（ varanasi ）） ； Bremen University（不来梅大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Comments Accepted at CoRL 2024: Workshop on Lifelong Learning for Home Robots

Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?

Marco Gaido, Sara Papi, Matteo Negri, Luisa Bentivogli

机构 * Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments Outstanding paper at the ACL 2024 main conference

Learning Spatially-Aware Language and Audio Embeddings

Bhavika Devnani, Skyler Seto, Zakaria Aldeneh, Alessandro Toso, Elena Menyaylenko, Barry-John Theobald, Jonathan Sheaffer, Miguel Sarabia

机构 * Georgia Institute of Technology（佐治亚理工学院） ； Apple（苹果公司） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments 26 pages, 7 figures, accepted at NeurIPS 2024

in-Car Biometrics (iCarB) Datasets for Driver Recognition: Face, Fingerprint, and Voice

Vedrana Krivokuca Hahn, Jeremy Maceiras, Alain Komaty, Philip Abbet, Sebastien Marcel

机构 * Idiap Research Institute（伊迪亚普研究所） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments 8 pages, 13 figures, 4 tables

VioPose: Violin Performance 4D Pose Estimation by Hierarchical Audiovisual Inference

Seong Jong Yoo, Snehesh Shrestha, Irina Muresanu, Cornelia Fermüller

机构 * University of Maryland, College Park（马里兰大学帕克分校） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments Accepted by WACV 2025 in Round 1. First two authors contributed equally

Proceedings of the 6th International Workshop on Reading Music Systems

Jorge Calvo-Zaragoza, Alexander Pacha, Elona Shatri

机构 * University of Alicante（阿利坎特大学） ； TU Wien（维也纳技术大学） ； Queen Mary University of London（伦敦大学玛丽皇后学院） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments Proceedings edited by Jorge Calvo-Zaragoza, Alexander Pacha and Elona Shatri

Speech2UnifiedExpressions: Synchronous Synthesis of Co-Speech Affective Face and Body Expressions from Affordable Inputs

Uttaran Bhattacharya, Aniket Bera, Dinesh Manocha

机构 * Adobe Inc.（奥多比公司） ； Purdue University（普渡大学） ； University of Maryland（马里兰大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments 14 pages, 7 figures, 2 tables

Journal ref In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 1st Workshop on Human Motion Generation, 2024, Seattle, Washington, USA

Relevance-guided Audio Visual Fusion for Video Saliency Prediction

Li Yu, Xuanzhe Sun, Pan Gao, Moncef Gabbouj

机构 * School of Computer Science, Nanjing University of Information Science and Technology（南京信息工程大学计算机科学学院） ； Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology（南京信息工程大学江苏省大气环境与装备技术协同创新中心） ； College of Computer and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

MEEG and AT-DGNN: Improving EEG Emotion Recognition with Music Introducing and Graph-based Learning

Minghao Xiao, Zhengxi Zhu, Kang Xie, Bin Jiang

机构 * Shenzhen Research Institute of Shandong University（山东大学深圳研究院） ； Key Lab of Information Network Security Ministry of Public Security（公安部信息网络安全重点实验室） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

V2A-Mark: Versatile Deep Visual-Audio Watermarking for Manipulation Localization and Copyright Protection

Xuanyu Zhang, Youmin Xu, Runyi Li, Jiwen Yu, Weiqi Li, Zhipei Xu, Jian Zhang

机构 * Peking University Shenzhen Graduate School（北京大学深圳研究生院） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV

Comments Accepted by ACM MM 2024

Towards Vision Mixture of Experts for Wildlife Monitoring on the Edge

Emmanuel Azuh Mensah, Anderson Lee, Haoran Zhang, Yitong Shan, Kurtis Heimerl

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models

Ming Shan Hee, Shivam Sharma, Rui Cao, Palash Nandi, Preslav Nakov, Tanmoy Chakraborty, Roy Ka-Wei Lee

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments Accepted at EMNLP'24 (Findings)

Large Language Model-assisted Speech and Pointing Benefits Multiple 3D Object Selection in Virtual Reality

Junlong Chen, Jens Grubert, Per Ola Kristensson

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments under review

Turn-by-Turn Indoor Navigation for the Visually Impaired

Santosh Srinivasaiah, Sai Kumar Nekkanti, Rohith Reddy Nedhunuri

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach

Cristina Palmero, Mikel deVelasco, Mohamed Amine Hmani, Aymen Mtibaa, Leila Ben Letaifa, Pau Buch-Cardona, Raquel Justo, Terry Amorese, Eduardo González-Fraile, Begoña Fernández-Ruanova, Jofre Tenorio-Laranga, Anna Torp Johansen, Micaela Rodrigues da Silva, Liva Jenny Martinussen, Maria Stylianou Korsnes, Gennaro Cordasco, Anna Esposito, Mounim A. El-Yacoubi, Dijana Petrovska-Delacrétaz, M. Inés Torres, Sergio Escalera

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Comments This work has been submitted to the IEEE for possible publication

Dynamic Hand Gesture-Featured Human Motor Adaptation in Tool Delivery using Voice Recognition

Haolin Fei, Stefano Tedeschi, Yanpei Huang, Andrew Kennedy, Ziwei Wang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments This work has been submitted to the IEEE for possible publication

Voice-Enabled AI Agents can Perform Common Scams

Richard Fang, Dylan Bowman, Daniel Kang

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

PAT: Parameter-Free Audio-Text Aligner to Boost Zero-Shot Audio Classification

Ashish Seth, Ramaneswaran Selvakumar, Sonal Kumar, Sreyan Ghosh, Dinesh Manocha

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments 18 pages

CLIP-VAD: Exploiting Vision-Language Models for Voice Activity Detection

Andrea Appiani, Cigdem Beyan

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotation

Sen Fang, Sizhou Chen, Yalin Feng, Xiaofeng Zhang, Teik Toe Teoh

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Zero-Shot Sing Voice Conversion: built upon clustering-based phoneme representations

Wangjin Zhou, Fengrun Zhang, Yiming Liu, Wenhao Guan, Yi Zhao, Tatsuya Kawahara

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 eess.AS

Latent CLAP Loss for Better Foley Sound Synthesis

Tornike Karchkhadze, Hassan Salami Kavaki, Mohammad Rasool Izadi, Bryce Irvin, Mikolaj Kegler, Ari Hertz, Shuo Zhang, Marko Stamenovic

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Journal ref EUSIPCO 2024 Proceedings, ISBN: 978-9-4645-9361-7

The USTC-NERCSLIP Systems for the CHiME-8 MMCSG Challenge

Ya Jiang, Hongbo Lan, Jun Du, Qing Wang, Shutong Niu

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions

Leena Mathur, Paul Pu Liang, Louis-Philippe Morency

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments EMNLP 2024 Main Conference