多模态大模型

PsyCounAssist: A Full-Cycle AI-Powered Psychological Counseling Assistant System

Xianghe Liu, Jiaqi Xu, Tao Sun

机构 * Beijing PsychTech Technology Co., Ltd.（北京心理科技技术有限公司） ； University of Hechi（贺州学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis

Ziqi Ni, Ao Fu, Yi Zhou

机构 * School of Computer Science and Engineering, Southeast University（计算机科学与工程学院，东南大学） ； Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, China（新一代人工智能技术及其交叉应用重点实验室，教育部，中国） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

Comments Accepted by ICMR 2025

Enhancing Audio-Language Models through Self-Supervised Post-Training with Text-Audio Pairs

Anshuman Sinha, Camille Migozzi, Aubin Rey, Chao Zhang

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments 29 pages, 15 figures

On-Device Watermarking: A Socio-Technical Imperative For Authenticity In The Age of Generative AI

Houssam Kherraz

机构 * Kensho Technologies 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.AI

Comments 10 pages, 3 figures, ICLR 2025, https://openreview.net/forum?id=ygE0U21vxM

CAFA: a Controllable Automatic Foley Artist

Roi Benita, Michael Finkelson, Tavi Halperin, Gleb Sterkin, Yossi Adi

机构 * Technion（以色列理工学院） ； Hebrew University of Jerusalem（耶路撒冷希伯来大学） ； Lightricks（丽塔克斯科技公司） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Renamed paper to "CAFA: a Controllable Automatic Foley Artist" from "Controllable Automatic Foley Artist". Updated link to demo page

A Review of Deep Learning Approaches for Non-Invasive Cognitive Impairment Detection

Muath Alsuhaibani, Ali Pourramezan Fard, Jian Sun, Farida Far Poor, Peter S. Pressman, Mohammad H. Mahoor

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Voice Interaction With Conversational AI Could Facilitate Thoughtful Reflection and Substantive Revision in Writing

Jiho Kim, Philippe Laban, Xiang 'Anthony' Chen, Kenneth C. Arnold

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Comments 5 pages; Accepted to Fourth Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2025) at NAACL 2025

Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey

Hong-Hanh Nguyen-Le, Van-Tuan Tran, Dinh-Thuc Nguyen, Nhien-An Le-Khac

机构 * University College Dublin（都柏林大学学院） ； Trinity College Dublin（都柏林圣三一学院） ； University of Science（科学大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 35 pages

LiDAR-based Object Detection with Real-time Voice Specifications

Anurag Kulkarni

机构 * Shivaji University（希瓦吉大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 4 figures, submitted as part of MSc research

DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities

Xiangyu Lu, Wang Xu, Haoyu Wang, Hongyun Zhou, Haiyan Zhao, Conghui Zhu, Tiejun Zhao, Muyun Yang

机构 * Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部） ； Tsinghua University（清华大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments 12 pages, 6 figures

Who Owns the Output? Bridging Law and Technology in LLMs Attribution

Emanuele Mezzi, Asimina Mertzani, Michael P. Manis, Siyanna Lilova, Nicholas Vadivoulis, Stamatis Gatirdakis, Styliani Roussou, Rodayna Hmede

机构 * Ethikon Institute（Ethikon研究所） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments 20 pages, 1 figure

Exploring In-Context Learning Capabilities of ChatGPT for Pathological Speech Detection

Mahdi Amiri, Hatef Otroshi Shahreza, Ina Kodrasi

机构 * Idiap Research Institute（Idiap研究所） ； École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments submitted to EUSIPCO 2025

OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs

John Murzaku, Owen Rambow

机构 * Stony Brook University（石溪大学） ； Institute for Advanced Computational Science（高级计算科学研究院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments Submitted to COLM 2025. Preprint

Your voice is your voice: Supporting Self-expression through Speech Generation and LLMs in Augmented and Alternative Communication

Yiwen Xu, Monideep Chakraborti, Tianyi Zhang, Katelyn Eng, Aanchan Mohan, Mirjana Prpa

机构 * Northeastern University（东北大学） ； Khoury College of Computer Sciences（科里计算机学院） ； Mercury Speech & Language（水星言语与语言机构） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

AI-Powered Assistive Technologies for Visual Impairment

Prudhvi Naayini, Praveen Kumar Myakala, Chiranjeevi Bura, Anil Kumar Jonnalagadda, Srikanth Kamatala

机构 * University of Colorado Boulder（科罗拉多大学博尔德分校） ； Google LLC（谷歌公司） ； Klein Tools（克莱因工具公司） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching

Micol Spitale, Srikar Babu, Serhan Cakmak, Jiaee Cheong, Hatice Gunes

机构 * University of Cambridge（剑桥大学） ； Politecnico di Milano（米兰理工大学） ； Indian Institute of Technology Madras（马德拉斯印度理工学院） ； Bogazici University（博阿齐奇大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Long-Video Audio Synthesis with Multi-Agent Collaboration

Yehang Zhang, Xinli Xu, Xiaojie Xu, Li Liu, Yingcong Chen

机构 * HKUST(GZ)（香港科技大学（广州）） ； HKUST（香港科技大学） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

StyleSpeaker: Audio-Enhanced Fine-Grained Style Modeling for Speech-Driven 3D Facial Animation

An Yang, Chenyu Liu, Pengcheng Xia, Jun Du

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.MM

LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision

Chunyu Li, Chao Zhang, Weikai Xu, Jingyu Lin, Jinghui Xie, Weiguo Feng, Bingyue Peng, Cunjian Chen, Weiwei Xing

机构 * ByteDance（字节跳动） ； Beijing Jiaotong University（北京交通大学） ； Monash University（莫纳什大学） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

MagicInfinite: Generating Infinite Talking Videos with Your Words and Voice

Hongwei Yi, Tian Ye, Shitong Shao, Xuancheng Yang, Jiantong Zhao, Hanzhong Guo, Terrance Wang, Qingyu Yin, Zeke Xie, Lei Zhu, Wei Li, Michael Lingelbach, Daquan Zhou

机构 * Hedra Inc.（Hedra公司） ； Peking University（北京大学） ； HKUST(GZ)（香港科技大学（广州）） ； HKU（香港大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments MagicInfinite is publicly accessible at https://www.hedra.com/. More examples are at https://magicinfinite.github.io/

LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM

Sambal Shikhar, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly, Jean Lahoud, Fahad Khan, Rao Muhammad Anwer, Salman Khan, Hisham Cholakkal

机构 * Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学（MBZUAI）） ； Linköping University（林雪平大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Recognition of Dysarthria in Amyotrophic Lateral Sclerosis patients using Hypernetworks

Loukas Ilias, Dimitris Askounis

机构 * School of ECE, NTUA（希腊国立雅典理工大学电气与计算机工程学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Co-creation for Sign Language Processing and Machine Translation

Lisa Lepp, Dimitar Shterionov, Mirella De Sisto, Grzegorz Chrupała

机构 * 1 Lisa Lepp: 2 Dimitar Shterionov: 3 Mirella De Sisto: 4 Grzegorz Chrupa a 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL

Comments Submitted to the MDPI special issue "Human and Machine Translation: Recent Trends and Foundations"

Megrez-Omni Technical Report

Boxun Li, Yadong Li, Zhiyuan Li, Congyi Liu, Weilin Liu, Guowei Niu, Zheyue Tan, Haiyang Xu, Zhuyu Yao, Tao Yuan, Dong Zhou, Yueqing Zhuang, Shengen Yan, Guohao Dai, Yu Wang

机构 * Infinigence-AI（智源人工智能研究院） ； Shanghai Jiao Tong University（上海交通大学） ； Tsinghua University（清华大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction

Snehesh Shrestha, Yantian Zha, Saketh Banagiri, Ge Gao, Yiannis Aloimonos, Cornelia Fermüller

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2403.02274

Journal ref 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

Toward a Dialogue System Using a Large Language Model to Recognize User Emotions with a Camera

Hiroki Tanioka, Tetsushi Ueta, Masahiko Sano

机构 * Tokushima University（德岛大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments 4 pages, 5 figures, 1 table, The 1st InterAI: Interactive AI for Human-Centered Robotics workshop in conjunction with IEEE Ro-MAN 2024, Pasadona, LA, USA, Aug. 2024

Journal ref The 1st InterAI Workshop: Interactive AI for Human-centered Robotics, 2024

Acoustic Prompt Tuning: Empowering Large Language Models with Audition Capabilities

Jinhua Liang, Xubo Liu, Wenwu Wang, Mark D. Plumbley, Huy Phan, Emmanouil Benetos

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Published at IEEE Transactions on Audio, Speech and Language Processing

Journal ref IEEE Transactions on Audio, Speech and Language Processing (2025)

AudioSpa: Spatializing Sound Events with Text

Linfeng Feng, Lei Zhao, Boyu Zhu, Xiao-Lei Zhang, Xuelong Li

机构 * School of Marine Science and Technology, Northwestern Polytechnical University（西北工业大学海洋科学与技术学院） ； Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院（TeleAI）） ； Research and Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis

Weiwei Lin, Chenghan He

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments ICLR 2025

Journal ref ICLR 2025

Foundation Models for CPS-IoT: Opportunities and Challenges

Ozan Baris, Yizhuo Chen, Gaofeng Dong, Liying Han, Tomoyoshi Kimura, Pengrui Quan, Ruijie Wang, Tianchen Wang, Tarek Abdelzaher, Mario Bergés, Paul Pu Liang, Mani Srivastava

机构 * CMU（卡内基梅隆大学） ； UIUC（伊利诺伊大学厄巴纳-香槟分校） ； UCLA（加利福尼亚大学洛杉矶分校） ； MIT（麻省理工学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI