多模态大模型

LiDAR-based Object Detection with Real-time Voice Specifications

Anurag Kulkarni

机构 * Shivaji University（希瓦吉大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments 10 pages, 4 figures, submitted as part of MSc research

DuplexMamba: Enhancing Real-time Speech Conversations with Duplex and Streaming Capabilities

Xiangyu Lu, Wang Xu, Haoyu Wang, Hongyun Zhou, Haiyan Zhao, Conghui Zhu, Tiejun Zhao, Muyun Yang

机构 * Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部） ； Tsinghua University（清华大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments 12 pages, 6 figures

Who Owns the Output? Bridging Law and Technology in LLMs Attribution

Emanuele Mezzi, Asimina Mertzani, Michael P. Manis, Siyanna Lilova, Nicholas Vadivoulis, Stamatis Gatirdakis, Styliani Roussou, Rodayna Hmede

机构 * Ethikon Institute（Ethikon研究所） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments 20 pages, 1 figure

Exploring In-Context Learning Capabilities of ChatGPT for Pathological Speech Detection

Mahdi Amiri, Hatef Otroshi Shahreza, Ina Kodrasi

机构 * Idiap Research Institute（Idiap研究所） ； École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Comments submitted to EUSIPCO 2025

OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs

John Murzaku, Owen Rambow

机构 * Stony Brook University（石溪大学） ； Institute for Advanced Computational Science（高级计算科学研究院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments Submitted to COLM 2025. Preprint

Your voice is your voice: Supporting Self-expression through Speech Generation and LLMs in Augmented and Alternative Communication

Yiwen Xu, Monideep Chakraborti, Tianyi Zhang, Katelyn Eng, Aanchan Mohan, Mirjana Prpa

机构 * Northeastern University（东北大学） ； Khoury College of Computer Sciences（科里计算机学院） ； Mercury Speech & Language（水星言语与语言机构） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

AI-Powered Assistive Technologies for Visual Impairment

Prudhvi Naayini, Praveen Kumar Myakala, Chiranjeevi Bura, Anil Kumar Jonnalagadda, Srikanth Kamatala

机构 * University of Colorado Boulder（科罗拉多大学博尔德分校） ； Google LLC（谷歌公司） ； Klein Tools（克莱因工具公司） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Exploring Causality for HRI: A Case Study on Robotic Mental Well-being Coaching

Micol Spitale, Srikar Babu, Serhan Cakmak, Jiaee Cheong, Hatice Gunes

机构 * University of Cambridge（剑桥大学） ； Politecnico di Milano（米兰理工大学） ； Indian Institute of Technology Madras（马德拉斯印度理工学院） ； Bogazici University（博阿齐奇大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Long-Video Audio Synthesis with Multi-Agent Collaboration

Yehang Zhang, Xinli Xu, Xiaojie Xu, Li Liu, Yingcong Chen

机构 * HKUST(GZ)（香港科技大学（广州）） ； HKUST（香港科技大学） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

StyleSpeaker: Audio-Enhanced Fine-Grained Style Modeling for Speech-Driven 3D Facial Animation

An Yang, Chenyu Liu, Pengcheng Xia, Jun Du

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.MM

LatentSync: Taming Audio-Conditioned Latent Diffusion Models for Lip Sync with SyncNet Supervision

Chunyu Li, Chao Zhang, Weikai Xu, Jingyu Lin, Jinghui Xie, Weiguo Feng, Bingyue Peng, Cunjian Chen, Weiwei Xing

机构 * ByteDance（字节跳动） ； Beijing Jiaotong University（北京交通大学） ； Monash University（莫纳什大学） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV

MagicInfinite: Generating Infinite Talking Videos with Your Words and Voice

Hongwei Yi, Tian Ye, Shitong Shao, Xuancheng Yang, Jiantong Zhao, Hanzhong Guo, Terrance Wang, Qingyu Yin, Zeke Xie, Lei Zhu, Wei Li, Michael Lingelbach, Daquan Zhou

机构 * Hedra Inc.（Hedra公司） ； Peking University（北京大学） ； HKUST(GZ)（香港科技大学（广州）） ； HKU（香港大学） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV

Comments MagicInfinite is publicly accessible at https://www.hedra.com/. More examples are at https://magicinfinite.github.io/

LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM

Sambal Shikhar, Mohammed Irfan Kurpath, Sahal Shaji Mullappilly, Jean Lahoud, Fahad Khan, Rao Muhammad Anwer, Salman Khan, Hisham Cholakkal

机构 * Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学（MBZUAI）） ； Linköping University（林雪平大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Recognition of Dysarthria in Amyotrophic Lateral Sclerosis patients using Hypernetworks

Loukas Ilias, Dimitris Askounis

机构 * School of ECE, NTUA（希腊国立雅典理工大学电气与计算机工程学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Co-creation for Sign Language Processing and Machine Translation

Lisa Lepp, Dimitar Shterionov, Mirella De Sisto, Grzegorz Chrupała

机构 * 1 Lisa Lepp: 2 Dimitar Shterionov: 3 Mirella De Sisto: 4 Grzegorz Chrupa a 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL

Comments Submitted to the MDPI special issue "Human and Machine Translation: Recent Trends and Foundations"

Megrez-Omni Technical Report

Boxun Li, Yadong Li, Zhiyuan Li, Congyi Liu, Weilin Liu, Guowei Niu, Zheyue Tan, Haiyang Xu, Zhuyu Yao, Tao Yuan, Dong Zhou, Yueqing Zhuang, Shengen Yan, Guohao Dai, Yu Wang

机构 * Infinigence-AI（智源人工智能研究院） ； Shanghai Jiao Tong University（上海交通大学） ； Tsinghua University（清华大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interaction

Snehesh Shrestha, Yantian Zha, Saketh Banagiri, Ge Gao, Yiannis Aloimonos, Cornelia Fermüller

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments arXiv admin note: substantial text overlap with arXiv:2403.02274

Journal ref 2025 20th ACM/IEEE International Conference on Human-Robot Interaction (HRI)

Toward a Dialogue System Using a Large Language Model to Recognize User Emotions with a Camera

Hiroki Tanioka, Tetsushi Ueta, Masahiko Sano

机构 * Tokushima University（德岛大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments 4 pages, 5 figures, 1 table, The 1st InterAI: Interactive AI for Human-Centered Robotics workshop in conjunction with IEEE Ro-MAN 2024, Pasadona, LA, USA, Aug. 2024

Journal ref The 1st InterAI Workshop: Interactive AI for Human-centered Robotics, 2024

Acoustic Prompt Tuning: Empowering Large Language Models with Audition Capabilities

Jinhua Liang, Xubo Liu, Wenwu Wang, Mark D. Plumbley, Huy Phan, Emmanouil Benetos

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Published at IEEE Transactions on Audio, Speech and Language Processing

Journal ref IEEE Transactions on Audio, Speech and Language Processing (2025)

AudioSpa: Spatializing Sound Events with Text

Linfeng Feng, Lei Zhao, Boyu Zhu, Xiao-Lei Zhang, Xuelong Li

机构 * School of Marine Science and Technology, Northwestern Polytechnical University（西北工业大学海洋科学与技术学院） ； Institute of Artificial Intelligence (TeleAI), China Telecom（中国电信人工智能研究院（TeleAI）） ； Research and Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 eess.AS

Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesis

Weiwei Lin, Chenghan He

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

Comments ICLR 2025

Journal ref ICLR 2025

Foundation Models for CPS-IoT: Opportunities and Challenges

Ozan Baris, Yizhuo Chen, Gaofeng Dong, Liying Han, Tomoyoshi Kimura, Pengrui Quan, Ruijie Wang, Tianchen Wang, Tarek Abdelzaher, Mario Bergés, Paul Pu Liang, Mani Srivastava

机构 * CMU（卡内基梅隆大学） ； UIUC（伊利诺伊大学厄巴纳-香槟分校） ； UCLA（加利福尼亚大学洛杉矶分校） ； MIT（麻省理工学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Adapter-Based Multi-Agent AVSR Extension for Pre-Trained ASR Models

Christopher Simic, Korbinian Riedhammer, Tobias Bocklet

机构 * Technische Hochschule Nuernberg（纽伦堡应用技术大学） 纠错

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments Accepted at ICASSP 2025

Divergent Emotional Patterns in Disinformation on Social Media? An Analysis of Tweets and TikToks about the DANA in Valencia

Iván Arcos, Paolo Rosso, Ramón Salaverría

机构 * PRHLT Research Center, Universitat Politècnica de València（瓦伦西亚理工大学PRHLT研究中心） ； ValgrAI Valencian Graduate School and Research Network of Artificial Intelligence（瓦伦西亚ValgrAI人工智能研究生学院及研究网络） ； School of Communication, Universidad de Navarra（纳瓦拉大学传播学院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Journal ref Proceedings of the 17th International Conference on Agents and Artificial Intelligence (ICAART 2025), Porto, Portugal, February 23-25, 2025

Semantic Web and Creative AI -- A Technical Report from ISWS 2023

Raia Abu Ahmad, Reham Alharbi, Roberto Barile, Martin Böckling, Francisco Bolanos, Sara Bonfitto, Oleksandra Bruns, Irene Celino, Yashrajsinh Chudasama, Martin Critelli, Claudia d'Amato, Giada D'Ippolito, Ioannis Dasoulas, Stefano De Giorgis, Vincenzo De Leo, Chiara Di Bonaventura, Marco Di Panfilo, Daniil Dobriy, John Domingue, Xuemin Duan, Michel Dumontier, Sefika Efeoglu, Ruben Eschauzier, Fakih Ginwa, Nicolas Ferranti, Arianna Graciotti, Philipp Hanisch, George Hannah, Golsa Heidari, Aidan Hogan, Hassan Hussein, Alexane Jouglar, Jan-Christoph Kalo, Manoé Kieffer, Antonis Klironomos, Inês Koch, Weronika Lajewska, Nicolas Lazzari, Mikael Lindekrans, Anna Sofia Lippolis, Majlinda Llugiqi, Eleonora Mancini, Eleonora Marzi, Laura Menotti, Daniela Milon Flores, Soulakshmee Nagowah, Kerstin Neubert, Emetis Niazmand, Ebrahim Norouzi, Beatriz Olarte Martinez, Anouk Michelle Oudshoorn, Andrea Poltronieri, Valentina Presutti, Disha Purohit, Ensiyeh Raoufi, Celian Ringwald, Johanna Rockstroh, Sebastian Rudolph, Harald Sack, Zafar Saeed, Mohammad Javad Saeedizade, Aya Sahbi, Cristian Santini, Aleksandra Simic, Dennis Sommer, Rita Sousa, Mary Ann Tan, Vidyashree Tarikere, Tabea Tietz, Liam Tirpitz, Arnaldo Tomasino, Frank van Harmelen, Joao Vissoci, Caitlin Woods, Bohui Zhang, Xinyue Zhang, Heng Zheng

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments Technical Report

Exploring GPT's Ability as a Judge in Music Understanding

Kun Fang, Ziyu Wang, Gus Xia, Ichiro Fujinaga

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 eess.AS

MoEE: Mixture of Emotion Experts for Audio-Driven Portrait Animation

Huaize Liu, Wenzhang Sun, Donglin Di, Shibo Sun, Jiahui Yang, Changqing Zou, Hujun Bao

机构 * Zhejiang Lab（之江实验室） ； Li Auto（理想汽车） ； Harbin Institute of Technology（哈尔滨工业大学） ； Zhejiang University（浙江大学） ； Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Aesthetic Matters in Music Perception for Image Stylization: A Emotion-driven Music-to-Visual Manipulation

Junjie Xu, Xingjiao Wu, Tanren Yao, Zihao Zhang, Jiayang Bei, Wu Wen, Liang He

机构 * East China Normal University（华东师范大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV

Decoding the Flow: CauseMotion for Emotional Causality Analysis in Long-form Conversations

Yuxuan Zhang, Yulong Li, Zichen Yu, Feilong Tang, Zhixiang Lu, Chong Li, Kang Dang, Jionglong Su

机构 * School of Artificial Intelligence and Advanced Computing, Xi’an Jiaotong-Liverpool University（西交利物浦大学人工智能与先进计算学院） ； Monash University（莫纳什大学） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL

Comments 7pages

Language-based Audio Retrieval with Co-Attention Networks

Haoran Sun, Zimu Wang, Qiuyi Chen, Jianjun Chen, Jia Wang, Haiyang Zhang

机构 * School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西交利物浦大学先进技术学院） 纠错

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 eess.AS

Comments Accepted at UIC 2024 proceedings. Accepted version