多模态大模型

OLKAVS: An Open Large-Scale Korean Audio-Visual Speech Dataset

Jeongkyun Park, Jung-Wook Hwang, Kwanghee Choi, Seung-Hyun Lee, Jun Hwan Ahn, Rae-Hong Park, Hyung-Min Park

机构 * 1 Department of Artificial Intelligence, Sogang University, Seoul 04107, Republic of Korea 2 Department of Electronic Engineering, Sogang University, Seoul 04107, Republic of Korea 3 Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA 4 Mindslab Inc., Gyeonggi-do 13493, Republic of Korea 5 ICT Convergence Disaster/Safety Research Institute, Sogang University, Seoul 04107, Republic of Korea 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multi-modal(abstract)；分类 cs.CV、cs.CL、cs.AI

Comments Accepted to ICASSP 2024

Seeing is Believing: Emotion-Aware Audio-Visual Language Modeling for Expressive Speech Generation

Weiting Tan, Jiachen Lian, Hirofumi Inaguma, Paden Tomasello, Philipp Koehn, Xutai Ma

机构 * Johns Hopkins University（约翰霍普金斯大学） ； Meta AI Research（Meta AI 研究） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；multimodal(abstract)；分类 cs.CV、cs.CL、cs.MM

Comments EMNLP 2025 (Findings)

MM-HSD: Multi-Modal Hate Speech Detection in Videos

Berta Céspedes-Sarrias, Carlos Collado-Capell, Pablo Rodenas-Ruiz, Olena Hrynenko, Andrea Cavallaro

机构 * EPFL（苏黎世联邦理工学院） ； Idiap Research Institute（日内瓦研究所） 纠错

专题命中 音频语音多模态 ：multi-modal(title,abstract)；cross-modal(abstract)；分类 cs.AI、cs.MM

Comments Accepted at ACM Multimedia 2025

Progressive Semantic Residual Quantization for Multimodal-Joint Interest Modeling in Music Recommendation

Shijia Wang, Tianpei Ouyang, Qiang Xiao, Dongjing Wang, Yintao Ren, Songpei Xu, Da Guo, Chuanjiang Luo

专题命中 音频语音多模态 ：multimodal(title,abstract)；cross-modal(abstract)

Exploring Machine Learning and Language Models for Multimodal Depression Detection

Javier Si Zhao Hong, Timothy Zoe Delaya, Sherwyn Chan Yin Kit, Pai Chet Ng, Xiaoxiao Miao

机构 * Singapore Institute of Technology（新加坡理工学院） ； Duke Kunshan University（杜克-昆山大学） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments This paper has been accepted by APCIPA ASC 2025

MoTAS: MoE-Guided Feature Selection from TTS-Augmented Speech for Enhanced Multimodal Alzheimer's Early Screening

Yongqi Shao, Binxin Mei, Cong Tan, Hong Huo, Tao Fang

机构 * Shanghai Jiao Tong University（上海交通大学） 纠错

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Spherical Vision Transformers for Audio-Visual Saliency Prediction in 360-Degree Videos

Mert Cokelek, Halit Ozsoy, Nevrez Imamoglu, Cagri Ozcinar, Inci Ayhan, Erkut Erdem, Aykut Erdem

机构 * Department of Computer Science and Engineering, Koç University（计算机科学与工程系，科克大学） ； Department of Psychology, Boğaziçi University（心理学系，博多伊大学） ； National Institute of Advanced Industrial Science and Technology (AIST), Intelligent Platforms Research Institute（国家先进工业科学与技术研究院（AIST），智能平台研究机构） ； Department of Psychology, Boğaziçi University University（心理学系，博多伊大学） ； Department of Computer Engineering, Hacettepe University（计算机工程系，哈切塞特佩大学） ； KUIS AI Center（KUIS人工智能中心） 纠错

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted for publication in IEEE Transaction on Pattern Analysis and Machine Intelligence (IEEE TPAMI)

Speech Emotion Recognition via Entropy-Aware Score Selection

ChenYi Chua, JunKai Wong, Chengxin Chen, Xiaoxiao Miao

机构 * Singapore Institute of Technology（新加坡理工学院） ； Institute Of Acoustics, Chinese Academy Of Sciences（中国科学院声学研究所） ； Duke Kunshan University（杜克大学昆山分校） 纠错

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI

Comments The paper has been accepted by APCIPA ASC 2025

A Solution of Ultra Wideband Based High-resolution and Lossless Audio Transmission

Fengyun Zhang

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 eess.AS

Comments 5 pages

Modality-Specific Speech Enhancement and Noise-Adaptive Fusion for Acoustic and Body-Conduction Microphone Framework

Yunsik Kim, Yoonyoung Chung

机构 * Department of Electrical Engineering（电气工程系） ； Department of Semiconductor Engineering（半导体工程系） ； Center for Semiconductor Technology Convergence（半导体技术融合中心） 纠错

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI

Journal ref Interspeech 2025