多模态大模型

Amharic LLaMA and LLaVA: Multimodal LLMs for Low Resource Languages

Michael Andersland

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

A Modular End-to-End Multimodal Learning Method for Structured and Unstructured Data

Marco D Alessandro, Enrique Calabrés, Mikel Elkano

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments 8 pages, 1 figure

AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation

Yasheng Sun, Wenqing Chu, Hang Zhou, Kaisiyuan Wang, Hideki Koike

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning

Xiwen Li, Tristalee Mangin, Surojit Saha, Evan Blanchard, Dillon Tang, Henry Poppe, Nathan Searle, Ouk Choi, Kerry Kelly, Ross Whitaker

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models

Jiawei Liang, Siyuan Liang, Man Luo, Aishan Liu, Dongchen Han, Ee-Chien Chang, Xiaochun Cao

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

MMMModal -- Multi-Images Multi-Audio Multi-turn Multi-Modal

Husein Zolkepli, Aisyah Razak, Kamarul Adha, Ariff Nazhan

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.CL

MasonPerplexity at Multimodal Hate Speech Event Detection 2024: Hate Speech and Target Detection Using Transformer Ensembles

Amrita Ganguly, Al Nahian Bin Emran, Sadiya Sayara Chowdhury Puspo, Md Nishat Raihan, Dhiman Goswami, Marcos Zampieri

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Overview of the L3DAS23 Challenge on Audio-Visual Extended Reality

Christian Marinoni, Riccardo Fosco Gramaccioni, Changan Chen, Aurelio Uncini, Danilo Comminiello

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted to 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)

Multi-Modal Emotion Recognition by Text, Speech and Video Using Pretrained Transformers

Minoo Shayaninasab, Bagher Babaali

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Multimodal Attention Merging for Improved Speech Recognition and Audio Event Classification

Anirudh S. Sundar, Chao-Han Huck Yang, David M. Chan, Shalini Ghosh, Venkatesh Ravichandran, Phani Sankar Nidadavolu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments 5 pages, 1 figure, ICASSP 2024 Workshop on Self-supervision in Audio, Speech and Beyond

An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition

Rongfan Liao, Siyang Song, Hatice Gunes

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted by IEEE Transactions on Affective Computing

Binding Touch to Everything: Learning Unified Multimodal Tactile Representations

Fengyu Yang, Chao Feng, Ziyang Chen, Hyoungseob Park, Daniel Wang, Yiming Dou, Ziyao Zeng, Xien Chen, Rit Gangopadhyay, Andrew Owens, Alex Wong

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

MM-TTS: Multi-modal Prompt based Style Transfer for Expressive Text-to-Speech Synthesis

Wenhao Guan, Yishuang Li, Tao Li, Hukai Huang, Feng Wang, Jiayan Lin, Lingyan Huang, Lin Li, Qingyang Hong

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 eess.AS

Comments Accepted at AAAI2024

Multi-Modal Gaze Following in Conversational Scenarios

Yuqi Hou, Zhongqun Zhang, Nora Horanyi, Jaewon Moon, Yihua Cheng, Hyung Jin Chang

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Reading Between the Frames: Multi-Modal Depression Detection in Videos from Non-Verbal Cues

David Gimeno-Gómez, Ana-Maria Bucur, Adrian Cosma, Carlos-David Martínez-Hinarejos, Paolo Rosso

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments Accepted at 46th European Conference on Information Retrieval (ECIR 2024)

Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy

Weijian Mai, Jian Zhang, Pengfei Fang, Zhijun Zhang

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

SDIF-DA: A Shallow-to-Deep Interaction Framework with Data Augmentation for Multi-modal Intent Detection

Shijue Huang, Libo Qin, Bingbing Wang, Geng Tu, Ruifeng Xu

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL

Comments Accepted by ICASSP 2024

Multimodal Classification of Teaching Activities from University Lecture Recordings

Oscar Sapena, Eva Onaindia

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments 18 pages

Journal ref Appl. Sci. 2022, 12, 4785

Improving Audio-Visual Segmentation with Bidirectional Generation

Dawei Hao, Yuxin Mao, Bowen He, Xiaodong Han, Yuchao Dai, Yiran Zhong

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments AAAI Camera Ready. Dawei Hao and Yuxin Mao contribute equality to this paper. Yiran Zhong is the corresponding author. The code will be released at https://github.com/OpenNLPLab/AVS-bidirectional

Hyperbolic Audio-visual Zero-shot Learning

Jie Hong, Zeeshan Hayder, Junlin Han, Pengfei Fang, Mehrtash Harandi, Lars Petersson

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments ICCV 2023

Hourglass-AVSR: Down-Up Sampling-based Computational Efficiency Model for Audio-Visual Speech Recognition

Fan Yu, Haoxu Wang, Ziyang Ma, Shiliang Zhang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments Accepted by ICASSP 2024

Cem Mil Podcasts: A Spoken Portuguese Document Corpus For Multi-modal, Multi-lingual and Multi-Dialect Information Access Research

Ekaterina Garmash, Edgar Tanaka, Ann Clifton, Joana Correia, Sharmistha Jat, Winstead Zhu, Rosie Jones, Jussi Karlgren

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL

Comments 12 pages, 1 figure

Journal ref Volume 14163 of Lecture Notes in Computer Science, pages 48-59, Springer, 2023

Joint Training or Not: An Exploration of Pre-trained Speech Models in Audio-Visual Speaker Diarization

Huan Zhao, Li Zhang, Yue Li, Yannan Wang, Hongji Wang, Wei Rao, Qing Wang, Lei Xie

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Multimodal Data and Resource Efficient Device-Directed Speech Detection with Large Foundation Models

Dominik Wagner, Alexander Churchill, Siddharth Sigtia, Panayiotis Georgiou, Matt Mirsamadi, Aarshee Mishra, Erik Marchi

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Multimodal Speech Emotion Recognition Using Modality-specific Self-Supervised Frameworks

Rutherford Agbeshi Patamia, Paulo E. Santos, Kingsley Nketia Acheampong, Favour Ekong, Kwabena Sarpong, She Kun

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

THInImg: Cross-modal Steganography for Presenting Talking Heads in Images

Lin Zhao, Hongxuan Li, Xuefei Ning, Xinru Jiang

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CV

Comments Accepted at WACV 2024

A Unified Framework for Multimodal, Multi-Part Human Motion Synthesis

Zixiang Zhou, Yu Wan, Baoyuan Wang

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments 19 pages, 18 figures

Centre Stage: Centricity-based Audio-Visual Temporal Action Detection

Hanyuan Wang, Majid Mirmehdi, Dima Damen, Toby Perrett

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted to VUA workshop at BMVC 2023

EPG2S: Speech Generation and Speech Enhancement based on Electropalatography and Audio Signals using Multimodal Learning

Li-Chin Chen, Po-Hsun Chen, Richard Tzong-Han Tsai, Yu Tsao

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Accepted By IEEE Signal Processing Letter

Journal ref IEEE Signal Processing Letters, vol. 29, p. 2582-2586, 2022

Multimodal Large Language Models: A Survey

Jiayang Wu, Wensheng Gan, Zefeng Chen, Shicheng Wan, Philip S. Yu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Comments IEEE BigData 2023. 10 pages