多模态大模型

Gumbel Rao Monte Carlo based Bi-Modal Neural Architecture Search for Audio-Visual Deepfake Detection

Aravinda Reddy PN, Raghavendra Ramachandra, Krothapalli Sreenivasa Rao, Pabitra Mitra Vinod Rathod

专题命中 音频语音多模态 ：audio-visual(title)；multimodal(abstract)；分类 eess.AS

Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond

Soyeon Caren Han, Feiqi Cao, Josiah Poon, Roberto Navigli

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments Accepted at ACM-MM 2024

MINER: Mining the Underlying Pattern of Modality-Specific Neurons in Multimodal Large Language Models

Kaichen Huang, Jiahao Huo, Yibo Yan, Kun Wang, Yutao Yue, Xuming Hu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Identifiable Shared Component Analysis of Unpaired Multimodal Mixtures

Subash Timilsina, Sagar Shrestha, Xiao Fu

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.AI

Robust Audio-Visual Speech Enhancement: Correcting Misassignments in Complex Environments with Advanced Post-Processing

Wenze Ren, Kuo-Hsuan Hung, Rong Chao, YouJin Li, Hsin-Min Wang, Yu Tsao

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments The 27th International Conference of the Oriental COCOSDA

StyleFusion TTS: Multimodal Style-control and Enhanced Feature Fusion for Zero-shot Text-to-speech Synthesis

Zhiyong Chen, Xinnuo Li, Zhiqi Ai, Shugong Xu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments The 7th Chinese Conference on Pattern Recognition and Computer Vision PRCV 2024

Journal ref The 7th Chinese Conference on Pattern Recognition and Computer Vision PRCV 2024

A Multimodal Dense Retrieval Approach for Speech-Based Open-Domain Question Answering

Georgios Sidiropoulos, Evangelos Kanoulas

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

MHS-STMA: Multimodal Hate Speech Detection via Scalable Transformer-Based Multilevel Attention Framework

Anusha Chhabra, Dinesh Kumar Vishwakarma

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

MRAC Track 1: 2nd Workshop on Multimodal, Generative and Responsible Affective Computing

Shreya Ghosh, Zhixi Cai, Abhinav Dhall, Dimitrios Kollias, Roland Goecke, Tom Gedeon

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Comments ACM MM Workshop 2024. Workshop webpage: https://react-ws.github.io/2024/

Safe Multimodal Communication in Human-Robot Collaboration

Davide Ferrari, Andrea Pupa, Alberto Signoretti, Cristian Secchi

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.AI

Journal ref Human-Friendly Robotics 2023

How Do You Perceive My Face? Recognizing Facial Expressions in Multi-Modal Context by Modeling Mental Representations

Florian Blume, Runfeng Qu, Pia Bideau, Martin Maier, Rasha Abdel Rahman, Olaf Hellwich

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV

Comments GCPR 2024

AVIN-Chat: An Audio-Visual Interactive Chatbot System with Emotional State Tuning

Chanhyuk Park, Jungbin Cho, Junwan Kim, Seongmin Lee, Jungsu Kim, Sanghoon Lee

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.AI

Journal ref Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI) Demo Track. 8763-8766 (2024)

A Comprehensive Review and Taxonomy of Audio-Visual Synchronization Techniques for Realistic Speech Animation

Jose Geraldo Fernandes, Sinval Nascimento, Daniel Dominguete, André Oliveira, Lucas Rotsen, Gabriel Souza, David Brochero, Luiz Facury, Mateus Vilela, Hebert Costa, Frederico Coelho, Antônio P. Braga

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Trust and ethical considerations in a multi-modal, explainable AI-driven chatbot tutoring system: The case of collaboratively solving Rubik's Cube

Kausik Lakkaraju, Vedant Khandelwal, Biplav Srivastava, Forest Agostinelli, Hengtao Tang, Prathamjeet Singh, Dezhi Wu, Matt Irvin, Ashish Kundu

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.AI

Comments Accepted at 'Neural Conversational AI Workshop - What's left to TEACH (Trustworthy, Enhanced, Adaptable, Capable, and Human-centric) chatbots?' at ICML 2023

MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controls

Yuxuan Bian, Ailing Zeng, Xuan Ju, Xian Liu, Zhaoyang Zhang, Wei Liu, Qiang Xu

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV

Exploring the Role of Audio in Multimodal Misinformation Detection

Moyang Liu, Yukun Liu, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Xuefei Liu, Guanjun Li

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey

Hao Yang, Yanyan Zhao, Yang Wu, Shilong Wang, Tian Zheng, Hongbo Zhang, Zongyang Ma, Wanxiang Che, Bing Qin

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL

Comments arXiv admin note: text overlap with arXiv:2210.14556 by other authors

SaSLaW: Dialogue Speech Corpus with Audio-visual Egocentric Information Toward Environment-adaptive Dialogue Speech Synthesis

Osamu Take, Shinnosuke Takamichi, Kentaro Seki, Yoshiaki Bando, Hiroshi Saruwatari

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 eess.AS

Comments 5 pages, accepted for INTERSPEECH 2024

Synchronous Multi-modal Semantic Communication System with Packet-level Coding

Yun Tian, Jingkai Ying, Zhijin Qin, Ye Jin, Xiaoming Tao

专题命中 音频语音多模态 ：multi-modal(title)；multimodal(abstract)；分类 cs.AI

Comments 12 pages, 9 figures

Spherical World-Locking for Audio-Visual Localization in Egocentric Videos

Heeseung Yun, Ruohan Gao, Ishwarya Ananthabhotla, Anurag Kumar, Jacob Donley, Chao Li, Gunhee Kim, Vamsi Krishna Ithapu, Calvin Murdock

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments ECCV2024

EMID: An Emotional Aligned Dataset in Audio-Visual Modality

Jialing Zou, Jiahao Mei, Guangze Ye, Tianyu Huai, Qiwei Shen, Daoguo Dong

专题命中 音频语音多模态 ：audio-visual(title)；cross-modal(abstract)；分类 cs.MM

Comments Accepted by ACM MM workshop McGE 2023: Proceedings of the 1st International Workshop on Multimedia Content Generation and Evaluation: New Methods and Practice

MaskAnyone Toolkit: Offering Strategies for Minimizing Privacy Risks and Maximizing Utility in Audio-Visual Data Archiving

Babajide Alamu Owoyele, Martin Schilling, Rohan Sawahn, Niklas Kaemer, Pavel Zherebenkov, Bhuvanesh Verma, Wim Pouw, Gerard de Melo

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.MM

Television Discourse Decoded: Comprehensive Multimodal Analytics at Scale

Anmol Agarwal, Pratyush Priyadarshi, Shiven Sinha, Shrey Gupta, Hitkul Jangra, Ponnurangam Kumaraguru, Kiran Garimella

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM

Comments KDD 2024 [Updates for Camera Ready version]

AVESFormer: Efficient Transformer Design for Real-Time Audio-Visual Segmentation

Zili Wang, Qi Yang, Linsu Shi, Jiazhong Yu, Qinghua Liang, Fei Li, Shiming Xiang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Double Multi-Head Attention Multimodal System for Odyssey 2024 Speech Emotion Recognition Challenge

Federico Costa, Miquel India, Javier Hernando

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments Odyssey 2024: The Speaker and Language Recognition Workshop

Journal ref Proc. The Speaker and Language Recognition Workshop (Odyssey 2024), 266-273

AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset

Zhixi Cai, Shreya Ghosh, Aman Pankaj Adatia, Munawar Hayat, Abhinav Dhall, Tom Gedeon, Kalin Stefanov

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted by ACM MM 2024

CogniVoice: Multimodal and Multilingual Fusion Networks for Mild Cognitive Impairment Assessment from Spontaneous Speech

Jiali Cheng, Mohamed Elgaar, Nidhi Vakil, Hadi Amiri

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 eess.AS

Comments INTERSPEECH 2024

Advancing Automated Deception Detection: A Multimodal Approach to Feature Extraction and Analysis

Mohamed Bahaa, Mena Hany, Ehab E. Zakaria

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV

Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognition

G Rajasekhar, Jahangir Alam

专题命中 音频语音多模态 ：audio-visual(title)；multimodal(abstract)；分类 cs.CV

Comments arXiv admin note: substantial text overlap with arXiv:2403.19554

video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models

Guangzhi Sun, Wenyi Yu, Changli Tang, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, Yuxuan Wang, Chao Zhang

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted at ICML 2024. arXiv admin note: substantial text overlap with arXiv:2310.05863