多模态大模型

Multimodal Transformer with Pointer Network for the DSTC8 AVSD Challenge

Hung Le, Nancy F. Chen

专题命中 音频语音多模态 ：multimodal(title)；audio-visual(abstract)；分类 cs.CV、cs.CL

Comments Accepted at DSTC Workshop at AAAI 2020

$M^3$T: Multi-Modal Continuous Valence-Arousal Estimation in the Wild

Yuan-Hang Zhang, Rulin Huang, Jiabei Zeng, Shiguang Shan, Xilin Chen

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV、eess.AS

Comments 6 pages, technical report; submission to ABAW Challenge at FG 2020

Analyzing Utility of Visual Context in Multimodal Speech Recognition Under Noisy Conditions

Tejas Srinivasan, Ramon Sanabria, Florian Metze

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to How2 Workshop, ICML 2019

An Analysis of Speech Enhancement and Recognition Losses in Limited Resources Multi-talker Single Channel Audio-Visual ASR

Luca Pasa, Giovanni Morrone, Leonardo Badino

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CL、eess.AS

SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild

Jean Kossaifi, Robert Walecki, Yannis Panagakis, Jie Shen, Maximilian Schmitt, Fabien Ringeval, Jing Han, Vedhas Pandit, Antoine Toisoul, Bjorn Schuller, Kam Star, Elnar Hajiyev, Maja Pantic

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.AI

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2019

Variational Bayesian Inference for Audio-Visual Tracking of Multiple Speakers

Yutong Ban, Xavier Alameda-Pineda, Laurent Girin, Radu Horaud

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.MM

Exploring Hate Speech Detection in Multimodal Publications

Raul Gomez, Jaume Gibert, Lluis Gomez, Dimosthenis Karatzas

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

A Multi-Modal Feature Embedding Approach to Diagnose Alzheimer Disease from Spoken Language

S. Soroush Haj Zargarbashi, Bagher Babaali

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments 14 pages, 4 figures

CochleaNet: A Robust Language-independent Audio-Visual Model for Speech Enhancement

Mandar Gogate, Kia Dashtipour, Ahsan Adeel, Amir Hussain

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments 34 pages, 11 figures, Submitted to Information Fusion

My lips are concealed: Audio-visual speech enhancement through obstructions

Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to Interspeech 2019

Synchronizing Audio-Visual Film Stimuli in Unity (version 5.5.1f1): Game Engines as a Tool for Research

Javier Sanz, Andreas Wulff-Abramsson, Carlos Aguilar-Paredes, Luis Emilio Bruni, Lydia Sanchez

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.MM、eess.AS

Comments 13 Pages

Investigating the Lombard Effect Influence on End-to-End Audio-Visual Speech Recognition

Pingchuan Ma, Stavros Petridis, Maja Pantic

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted for publication at Interspeech 2019

Who said that?: Audio-visual speaker diarisation of real-world meetings

Joon Son Chung, Bong-Jin Lee, Icksang Han

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to Interspeech 2019

Self-Supervised Audio-Visual Co-Segmentation

Andrew Rouditchenko, Hang Zhao, Chuang Gan, Josh McDermott, Antonio Torralba

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICASSP 2019

Convolutional Attention Networks for Multimodal Emotion Recognition from Speech and Text Data

Chan Woo Lee, Kyu Ye Song, Jihoon Jeong, Woo Yong Choi

专题命中 音频语音多模态 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Inaccurate scientific facts listed in document

Multimodal music information processing and retrieval: survey and future challenges

Federico Simonetta, Stavros Ntalampiras, Federico Avanzini

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.MM、eess.AS

Journal ref in 2019 International Workshop on Multilayer Music Representation and Processing, Milano, IEEE 2019

On Attention Modules for Audio-Visual Synchronization

Naji Khosravan, Shervin Ardeshir, Rohit Puri

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、cs.MM

Noise-tolerant Audio-visual Online Person Verification using an Attention-based Neural Network Fusion

Suwon Shon, Tae-Hyun Oh, James Glass

专题命中 音频语音多模态 ：audio-visual(title)；multi-modal(abstract)；分类 cs.CV、eess.AS

Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

Andrew Owens, Alexei A. Efros

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

End-to-end Multimodal Emotion and Gender Recognition with Dynamic Joint Loss Weights

Myungsu Chae, Tae-Ho Kim, Young Hoon Shin, June-Woo Kim, Soo-Young Lee

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、eess.AS

Comments IROS 2018 Workshop on Crossmodal Learning for Intelligent Robotics

Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces

Yu-An Chung, Wei-Hung Weng, Schrasing Tong, James Glass

专题命中 音频语音多模态 ：cross-modal(title,abstract)；分类 cs.CL、eess.AS

Comments Accepted to NIPS 2018. v2 added the majority word baseline results and other minor fixes. arXiv admin note: text overlap with arXiv:1710.04087 by other authors

Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation

Ariel Ephrat, Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, William T. Freeman, Michael Rubinstein

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Comments Accepted to SIGGRAPH 2018. Project webpage: https://looking-to-listen.github.io

Journal ref ACM Trans. Graph. 37(4): 112:1-112:11 (2018)

Multi-Modal Data Augmentation for End-to-End ASR

Adithya Renduchintala, Shuoyang Ding, Matthew Wiesner, Shinji Watanabe

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CL、eess.AS

Comments 5 Pages, 1 Figure, accepted at INTERSPEECH 2018

Deep Learning for Lip Reading using Audio-Visual Information for Urdu Language

M Faisal, Sanaullah Manzoor

专题命中 音频语音多模态 ：audio-visual(title,abstract)；分类 cs.CV、eess.AS

Symbol Grounding Association in Multimodal Sequences with Missing Elements

Federico Raue, Andreas Dengel, Thomas M. Breuel, Marcus Liwicki

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Under review on Journal of Artificial Intelligence Research (JAIR) -- Special Track on Deep Learning, Knowledge Representation, and Reasoning

Benchmarking Multimodal Sentiment Analysis

Erik Cambria, Devamanyu Hazarika, Soujanya Poria, Amir Hussain, R. B. V. Subramaanyam

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted in CICLing 2017

Cats and Captions vs. Creators and the Clock: Comparing Multimodal Content to Context in Predicting Relative Popularity

Jack Hessel, Lillian Lee, David Mimno

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 10 pages, data and models available at http://www.cs.cornell.edu/~jhessel/cats/cats.html, Proceedings of WWW 2017

Developing a comprehensive framework for multimodal feature extraction

Quinten McNamara, Alejandro de la Vega, Tal Yarkoni

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos

Amir Zadeh, Rowan Zellers, Eli Pincus, Louis-Philippe Morency

专题命中 音频语音多模态 ：multimodal(title,abstract)；分类 cs.CL、cs.MM

Comments Accepted as Journal Publication in IEEE Intelligent Systems

Journal ref IEEE Intelligent Systems 31.6 (2016): 82-88

Multi-Modal Human-Machine Communication for Instructing Robot Grasping Tasks

P. C. McGuire, J. Fritsch, J. J. Steil, F. Roethling, G. A. Fink, S. Wachsmuth, G. Sagerer, H. Ritter

专题命中 音频语音多模态 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments 7 pages, 8 figures

Journal ref Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Lausanne, Switzerland, IEEE publications, pp. 1082-1089 (2002)