多模态大模型 - arXivDaily 专题

2003.12265 2020-03-30 cs.MM cs.IR cs.LG 79%

Unsupervised Cross-Modal Audio Representation Learning from Unstructured Multilingual Text

Alexander Schindler, Sergiu Gordea, Peter Knees

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.MM

Comments This is the long version of our SAC2020 poster presentation

Journal ref In Proceedings of the 35th ACM/SIGAPP Symposium On Applied Computing (SAC2020), March 30-April 3, 2020, Brno, Czech Republic

2003.06656 2020-03-17 eess.AS cs.SD eess.IV 79%

Audio-Visual Spatial Aligment Requirements of Central and Peripheral Object Events

Davide Berghi, Hanne Stenzel, Marco Volino, Adrian Hilton, Philip J. B. Jackson

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

Comments Two-pages poster abstract

Journal ref IEEE VR 2020

2002.12766 2020-03-02 eess.AS cs.SD 79%

Multi-Modal Continuous Valence And Arousal Prediction in the Wild Using Deep 3D Features and Sequence Modeling

Sowmya Rasipuram, Junaid Hamid Bhat, Anutosh Maitra

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

2001.00854 2020-02-25 eess.AS cs.SD 79%

Re-synchronization using the Hand Preceding Model for Multi-modal Fusion in Automatic Continuous Cued Speech Recognition

Li Liu, Gang Feng, Denis Beautemps, Xiao-Ping Zhang

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

Journal ref IEEE TMM-2020

2001.04758 2020-01-15 cs.CV 79%

Deep Audio-Visual Learning: A Survey

Hao Zhu, Mandi Luo, Rui Wang, Aihua Zheng, Ran He

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

1905.10693 2020-01-09 cs.CV 79%

DAVE: A Deep Audio-Visual Embedding for Dynamic Saliency Prediction

Hamed R. Tavakoli, Ali Borji, Esa Rahtu, Juho Kannala

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

2001.01656 2020-01-07 eess.AS cs.SD 79%

Audio-visual Recognition of Overlapped speech for the LRS2 dataset

Jianwei Yu, Shi-Xiong Zhang, Jian Wu, Shahram Ghorbani, Bo Wu, Shiyin Kang, Shansong Liu, Xunying Liu, Helen Meng, Dong Yu

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

Comments 5 pages, 5 figures, submitted to icassp2019

1912.12134 2019-12-30 cs.CV eess.SP 79%

Large-scale Multi-modal Person Identification in Real Unconstrained Environments

Jiajie Ye, Yisheng Guan, Junfa Liu, Xinghong Huang, Hong Zhang

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CV

Comments 6 pages, IEEE International Conference on Robotics and Biomimetics 2019

1911.12798 2019-12-02 cs.CL 79%

Multimodal Machine Translation through Visuals and Speech

Umut Sulubacak, Ozan Caglayan, Stig-Arne Grönroos, Aku Rouhe, Desmond Elliott, Lucia Specia, Jörg Tiedemann

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments 34 pages, 4 tables, 8 figures. Submitted (Nov 2019) to the Machine Translation journal (Springer)

1909.05568 2019-12-02 cs.CV 79%

On the Effect of Observed Subject Biases in Apparent Personality Analysis from Audio-visual Signals

Ricardo Darío Pérez Principi, Cristina Palmero, Julio C. S. Jacques Junior, Sergio Escalera

专题命中音频语音多模态：audio-visual(title)；multi-modal(abstract)；分类 cs.CV

Comments Accepted in IEEE Transactions on Affective Computing (TAC)

1911.03930 2019-11-12 eess.AS cs.LG cs.SD 79%

Robust Unsupervised Audio-visual Speech Enhancement Using a Mixture of Variational Autoencoders

Mostafa Sadeghi, Xavier Alameda-Pineda

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

1910.13215 2019-11-12 cs.CL 79%

Transformer-based Cascaded Multimodal Speech Translation

Zixiu Wu, Ozan Caglayan, Julia Ive, Josiah Wang, Lucia Specia

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to IWSLT 2019

1905.12605 2019-11-05 eess.AS cs.LG 79%

Deep-Learning-Based Audio-Visual Speech Enhancement in Presence of Lombard Effect

Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson, Jesper Jensen

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

1811.06250 2019-11-05 eess.AS cs.LG cs.SD eess.IV 79%

Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems

Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson, Jesper Jensen

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

1811.06234 2019-11-05 eess.AS cs.LG cs.SD eess.IV 79%

On Training Targets and Objective Functions for Deep-Learning-Based Audio-Visual Speech Enhancement

Daniel Michelsanti, Zheng-Hua Tan, Sigurdur Sigurdsson, Jesper Jensen

专题命中音频语音多模态：audio-visual(title,abstract)；分类 eess.AS

1910.07193 2019-10-17 cs.NI cs.MM 79%

Scalable Intelligence-Enabled Networking with Traffic Engineering in 5G Scenarios for Future Audio-Visual-Tactile Internet

Yiqiang Sheng

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.MM

1909.04302 2019-09-11 cs.CL cs.LG 79%

Multimodal Embeddings from Language Models

Shao-Yen Tseng, Panayiotis Georgiou, Shrikanth Narayanan

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

1909.03522 2019-09-10 cs.LG cs.SD eess.AS 79%

MIDI-Sandwich2: RNN-based Hierarchical Multi-modal Fusion Generation VAE networks for multi-track symbolic music generation

Xia Liang, Junmin Wu, Jing Cao

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

1907.08696 2019-07-23 cs.IR cs.CL cs.LG stat.ML 79%

Multi-modal Sentiment Analysis using Deep Canonical Correlation Analysis

Zhongkai Sun, Prathusha K Sarma, William Sethares, Erik P. Bucy

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.CL

1907.01615 2019-07-04 cs.LG cs.AI 79%

E-Sports Talent Scouting Based on Multimodal Twitch Stream Data

Anna Belova, Wen He, Ziyi Zhong

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

1906.03098 2019-06-10 cs.LG cs.AI cs.HC cs.RO stat.ML 79%

Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach

Ognjen Rudovic, Meiru Zhang, Bjorn Schuller, Rosalind W. Picard

专题命中音频语音多模态：multi-modal(title,abstract)；分类 cs.AI

1807.07860 2019-04-24 cs.CV cs.LG 79%

Talking Face Generation by Adversarially Disentangled Audio-Visual Representation

Hang Zhou, Yu Liu, Ziwei Liu, Ping Luo, Xiaogang Wang

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments AAAI Conference on Artificial Intelligence (AAAI 2019) Oral Presentation. Code, models, and video results are available on our webpage: https://liuziwei7.github.io/projects/TalkingFace.html

1811.03865 2019-02-21 cs.CL 79%

Multimodal Grounding for Sequence-to-Sequence Speech Recognition

Ozan Caglayan, Ramon Sanabria, Shruti Palaskar, Loïc Barrault, Florian Metze

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments ICASSP 2019

1711.06834 2019-02-18 cs.RO cs.CV cs.NE 79%

Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction

Stéphane Lathuilière, Benoit Massé, Pablo Mesejo, Radu Horaud

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Paper submitted to Pattern Recognition Letters

Journal ref Pattern Recognition Letters, vol. 118, 2019, 61-71

1902.04397 2019-02-13 cs.IR cs.MM 79%

Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies

Meinard Müller, Andreas Arzt, Stefan Balke, Matthias Dorfer, Gerhard Widmer

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.MM

Journal ref IEEE Signal Processing Magazine (Volume: 36, Issue: 1, Jan. 2019)

1812.04891 2019-01-30 cs.CL 79%

A Multimodal LSTM for Predicting Listener Empathic Responses Over Time

Zhi-Xuan Tan, Arushi Goel, Thanh-Son Nguyen, Desmond C. Ong

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

1809.02108 2018-12-27 cs.CV 79%

Deep Audio-Visual Speech Recognition

Triantafyllos Afouras, Joon Son Chung, Andrew Senior, Oriol Vinyals, Andrew Zisserman

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted for publication by IEEE Transactions on Pattern Analysis and Machine Intelligence

1704.05915 2018-12-11 cs.HC cs.AI cs.DC 79%

User-driven Intelligent Interface on the Basis of Multimodal Augmented Reality and Brain-Computer Interaction for People with Functional Disabilities

S. Stirenko, Yu. Gordienko, T. Shemsedinov, O. Alienin, Yu. Kochura, N. Gordienko, A. Rojbi, J. R. López Benito, E. Artetxe González

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Comments 10 pages, 11 figures, 1 table, submitted to Future of Information and Communication Conference (FICC) 2018, 5-6 April 2018, Singapore

Journal ref In: Arai K., Kapoor S., Bhatia R. (eds) Advances in Information and Communication Networks. FICC 2018. Advances in Intelligent Systems and Computing, vol 886, pp.612-631. Springer, Cham

1811.12214 2018-11-30 cs.SD eess.AS 79%

Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer

Chien-Yu Lu, Min-Xin Xue, Chia-Che Chang, Che-Rung Lee, Li Su

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

1811.10199 2018-11-27 cs.CV 79%

Cross-domain Deep Feature Combination for Bird Species Classification with Audio-visual Data

Bold Naranchimeg, Chao Zhang, Takuya Akashi

专题命中音频语音多模态：audio-visual(title)；multimodal(abstract)；分类 cs.CV

1. 音频语音多模态 4585 篇

Unsupervised Cross-Modal Audio Representation Learning from Unstructured Multilingual Text

Audio-Visual Spatial Aligment Requirements of Central and Peripheral Object Events

Multi-Modal Continuous Valence And Arousal Prediction in the Wild Using Deep 3D Features and Sequence Modeling

Re-synchronization using the Hand Preceding Model for Multi-modal Fusion in Automatic Continuous Cued Speech Recognition

Deep Audio-Visual Learning: A Survey

DAVE: A Deep Audio-Visual Embedding for Dynamic Saliency Prediction

Audio-visual Recognition of Overlapped speech for the LRS2 dataset

Large-scale Multi-modal Person Identification in Real Unconstrained Environments

Multimodal Machine Translation through Visuals and Speech

On the Effect of Observed Subject Biases in Apparent Personality Analysis from Audio-visual Signals

Robust Unsupervised Audio-visual Speech Enhancement Using a Mixture of Variational Autoencoders

Transformer-based Cascaded Multimodal Speech Translation

Deep-Learning-Based Audio-Visual Speech Enhancement in Presence of Lombard Effect

Effects of Lombard Reflex on the Performance of Deep-Learning-Based Audio-Visual Speech Enhancement Systems

On Training Targets and Objective Functions for Deep-Learning-Based Audio-Visual Speech Enhancement

Scalable Intelligence-Enabled Networking with Traffic Engineering in 5G Scenarios for Future Audio-Visual-Tactile Internet

Multimodal Embeddings from Language Models

MIDI-Sandwich2: RNN-based Hierarchical Multi-modal Fusion Generation VAE networks for multi-track symbolic music generation

Multi-modal Sentiment Analysis using Deep Canonical Correlation Analysis

E-Sports Talent Scouting Based on Multimodal Twitch Stream Data

Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach

Talking Face Generation by Adversarially Disentangled Audio-Visual Representation

Multimodal Grounding for Sequence-to-Sequence Speech Recognition

Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction

Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies

A Multimodal LSTM for Predicting Listener Empathic Responses Over Time

Deep Audio-Visual Speech Recognition

User-driven Intelligent Interface on the Basis of Multimodal Augmented Reality and Brain-Computer Interaction for People with Functional Disabilities

Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer

Cross-domain Deep Feature Combination for Bird Species Classification with Audio-visual Data