多模态大模型

Speak or Chat with Me: End-to-End Spoken Language Understanding System with Flexible Inputs

Sujeong Cha, Wangrui Hou, Hyun Jung, My Phung, Michael Picheny, Hong-Kwang Kuo, Samuel Thomas, Edmilson Morais

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to Interspeech 2021

Learning to Detect Bipolar Disorder and Borderline Personality Disorder with Language and Speech in Non-Clinical Interviews

Bo Wang, Yue Wu, Niall Taylor, Terry Lyons, Maria Liakata, Alejo J Nevado-Holgado, Kate E A Saunders

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling

Songxiang Liu, Yuewen Cao, Disong Wang, Xixin Wu, Xunying Liu, Helen Meng

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 cs.CL、eess.AS

Comments Accepted by IEEE/ACM Transactions on Audio, Speech and Language Processing

Robust Robotic Pouring using Audition and Haptics

Hongzhuo Liang, Chuangchuang Zhou, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Fuchun Sun, Marcus Stoffel, Jianwei Zhang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI、eess.AS

Comments accepted by IROS2020

Journal ref 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

A Neural Lip-Sync Framework for Synthesizing Photorealistic Virtual News Anchors

Ruobing Zheng, Zhou Zhu, Bo Song, Changjiang Ji

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted by ICPR2020

REXUP: I REason, I EXtract, I UPdate with Structured Compositional Reasoning for Visual Question Answering

Siwen Luo, Soyeon Caren Han, Kaiyuan Sun, Josiah Poon

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICONIP 2020

Attention Forcing for Machine Translation

Qingyun Dou, Yiting Lu, Potsawee Manakul, Xixin Wu, Mark J. F. Gales

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:1909.12289

Audio Description from Image by Modal Translation Network

Hailong Ning, Xiangtao Zheng, Yuan Yuan, Xiaoqiang Lu

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Investigations on Audiovisual Emotion Recognition in Noisy Conditions

Michael Neumann, Ngoc Thang Vu

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Published at the IEEE workshop on Spoken Language Technology (SLT) 2021

DINO: A Conditional Energy-Based GAN for Domain Translation

Konstantinos Vougioukas, Stavros Petridis, Maja Pantic

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICLR 2021

Detecting Adversarial Attacks On Audiovisual Speech Recognition

Pingchuan Ma, Stavros Petridis, Maja Pantic

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted to ICASSP 2021

CASA-Based Speaker Identification Using Cascaded GMM-CNN Classifier in Noisy and Emotional Talking Conditions

Ali Bou Nassif, Ismail Shahin, Shibani Hamsa, Nawel Nemmour, Keikichi Hirose

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.AI、eess.AS

Comments Published in Applied Soft Computing journal

Journal ref Applied Soft Computing, Elsevier, 2021

Replacing Human Audio with Synthetic Audio for On-device Unspoken Punctuation Prediction

Daria Soboleva, Ondrej Skopek, Márius Šajgalík, Victor Cărbune, Felix Weissenberger, Julia Proskurnia, Bogdan Prisacari, Daniel Valcarce, Justin Lu, Rohit Prabhavalkar, Balint Miklos

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to IEEE ICASSP 2021

Music source separation conditioned on 3D point clouds

Francesc Lluís, Vasileios Chatziioannou, Alex Hofmann

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、eess.AS

My Teacher Thinks The World Is Flat! Interpreting Automatic Essay Scoring Mechanism

Swapnil Parekh, Yaman Kumar Singla, Changyou Chen, Junyi Jessy Li, Rajiv Ratn Shah

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

GANterpretations

Pablo Samuel Castro

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI、eess.AS

Comments In 4th Workshop on Machine Learning for Creativity and Design at NeurIPS 2020, Vancouver, Canada

Perception of prosodic variation for speech synthesis using an unsupervised discrete representation of F0

Zack Hodari, Catherine Lai, Simon King

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Comments Published to the 10th ISCA International Conference on Speech Prosody (SP2020)

Remixing Music with Visual Conditioning

Li-Chia Yang, Alexander Lerch

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.MM、eess.AS

Journal ref 2020 IEEE International Symposium on Multimedia

Enactive Mandala: Audio-visualizing Brain Waves

Tomohiro Tokunaga, Michael J. Lyons

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.MM、eess.AS

Comments 2 pages, 2 figures

Journal ref Proceedings of the International Conference on New Interfaces for Musical Expression, 2013

ConfNet2Seq: Full Length Answer Generation from Spoken Questions

Vaishali Pal, Manish Shrivastava, Laurent Besacier

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments Accepted at Text, Speech and Dialogue, 2020

Journal ref ConfNet2Seq, Text, Speech, and Dialogue - 23rd International Conference, {TSD}, Brno, Czech Republic, September 8-11, 2020, Proceedings, 12284, 2020, 524-531 (2020)

CSTNet: Contrastive Speech Translation Network for Self-Supervised Speech Representation Learning

Sameer Khurana, Antoine Laurent, James Glass

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Xiaomingbot: A Multilingual Robot News Reporter

Runxin Xu, Jun Cao, Mingxuan Wang, Jiaze Chen, Hao Zhou, Ying Zeng, Yuping Wang, Li Chen, Xiang Yin, Xijin Zhang, Songcheng Jiang, Yuxuan Wang, Lei Li

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to ACL 2020 - system demonstration

Cross-Task Transfer for Geotagged Audiovisual Aerial Scene Recognition

Di Hu, Xuhong Li, Lichao Mou, Pu Jin, Dong Chen, Liping Jing, Xiaoxiang Zhu, Dejing Dou

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments ECCV 2020

AutoFoley: Artificial Synthesis of Synchronized Sound Tracks for Silent Videos with Deep Learning

Sanchita Ghose, John J. Prevost

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、eess.AS

Comments 14 pages, 14 figures

Journal ref IEEE TRANSACTIONS ON MULTIMEDIA, 2020

Telling Left from Right: Learning Spatial Correspondence of Sight and Sound

Karren Yang, Bryan Russell, Justin Salamon

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments CVPR 2020

History-Aware Question Answering in a Blocks World Dialogue System

Benjamin Kane, Georgiy Platonov, Lenhart K. Schubert

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 16 pages, 4 figures

Active Speakers in Context

Juan Leon Alcazar, Fabian Caba Heilbron, Long Mai, Federico Perazzi, Joon-Young Lee, Pablo Arbelaez, Bernard Ghanem

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Using Cell Phone Pictures of Sheet Music To Retrieve MIDI Passages

TJ Tsai, Daniel Yang, Mengyi Shan, Thitaree Tanprasert, Teerapat Jenrungrot

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments 13 pages, 8 figures, 3 tables. Accepted article in IEEE Transactions on Multimedia. arXiv admin note: text overlap with arXiv:2004.10347

Towards Linking the Lakh and IMSLP Datasets

TJ Tsai

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments 5 pages, 4 figures, 1 table. Accepted paper at the International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2020

MIDI Passage Retrieval Using Cell Phone Pictures of Sheet Music

Daniel Yang, Thitaree Tanprasert, Teerapat Jenrungrot, Mengyi Shan, TJ Tsai

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments 8 pages, 8 figures, 1 table. Accepted paper at the International Society for Music Information Retrieval Conference (ISMIR) 2019