多模态大模型

Learning Embodied Semantics via Music and Dance Semiotic Correlations

Francisco Afonso Raposo, David Martins de Matos, Ricardo Ribeiro

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、eess.AS

Comments 24 pages, 1 figure, 5 tables

Journal ref Neural Computing and Applications, vol. 33, pp. 14481-14493, 2021

MusCaps: Generating Captions for Music Audio

Ilaria Manco, Emmanouil Benetos, Elio Quinton, Gyorgy Fazekas

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to IJCNN 2021 for the Special Session on Representation Learning for Audio, Speech, and Music Processing

PoseKernelLifter: Metric Lifting of 3D Human Pose using Sound

Zhijian Yang, Xiaoran Fan, Volkan Isler, Hyun Soo Park

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、eess.AS

Sub-word Level Lip Reading With Visual Attention

K R Prajwal, Triantafyllos Afouras, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、cs.CL

Learning Signal-Agnostic Manifolds of Neural Fields

Yilun Du, Katherine M. Collins, Joshua B. Tenenbaum, Vincent Sitzmann

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments NeurIPS 2021, additional results and code at https://yilundu.github.io/gem/

Speech Emotion Recognition Using Quaternion Convolutional Neural Networks

Aneesh Muppidi, Martin Radfar

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CL、eess.AS

Comments Published in ICASSP 2021

E-ffective: A Visual Analytic System for Exploring the Emotion and Effectiveness of Inspirational Speeches

Kevin Maher, Zeyuan Huang, Jiancheng Song, Xiaoming Deng, Yu-Kun Lai, Cuixia Ma, Hao Wang, Yong-Jin Liu, Hongan Wang

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments IEEE Transactions of Visualization and Computer Graphics (TVCG, Proc. VIS 2021), to appear

Knowledge distillation from language model to acoustic model: a hierarchical multi-task learning approach

Mun-Hak Lee, Joon-Hyuk Chang

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments 4page + 1page for citation + 2 pages for appendix

S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representations

Wen-Chin Huang, Shu-Wen Yang, Tomoki Hayashi, Hung-Yi Lee, Shinji Watanabe, Tomoki Toda

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 cs.CL、eess.AS

Comments Submitted to ICASSP 2022. Code available at: https://github.com/s3prl/s3prl/tree/master/s3prl/downstream/a2o-vc-vcc2020

Echo-Reconstruction: Audio-Augmented 3D Scene Reconstruction

Justin Wilson, Nicholas Rewkowski, Ming C. Lin, Henry Fuchs

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

V-SlowFast Network for Efficient Visual Sound Separation

Lingyu Zhu, Esa Rahtu

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments total 21 pages: main paper 8 pages, references 3 pages, supplementary material 10 pages

Identifying Visible Actions in Lifestyle Vlogs

Oana Ignat, Laura Burdick, Jia Deng, Rada Mihalcea

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV、cs.CL

Comments Accepted at ACL 2019

Look Who's Talking: Active Speaker Detection in the Wild

You Jin Kim, Hee-Soo Heo, Soyeon Choe, Soo-Whan Chung, Yoohwan Kwon, Bong-Jin Lee, Youngki Kwon, Joon Son Chung

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments To appear in Interspeech 2021. Data will be available from https://github.com/clovaai/lookwhostalking

Spot the conversation: speaker diarisation in the wild

Joon Son Chung, Jaesung Huh, Arsha Nagrani, Triantafyllos Afouras, Andrew Zisserman

专题命中 音频语音多模态 ：audio-visual(abstract)；分类 cs.CV、eess.AS

Comments The dataset will be available for download from http://www.robots.ox.ac.uk/~vgg/data/voxceleb/voxconverse.html . The development set will be released in July 2020, and the test set will be released in October 2020

Emotion recognition by fusing time synchronous and time asynchronous representations

Wen Wu, Chao Zhang, Philip C. Woodland

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Journal ref ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021, pp. 6269-6273

One Billion Audio Sounds from GPU-enabled Modular Synthesis

Joseph Turian, Jordie Shier, George Tzanetakis, Kirk McNally, Max Henry

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.AI、eess.AS

DeHumor: Visual Analytics for Decomposing Humor

Xingbo Wang, Yao Ming, Tongshuang Wu, Haipeng Zeng, Yong Wang, Huamin Qu

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、cs.MM

Comments 15 pages. A preprint version of a publication at IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021

Sketching sounds: an exploratory study on sound-shape associations

Sebastian Löbbers, Mathieu Barthet, György Fazekas

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS

Comments accepted for International Computer Music Conference (ICMC) 2021

The Influence of Audio on Video Memorability with an Audio Gestalt Regulated Video Memorability System

Lorin Sweeney, Graham Healy, Alan F. Smeaton

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CV、cs.MM

Comments 6 pages, 3 figures, 4 tables, paper accepted in CBMI 2021 for publication and oral presentation

Alzheimer's Dementia Recognition Using Acoustic, Lexical, Disfluency and Speech Pause Features Robust to Noisy Inputs

Morteza Rohanian, Julian Hough, Matthew Purver

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.CL、eess.AS

Comments INTERSPEECH 2021. arXiv admin note: substantial text overlap with arXiv:2106.09668

PixInWav: Residual Steganography for Hiding Pixels in Audio

Margarita Geleta, Cristina Punti, Kevin McGuinness, Jordi Pons, Cristian Canton, Xavier Giro-i-Nieto

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.MM、eess.AS

Comments Extended abstract presented in CVPR 2021 Women in Computer Vision Workshop

LiRA: Learning Visual Speech Representations from Audio through Self-supervision

Pingchuan Ma, Rodrigo Mira, Stavros Petridis, Björn W. Schuller, Maja Pantic

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted for publication at Interspeech 2021

Speak or Chat with Me: End-to-End Spoken Language Understanding System with Flexible Inputs

Sujeong Cha, Wangrui Hou, Hyun Jung, My Phung, Michael Picheny, Hong-Kwang Kuo, Samuel Thomas, Edmilson Morais

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CL、eess.AS

Comments Accepted to Interspeech 2021

Learning to Detect Bipolar Disorder and Borderline Personality Disorder with Language and Speech in Non-Clinical Interviews

Bo Wang, Yue Wu, Niall Taylor, Terry Lyons, Maria Liakata, Alejo J Nevado-Holgado, Kate E A Saunders

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、eess.AS

Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modeling

Songxiang Liu, Yuewen Cao, Disong Wang, Xixin Wu, Xunying Liu, Helen Meng

专题命中 音频语音多模态 ：any-to-any(abstract)；分类 cs.CL、eess.AS

Comments Accepted by IEEE/ACM Transactions on Audio, Speech and Language Processing

Robust Robotic Pouring using Audition and Haptics

Hongzhuo Liang, Chuangchuang Zhou, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Fuchun Sun, Marcus Stoffel, Jianwei Zhang

专题命中 音频语音多模态 ：multimodal(abstract)；分类 cs.AI、eess.AS

Comments accepted by IROS2020

Journal ref 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

A Neural Lip-Sync Framework for Synthesizing Photorealistic Virtual News Anchors

Ruobing Zheng, Zhou Zhu, Bo Song, Changjiang Ji

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.CV、eess.AS

Comments Accepted by ICPR2020

REXUP: I REason, I EXtract, I UPdate with Structured Compositional Reasoning for Visual Question Answering

Siwen Luo, Soyeon Caren Han, Kaiyuan Sun, Josiah Poon

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICONIP 2020

Attention Forcing for Machine Translation

Qingyun Dou, Yiting Lu, Potsawee Manakul, Xixin Wu, Mark J. F. Gales

专题命中 音频语音多模态 ：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments arXiv admin note: text overlap with arXiv:1909.12289

Audio Description from Image by Modal Translation Network

Hailong Ning, Xiangtao Zheng, Yuan Yuan, Xiaoqiang Lu

专题命中 音频语音多模态 ：cross-modal(abstract)；分类 cs.MM、eess.AS