多模态大模型 - arXivDaily 专题

2311.04160 2023-11-08 cs.HC 78%

"Tell me about that church": Exploring the Design and User Experience of In-Vehicle Multi-modal Intuitive Interface in the Context of Driving Scenario

Yueteng Yu, Yan Zhang, Gary Burnett

专题命中音频语音多模态：multi-modal(title,abstract)

2309.14118 2023-11-07 cs.LG 78%

MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks

Vinitra Swamy, Malika Satayeva, Jibril Frej, Thierry Bossy, Thijs Vogels, Martin Jaggi, Tanja Käser, Mary-Anne Hartley

专题命中音频语音多模态：multimodal(title,abstract)

Comments Accepted as a full paper at NeurIPS 2023 in New Orleans, USA

2310.15247 2023-10-25 cs.SD cs.CV cs.LG cs.MM eess.AS 78%

SyncFusion: Multimodal Onset-synchronized Video-to-Audio Foley Synthesis

Marco Comunità, Riccardo F. Gramaccioni, Emilian Postolache, Emanuele Rodolà, Danilo Comminiello, Joshua D. Reiss

专题命中音频语音多模态：multimodal(title)；分类 cs.CV、cs.MM、eess.AS

2310.01733 2023-10-04 eess.SY cs.SY 78%

Health Guardian: Using Multi-modal Data to Understand Individual Health

Vince S. Siu, Kuan Yu Hsieh, Italo Buleje, Takashi Itoh, Tian Hao, Ben Civjan, Nigel Hinds, Bing Dang, Jeffrey L. Rogers, Bo Wen

专题命中音频语音多模态：multi-modal(title,abstract)

Comments 10 pages, 6 figures

Journal ref IEEE International Conference on Digital Health (ICDH), 2023, pp. 65-74

2306.12249 2023-06-22 cs.SD cs.AI cs.IR cs.LG cs.MM eess.AS 78%

Knowledge-based Multimodal Music Similarity

Andrea Poltronieri

专题命中音频语音多模态：multimodal(title)；分类 cs.AI、cs.MM、eess.AS

Comments 11 pages, 1 figure

2305.03699 2023-05-24 cs.HC 78%

Multimodal User Authentication in Smart Environments: Survey of User Attitudes

Aishat Aloba, Sarah Morrison-Smith, Aaliyah Richlen, Kimberly Suarez, Yu-Peng Chen, Shaghayegh Esmaeili, Damon L. Woodard, Jaime Ruiz, Lisa Anthony

专题命中音频语音多模态：multimodal(title,abstract)

Comments 23 pages, 4 figures

2305.12063 2023-05-23 cs.LG cs.HC 78%

Efficient Multimodal Neural Networks for Trigger-less Voice Assistants

Sai Srujana Buddi, Utkarsh Oggy Sarawgi, Tashweena Heeramun, Karan Sawnhey, Ed Yanosik, Saravana Rathinam, Saurabh Adya

专题命中音频语音多模态：multimodal(title,abstract)

2304.10588 2023-04-24 cs.RO cs.HC 78%

Detecting Worker Attention Lapses in Human-Robot Interaction: An Eye Tracking and Multimodal Sensing Study

Zhuangzhuang Dai, Jinha Park, Aleksandra Kaszowska, Chen Li

专题命中音频语音多模态：multimodal(title,abstract)

Comments 6 pages

2304.04385 2023-04-12 cs.LG 78%

On Robustness in Multimodal Learning

Brandon McKinzie, Joseph Cheng, Vaishaal Shankar, Yinfei Yang, Jonathon Shlens, Alexander Toshev

专题命中音频语音多模态：multimodal(title,abstract)

2304.03022 2023-04-07 cs.IR 78%

TagGPT: Large Language Models are Zero-shot Multimodal Taggers

Chen Li, Yixiao Ge, Jiayong Mao, Dian Li, Ying Shan

专题命中音频语音多模态：multimodal(title,abstract)

Comments 13 pages, 6 figures

2302.07547 2023-02-16 stat.AP cs.HC 78%

Multimodal N-of-1 trials: A Novel Personalized Healthcare Design

Jingjing Fu, Shuheng Liu, Siqi Du, Siqiao Ruan, Xuliang Guo, Weiwei Pan, Abhishek Sharma, Stefan Konigorski

专题命中音频语音多模态：multimodal(title,abstract)

2302.03227 2023-02-08 cs.LG eess.SP q-bio.NC 78%

Automatic Sleep Stage Classification with Cross-modal Self-supervised Features from Deep Brain Signals

Chen Gong, Yue Chen, Yanan Sui, Luming Li

专题命中音频语音多模态：cross-modal(title,abstract)

Comments 4 pages, 5 figures, 11th International IEEE EMBS Conference on Neural Engineering (NER)

2205.02670 2023-01-10 cs.LG 78%

Unsupervised Mismatch Localization in Cross-Modal Sequential Data with Application to Mispronunciations Localization

Wei Wei, Huang Hengguan, Gu Xiangming, Wang Hao, Wang Ye

专题命中音频语音多模态：cross-modal(title,abstract)

2207.12566 2022-12-02 cs.HC 78%

Eliciting Multimodal Gesture+Speech Interactions in a Multi-Object Augmented Reality Environment

Xiaoyan Zhou, Adam S. Williams, Francisco R. Ortega

专题命中音频语音多模态：multimodal(title,abstract)

2009.06591 2022-08-09 cs.HC 78%

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Adam S. Williams, Francisco R. Ortega

专题命中音频语音多模态：multimodal(title,abstract)

Journal ref Proceedings of the ACM on Human-Computer Interaction Volume 4 Issue ISS November 2020

2202.01439 2022-07-06 cs.HC 78%

Sensing the Breath: A Multimodal Singing Tutoring Interface with Breath Guidance

Ziyue Piao, Gus Xia

专题命中音频语音多模态：multimodal(title,abstract)

Comments New Interfaces for Musical Expression 2022 Demo Paper

2203.13932 2022-03-29 cs.MM cs.CV cs.SD eess.AS 78%

A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals

Yuanchao Li, Catherine Lai

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM、eess.AS

Comments 5 pages, 2 figures, submitted to INTERSPEECH 2022

2203.01977 2022-03-07 cs.MM cs.CV cs.RO cs.SD eess.AS 78%

Audio-Visual Object Classification for Human-Robot Collaboration

A. Xompero, Y. L. Pang, T. Patten, A. Prabhakar, B. Calli, A. Cavallaro

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM、eess.AS

Comments 5 pages, 2 figures, 1 table; accepted at ICASSP 2022; Challenge webpage, see https://corsmal.eecs.qmul.ac.uk/challenge.html

2109.04833 2022-02-21 cs.LG 78%

Multimodal Federated Learning on IoT Data

Yuchen Zhao, Payam Barnaghi, Hamed Haddadi

专题命中音频语音多模态：multimodal(title,abstract)

Comments 12 pages, IoTDI '22, May 3-6, 2022, Milan, Italy

2202.01639 2022-02-04 cs.HC 78%

Mathematical Content Browsing for Print-Disabled Readers Based on Virtual-World Exploration and Audio-Visual Sensory substitution

Rynhardt Kruger, Febe de Wet, Thomas Niesler

专题命中音频语音多模态：audio-visual(title,abstract)

2201.02237 2022-01-10 cs.RO cs.HC 78%

Multi-modal data fusion of Voice and EMG data for Robotic Control

Tauheed Khan Mohd, Jackson Carvalho, Ahmad Y Javaid

专题命中音频语音多模态：multi-modal(title,abstract)

2110.04634 2021-10-12 cs.RO 78%

Multimodal Sensory Learning for Real-time, Adaptive Manipulation

Ahalya Prabhakar, Stanislas Furrer, Lorenzo Panchetti, Maxence Perret, Aude Billard

专题命中音频语音多模态：multimodal(title,abstract)

Comments Presented at AI-HRI symposium as part of AAAI-FSS 2021 (arXiv:2109.10836)

2110.03660 2021-10-08 cs.HC cs.LG 78%

Developing Medical AI : a cloud-native audio-visual data collection study

Sagi Schein, Greg Arutiunian, Vitaly Burshtein, Gal Sadeh, Michelle Townshend, Bruce Friedman, Shada Sadr-azodi

专题命中音频语音多模态：audio-visual(title,abstract)

2010.09948 2021-10-05 cs.RO 78%

Object Permanence Through Audio-Visual Representations

Fanjun Bu, Chien-Ming Huang

专题命中音频语音多模态：audio-visual(title)；multimodal(abstract)

Comments 8 pages, 4 figures, 2 tables, published in IEEE Access

Journal ref in IEEE Access, vol. 9, pp. 131574-131582, 2021

2109.10571 2021-09-23 cs.RO 78%

Audio-Visual Grounding Referring Expression for Robotic Manipulation

Yefei Wang, Kaili Wang, Yi Wang, Di Guo, Huaping Liu, Fuchun Sun

专题命中音频语音多模态：audio-visual(title,abstract)

2102.03796 2021-08-04 cs.HC 78%

Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Akinari Onishi

专题命中音频语音多模态：multimodal(title,abstract)

2105.13132 2021-06-23 cs.LG cs.CY 78%

Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution

Yang Li, Zinc Zhang, Hutchin Huang

专题命中音频语音多模态：multimodal(title)；multi-modal(abstract)

Comments Our code is available at: https://github.com/yangland/hatefulchallenge

2105.05182 2021-05-13 cs.HC 78%

PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback

Yaohua Bu, Tianyi Ma, Weijun Li, Hang Zhou, Jia Jia, Shengqi Chen, Kaiyuan Xu, Dachuan Shi, Haozhe Wu, Zhihan Yang, Kun Li, Zhiyong Wu, Yuanchun Shi, Xiaobo Lu, Ziwei Liu

专题命中音频语音多模态：audio-visual(title,abstract)

2105.04489 2021-05-11 cs.CV cs.CL cs.LG cs.SD eess.AS 78%

Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions

Mathew Monfort, SouYoung Jin, Alexander Liu, David Harwath, Rogerio Feris, James Glass, Aude Oliva

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.CL、eess.AS

Comments To appear at CVPR 2021

2105.00708 2021-05-04 cs.SD cs.CV cs.MM eess.AS 78%

Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation

Yan-Bo Lin, Yu-Chiang Frank Wang

专题命中音频语音多模态：audio-visual(title)；分类 cs.CV、cs.MM、eess.AS

Comments AAAI'21

1. 音频语音多模态 4587 篇

"Tell me about that church": Exploring the Design and User Experience of In-Vehicle Multi-modal Intuitive Interface in the Context of Driving Scenario

MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks

SyncFusion: Multimodal Onset-synchronized Video-to-Audio Foley Synthesis

Health Guardian: Using Multi-modal Data to Understand Individual Health

Knowledge-based Multimodal Music Similarity

Multimodal User Authentication in Smart Environments: Survey of User Attitudes

Efficient Multimodal Neural Networks for Trigger-less Voice Assistants

Detecting Worker Attention Lapses in Human-Robot Interaction: An Eye Tracking and Multimodal Sensing Study

On Robustness in Multimodal Learning

TagGPT: Large Language Models are Zero-shot Multimodal Taggers

Multimodal N-of-1 trials: A Novel Personalized Healthcare Design

Automatic Sleep Stage Classification with Cross-modal Self-supervised Features from Deep Brain Signals

Unsupervised Mismatch Localization in Cross-Modal Sequential Data with Application to Mispronunciations Localization

Eliciting Multimodal Gesture+Speech Interactions in a Multi-Object Augmented Reality Environment

Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation

Sensing the Breath: A Multimodal Singing Tutoring Interface with Breath Guidance

A Cross-Domain Approach for Continuous Impression Recognition from Dyadic Audio-Visual-Physio Signals

Audio-Visual Object Classification for Human-Robot Collaboration

Multimodal Federated Learning on IoT Data

Mathematical Content Browsing for Print-Disabled Readers Based on Virtual-World Exploration and Audio-Visual Sensory substitution

Multi-modal data fusion of Voice and EMG data for Robotic Control

Multimodal Sensory Learning for Real-time, Adaptive Manipulation

Developing Medical AI : a cloud-native audio-visual data collection study

Object Permanence Through Audio-Visual Representations

Audio-Visual Grounding Referring Expression for Robotic Manipulation

Brain-computer interface with rapid serial multimodal presentation using artificial facial images and voice

Enhance Multimodal Model Performance with Data Augmentation: Facebook Hateful Meme Challenge Solution

PTeacher: a Computer-Aided Personalized Pronunciation Training System with Exaggerated Audio-Visual Corrective Feedback

Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions

Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation