多模态大模型 - arXivDaily 专题

1704.05915 2018-12-11 cs.HC cs.AI cs.DC 79%

User-driven Intelligent Interface on the Basis of Multimodal Augmented Reality and Brain-Computer Interaction for People with Functional Disabilities

S. Stirenko, Yu. Gordienko, T. Shemsedinov, O. Alienin, Yu. Kochura, N. Gordienko, A. Rojbi, J. R. López Benito, E. Artetxe González

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Comments 10 pages, 11 figures, 1 table, submitted to Future of Information and Communication Conference (FICC) 2018, 5-6 April 2018, Singapore

Journal ref In: Arai K., Kapoor S., Bhatia R. (eds) Advances in Information and Communication Networks. FICC 2018. Advances in Intelligent Systems and Computing, vol 886, pp.612-631. Springer, Cham

1811.12214 2018-11-30 cs.SD eess.AS 79%

Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer

Chien-Yu Lu, Min-Xin Xue, Chia-Che Chang, Che-Rung Lee, Li Su

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

1811.10199 2018-11-27 cs.CV 79%

Cross-domain Deep Feature Combination for Bird Species Classification with Audio-visual Data

Bold Naranchimeg, Chao Zhang, Takuya Akashi

专题命中音频语音多模态：audio-visual(title)；multimodal(abstract)；分类 cs.CV

1811.04448 2018-11-13 cs.SD eess.AS 79%

A Multi-modal Deep Neural Network approach to Bird-song identification

Botond Fazeka, Alexander Schindler, Thomas Lidy, Andreas Rauber

专题命中音频语音多模态：multi-modal(title,abstract)；分类 eess.AS

Comments LifeCLEF 2017 working notes, Dublin, Ireland

1603.09725 2018-10-15 cs.CV cs.SD 79%

Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion

Israel D. Gebru, Silèye Ba, Xiaofei Li, Radu Horaud

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments 14 pages, 6 figures, 5 tables

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(6), 1086 - 1099, 2018

1810.00108 2018-10-02 cs.CV 79%

Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecture

Stavros Petridis, Themos Stafylakis, Pingchuan Ma, Georgios Tzimiropoulos, Maja Pantic

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments Accepted to IEEE SLT 2018

1701.06708 2018-09-18 cs.CV 79%

Speech Map: A Statistical Multimodal Atlas of 4D Tongue Motion During Speech from Tagged and Cine MR Images

Jonghye Woo, Fangxu Xing, Maureen Stone, Jordan Green, Timothy G. Reese, Thomas J. Brady, Van J. Wedeen, Jerry L. Prince, Georges El Fakhri

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV

Comments Accepted at Journal of Computer Methods in Biomechanics and Biomedical Engineering

1809.04203 2018-09-13 cs.CL 79%

Multimodal neural pronunciation modeling for spoken languages with logographic origin

Minh Nguyen, Gia H. Ngo, Nancy F. Chen

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments Accepted to EMNLP 2018

1808.05561 2018-08-17 cs.CV 79%

Emotion Recognition in Speech using Cross-Modal Transfer in the Wild

Samuel Albanie, Arsha Nagrani, Andrea Vedaldi, Andrew Zisserman

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.CV

Comments Conference paper at ACM Multimedia 2018

1808.00118 2018-08-02 cs.CV cs.CR cs.HC 79%

Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

Serguei A. Mokhov, Miao Song, Jashanjot Singh, Joey Paquet, Mourad Debbabi, Sudhir Mudur

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV

Comments reformatted; ICPRAI 2018 conference proceedings, pp. 151-157, CENPARMI, Concordia University, Montreal

1806.08612 2018-06-25 cs.CV 79%

Ad-Net: Audio-Visual Convolutional Neural Network for Advertisement Detection In Videos

Shervin Minaee, Imed Bouazizi, Prakash Kolan, Hossein Najafzadeh

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

1804.04121 2018-06-20 cs.CV cs.SD 79%

The Conversation: Deep Audio-Visual Speech Enhancement

Triantafyllos Afouras, Joon Son Chung, Andrew Zisserman

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments To appear in Interspeech 2018. We provide supplementary material with interactive demonstrations on http://www.robots.ox.ac.uk/~vgg/demo/theconversation

1806.02923 2018-06-11 cs.CL 79%

Multimodal Relational Tensor Network for Sentiment and Emotion Classification

Saurav Sahay, Shachi H Kumar, Rui Xia, Jonathan Huang, Lama Nachman

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

1804.00326 2018-04-04 cs.CV 79%

Seeing Voices and Hearing Faces: Cross-modal biometric matching

Arsha Nagrani, Samuel Albanie, Andrew Zisserman

专题命中音频语音多模态：cross-modal(title,abstract)；分类 cs.CV

Comments To appear in: IEEE Computer Vision and Pattern Recognition (CVPR), 2018

1802.08332 2018-02-26 cs.CL 79%

Deep Multimodal Learning for Emotion Recognition in Spoken Language

Yue Gu, Shuhong Chen, Ivan Marsic

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments ICASSP 2018

1802.01115 2018-02-06 cs.CV 79%

End2You -- The Imperial Toolkit for Multimodal Profiling by End-to-End Learning

Panagiotis Tzirakis, Stefanos Zafeiriou, Bjorn W. Schuller

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV

1711.11118 2017-12-01 cs.CL 79%

Multimodal Attribute Extraction

Robert L. Logan, Samuel Humeau, Sameer Singh

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments AKBC 2017 Workshop Paper

1711.08976 2017-11-30 cs.IR cs.SD eess.AS 79%

Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval

Yi Yu, Suhua Tang, Francisco Raposo, Lei Chen

专题命中音频语音多模态：cross-modal(title,abstract)；分类 eess.AS

1711.01775 2017-11-07 cs.MM cs.HC cs.RO 79%

Multimodal Signal Processing and Learning Aspects of Human-Robot Interaction for an Assistive Bathing Robot

A. Zlatintsi, I. Rodomagoulakis, P. Koutras, A. C. Dometios, V. Pitsikalis, C. S. Tzafestas, P. Maragos

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.MM

1706.02757 2017-06-12 cs.RO cs.CL cs.HC 79%

Sympathy Begins with a Smile, Intelligence Begins with a Word: Use of Multimodal Features in Spoken Human-Robot Interaction

Jekaterina Novikova, Christian Dondrup, Ioannis Papaioannou, Oliver Lemon

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments Robo-NLP workshop at ACL 2017. 9 pages, 5 figures, 6 tables

1509.01509 2017-01-31 cs.CV cs.LG stat.ML 79%

EM Algorithms for Weighted-Data Clustering with Application to Audio-Visual Scene Analysis

Israel D. Gebru, Xavier Alameda-Pineda, Florence Forbes, Radu Horaud

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

Comments 14 pages, 4 figures, 4 tables

Journal ref IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 38, number 12, 2402 - 2415, 2016

1611.08666 2016-11-29 cs.LG cs.AI cs.RO 79%

Training an Interactive Humanoid Robot Using Multimodal Deep Reinforcement Learning

Heriberto Cuayáhuitl, Guillaume Couly, Clément Olalainty

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Comments NIPS Workshop on Future of Interactive Learning Machines, 2016

1604.02946 2016-11-23 cs.CV 79%

Kernel-based Sensor Fusion with Application to Audio-Visual Voice Activity Detection

David Dov, Ronen Talmon, Israel Cohen

专题命中音频语音多模态：audio-visual(title,abstract)；分类 cs.CV

1303.5395 2013-03-25 cs.AI 79%

Lattice-Based Graded Logic: a Multimodal Approach

Philippe Chatalic, Christine Froidevaux

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Comments Appears in Proceedings of the Eighth Conference on Uncertainty in Artificial Intelligence (UAI1992)

1201.3720 2012-01-19 cs.CV 79%

A Multimodal Biometric System Using Linear Discriminant Analysis For Improved Performance

Aamir Khan, Muhammad Farhan, Aasim Khurshid, Adeel Akram

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CV

Journal ref IJCSI International Journal of Computer Science Issues, Vol. 8, Issue 6, No 2, 2011, 122-127

1109.6361 2011-09-30 cs.AI 79%

Cognitive Principles in Robust Multimodal Interpretation

J. Y. Chai, Z. Prasov, S. Qu

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.AI

Journal ref Journal Of Artificial Intelligence Research, Volume 27, pages 55-83, 2006

1005.4014 2010-05-24 cs.MM 79%

A Study on Potential of Integrating Multimodal Interaction into Musical Conducting Education

Gilbert Phuah Leong Siang, Nor Azman Ismail, Pang Yee Yong

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.MM

Comments http://www.journalofcomputing.org

Journal ref Journal of Computing, Volume 2, Issue 5, May 2010

cmp-lg/9406002 2009-11-30 cmp-lg cs.CL 79%

Speech Dialogue with Facial Displays: Multimodal Human-Computer Conversation

Katashi Nagao, Akikazu Takeuchi

专题命中音频语音多模态：multimodal(title,abstract)；分类 cs.CL

Comments 8 pages, Postscript file, UNIX compressed, uuencoded

2607.11007 2026-07-29 cs.LG 版本更新 79%

TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

超越表格数据的TabPFN：多模态嵌入的校准与准确性

Jingxiang Zhang, Lujia Zhong, Zijie Zhu, Shuo Huang, Yuang Xu

专题命中音频语音多模态：multimodal(title,abstract)

AI总结研究少样本多模态分类中轻量级头部校准不佳问题，核心方法是评估TabPFN作为即插即用分类头。贡献在于其在多数据集、多编码器和多模态评估中表现优异，能提升校准并保持准确率，为校准敏感多模态分类提供指导。

Comments 19 pages, 13 figures, 10 tables. Jingxiang Zhang and Lujia Zhong contributed equally. Code: https://github.com/Jingxiang-Zhang/tabpfn-multimodal-embeddings

2601.07367 2026-03-03 cs.SD 79%

FOCAL: A Novel Benchmarking Technique for Multi-modal Agents

FOCAL：多模态代理的新型基准测试技术

Anupam Purwar, Aditya Choudhary

专题命中音频语音多模态：multi-modal(title,abstract)

AI总结 FOCAL提出了一种新型基准测试技术，用于评估多模态代理的端到端推理能力、误差传播及对话效果。

Comments We present a framework for evaluation of Multi-modal Agents consisting of Voice-to-voice model components viz. Text to Speech (TTS), Retrieval Augmented Generation (RAG) and Speech-to-text (STT)

1. 音频语音多模态 4587 篇

User-driven Intelligent Interface on the Basis of Multimodal Augmented Reality and Brain-Computer Interaction for People with Functional Disabilities

Play as You Like: Timbre-enhanced Multi-modal Music Style Transfer

Cross-domain Deep Feature Combination for Bird Species Classification with Audio-visual Data

A Multi-modal Deep Neural Network approach to Bird-song identification

Audio-Visual Speaker Diarization Based on Spatiotemporal Bayesian Fusion

Audio-Visual Speech Recognition With A Hybrid CTC/Attention Architecture

Speech Map: A Statistical Multimodal Atlas of 4D Tongue Motion During Speech from Tagged and Cine MR Images

Multimodal neural pronunciation modeling for spoken languages with logographic origin

Emotion Recognition in Speech using Cross-Modal Transfer in the Wild

Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

Ad-Net: Audio-Visual Convolutional Neural Network for Advertisement Detection In Videos

The Conversation: Deep Audio-Visual Speech Enhancement

Multimodal Relational Tensor Network for Sentiment and Emotion Classification

Seeing Voices and Hearing Faces: Cross-modal biometric matching

Deep Multimodal Learning for Emotion Recognition in Spoken Language

End2You -- The Imperial Toolkit for Multimodal Profiling by End-to-End Learning

Multimodal Attribute Extraction

Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval

Multimodal Signal Processing and Learning Aspects of Human-Robot Interaction for an Assistive Bathing Robot

Sympathy Begins with a Smile, Intelligence Begins with a Word: Use of Multimodal Features in Spoken Human-Robot Interaction

EM Algorithms for Weighted-Data Clustering with Application to Audio-Visual Scene Analysis

Training an Interactive Humanoid Robot Using Multimodal Deep Reinforcement Learning

Kernel-based Sensor Fusion with Application to Audio-Visual Voice Activity Detection

Lattice-Based Graded Logic: a Multimodal Approach

A Multimodal Biometric System Using Linear Discriminant Analysis For Improved Performance

Cognitive Principles in Robust Multimodal Interpretation

A Study on Potential of Integrating Multimodal Interaction into Musical Conducting Education

Speech Dialogue with Facial Displays: Multimodal Human-Computer Conversation

TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

FOCAL: A Novel Benchmarking Technique for Multi-modal Agents