arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.04856cs.CLcs.LGstat.ML
收藏

多模态深度学习

Multimodal Deep Learning

LMU(慕尼黑大学)
Cem Akkus, Luyang Chu, Vladana Djakovic, Steffen Jauch-Walser, Philipp Koch, Giacomo Loss, Christopher Marquardt, Marco Moldovan, Nadja Sauter, Maximilian Schne… 展开作者

Cem Akkus, Luyang Chu, Vladana Djakovic, Steffen Jauch-Walser, Philipp Koch, Giacomo Loss, Christopher Marquardt, Marco Moldovan, Nadja Sauter, Maximilian Schneider, Rickmer Schulte, Karol Urbanczyk, Jann Goschenhofer, Christian Heumann, Rasmus Hvingelby, Daniel Schalk, Matthias Aßenmacher

摘要:

这本书是一个研讨会的结果,在研讨会上,我们回顾了多模态方法,并试图创建一个坚实的领域概述,从深度学习的两个子领域的当前最先进的方法开始。此外,建模框架进行了讨论,其中一个模态转换成其他,以及模型,其中一个模态被用来增强表示学习的其他。为了结束第二部分,重点介绍了同时处理这两种模式的架构。最后,我们还介绍了其他模态以及通用多模态模型,这些模型能够在一个统一的架构中处理不同模态上的不同任务。一个有趣的应用(生成艺术)最终结束了这本小册子。

英文摘要:

This book is the result of a seminar in which we reviewed multimodal approaches and attempted to create a solid overview of the field, starting with the current state-of-the-art approaches in the two subfields of Deep Learning individually. Further, modeling frameworks are discussed where one modality is transformed into the other, as well as models in which one modality is utilized to enhance representation learning for the other. To conclude the second part, architectures with a focus on handling both modalities simultaneously are introduced. Finally, we also cover other modalities as well as general-purpose multi-modal models, which are able to handle different tasks on different modalities within one unified architecture. One interesting application (Generative Art) eventually caps off this booklet.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑