arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.04723eess.IVcs.AIcs.CLeess.SP
收藏

综述:数据模态转换中基于 Transformer 的模型

Survey: Transformer-based Models in Data Modality Conversion

School of Computing, Queen’s University(女王大学计算学院)

Elyas Rashno, Amir Eskandari, Aman Anand, Farhana Zulkernine

AI总结:

本文综述基于 Transformer 的模型在文本、视觉和语音模态转换中的架构、方法与应用,展现其在 AI 内容生成和理解中的通用性与可扩展性。

摘要:

Transformer 已在人工智能的多个领域取得重大进展,包括自然语言处理、计算机视觉和音频处理。这一成功自然引起了学术界和工业界研究人员的广泛关注。因此,人们为这些领域开发了众多 Transformer 变体(通常称为 X-formers)。然而,目前仍缺乏对这些特定模态转换的全面、系统综述。模态转换是指将数据从一种表示形式转换为另一种表示形式,模拟人类整合和解释感官信息的方式。本文全面综述了应用于文本、视觉和语音等主要模态的基于 Transformer 的模型,讨论了它们的架构、转换方法和应用。通过综合关于模态转换的文献,本综述旨在强调 Transformer 在推动 AI 驱动的内容生成与理解方面的多功能性和可扩展性。

英文摘要:

Transformers have made significant strides across various artificial intelligence domains, including natural language processing, computer vision, and audio processing. This success has naturally garnered considerable interest from both academic and industry researchers. Consequently, numerous Transformer variants (often referred to as X-formers) have been developed for these fields. However, a thorough and systematic review of these modality-specific conversions remains lacking. Modality Conversion involves the transformation of data from one form of representation to another, mimicking the way humans integrate and interpret sensory information. This paper provides a comprehensive review of transformer-based models applied to the primary modalities of text, vision, and speech, discussing their architectures, conversion methodologies, and applications. By synthesizing the literature on modality conversion, this survey aims to underline the versatility and scalability of transformers in advancing AI-driven content generation and understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑