arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2309.02428cs.LG
收藏

通过张量化增强深度学习模型:综合综述与框架

Enhancing Deep Learning Models through Tensorization: A Comprehensive Survey and Framework

University of Hertfordshire(赫特福德大学)

Manal Helal

AI总结:

本文全面综述张量化技术,介绍其步骤、数据源与多路分析方法,通过BSS实验验证多路分析表达力更强,且能减少参数、加速处理,还梳理了该技术与不同深度神经网络的整合应用。

摘要:

公共领域数据的迅猛增长以及深度学习模型架构的日趋复杂,凸显了对更高效的数据表示与分析技术的需求。本文受(Helal, 2023)研究的启发,旨在对张量化进行全面概述。这一变革性方法弥合了数据固有的多维本质与基于线性代数的机器学习算法中常用的简化二维矩阵之间的差距。本文探讨了张量化的相关步骤、多维数据源、所采用的各类多路分析方法以及这些方法的优势。文中给出了盲源分离(Blind Source Separation, BSS)的一个小型示例,在Python环境下对比了二维算法与多路算法的表现。结果表明,多路分析的表达能力更强。与维度灾难的直觉认知相反,以原生形式使用多维数据集并应用基于多线性代数的多路分析方法,展现出捕捉各维度间复杂相互关系的强大能力,同时还能出人意料地减少模型参数数量并加快处理速度。本文还通过不同应用领域的案例研究,综述了多路分析方法及其与各类深度神经网络模型的整合情况。

英文摘要:

The burgeoning growth of public domain data and the increasing complexity of deep learning model architectures have underscored the need for more efficient data representation and analysis techniques. This paper is motivated by the work of (Helal, 2023) and aims to present a comprehensive overview of tensorization. This transformative approach bridges the gap between the inherently multidimensional nature of data and the simplified 2-dimensional matrices commonly used in linear algebra-based machine learning algorithms. This paper explores the steps involved in tensorization, multidimensional data sources, various multiway analysis methods employed, and the benefits of these approaches. A small example of Blind Source Separation (BSS) is presented comparing 2-dimensional algorithms and a multiway algorithm in Python. Results indicate that multiway analysis is more expressive. Contrary to the intuition of the dimensionality curse, utilising multidimensional datasets in their native form and applying multiway analysis methods grounded in multilinear algebra reveal a profound capacity to capture intricate interrelationships among various dimensions while, surprisingly, reducing the number of model parameters and accelerating processing. A survey of the multi-away analysis methods and integration with various Deep Neural Networks models is presented using case studies in different application domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑