arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.17347cs.LGcs.AI
收藏

深度与广度学习:通过数据间与数据内表示的协同学习增强数据驱动推断

Deep-and-Wide Learning: Enhancing Data-Driven Inference via Synergistic Learning of Inter- and Intra-Data Representations

Stanford University(斯坦福大学);Institute for Computational and Mathematical Engineering, Stanford University(斯坦福大学计算与数学工程研究所)

Md Tauhidul Islam, Lei Xing

AI总结:

提出深度与广度学习(DWL)方案及双交互通道网络(D-Net),通过协同提取数据内与数据间特征,在有限数据下显著提升推断准确率与计算效率。

摘要:

深度学习的进步正在彻底改变科学与工程领域。深度学习的巨大成功主要归功于其从输入数据中提取本质高维(HD)特征并基于此信息做出推断决策的能力。然而,当前的深度神经网络(DNN)模型面临诸多挑战,例如需要大量数据和计算资源。在此,我们引入一种新的学习方案,称为深度与广度学习(DWL),以系统地捕获不仅单个输入数据内部(数据内特征)而且跨数据(数据间特征)的特征。此外,我们提出双交互通道网络(D-Net)来实现DWL,该网络利用我们对低维(LD)数据间特征提取的贝叶斯公式及其与数据集传统HD表示的协同交互,从而大幅提升计算效率和推断能力。所提出的技术已应用于跨学科数据的分类和回归任务。结果表明,DWL在有限训练数据下以显著优势在准确率上超越最先进的DNN,并将计算效率提高了数个数量级。所提出的DWL策略显著改变了数据驱动学习技术,包括新兴的大型基础模型,并为不断发展的AI领域提供了重要见解。

英文摘要:

Advancements in deep learning are revolutionizing science and engineering. The immense success of deep learning is largely due to its ability to extract essential high-dimensional (HD) features from input data and make inference decisions based on this information. However, current deep neural network (DNN) models face several challenges, such as the requirements of extensive amounts of data and computational resources. Here, we introduce a new learning scheme, referred to as deep-and-wide learning (DWL), to systematically capture features not only within individual input data (intra-data features) but also across the data (inter-data features). Furthermore, we propose a dual-interactive-channel network (D-Net) to realize the DWL, which leverages our Bayesian formulation of low-dimensional (LD) inter-data feature extraction and its synergistic interaction with the conventional HD representation of the dataset, for substantially enhanced computational efficiency and inference. The proposed technique has been applied to data across various disciplines for both classification and regression tasks. Our results demonstrate that DWL surpasses state-of-the-art DNNs in accuracy by a substantial margin with limited training data and improves the computational efficiency by order(s) of magnitude. The proposed DWL strategy dramatically alters the data-driven learning techniques, including emerging large foundation models, and sheds significant insights into the evolving field of AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑