arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.02890stat.MLcs.LG
收藏

基于核嵌入泛函深度神经网络的非线性泛函回归

Nonlinear functional regression by functional deep neural network with kernel embedding

Department of Statistics and Actuarial Science, University of Hong Kong(香港大学统计与精算科学系);Department of Mathematics, Hong Kong Baptist University(香港 Baptist 大学数学系);School of Mathematics and Statistics, University of Sydney(悉尼大学数学与统计学学院);Department of Electrical Engineering, ESAT-STADIUS, KU Leuven(KU Leuven 电子工程系)

Zhongjie Shi, Jun Fan, Linhao Song, Ding-Xuan Zhou, Johan A. K. Suykens

AI总结:

针对非线性泛函回归中无限维泛函数据的降维难题,提出融合自适应核嵌入、Mercer 特征函数投影与深度 ReLU 网络的泛函神经网络,并给出逼近速率、ERM 泛化界及实验验证。

摘要:

近年来,深度学习已被广泛应用于泛函数据分析(FDA),并取得了显著的经验成功。然而,泛函数据的无限维数要求泛函学习任务采用有效的降维方法,尤其是在非线性泛函回归中。本文提出一种具有自适应且离散化不变降维方法的泛函深度神经网络。该泛函网络架构由三部分组成:首先是核嵌入步骤,其特征在于采用带有自适应平滑核的积分变换;其次是投影步骤,利用基于投影 Mercer 核的特征函数基进行降维;最后采用深度 ReLU 神经网络进行预测。文中推导了所提出泛函网络在各类输入函数空间上逼近非线性平滑泛函的显式速率。此外,通过运用一种新颖的两阶段 oracle 不等式以及已建立的泛函逼近结果,我们对应用于该泛函网络的经验风险最小化(ERM)算法进行了泛化分析。最后,我们在模拟数据集和真实数据集上开展数值实验,以证明该泛函网络的有效性和优势。

英文摘要:

Recently, deep learning has been widely applied in functional data analysis (FDA) with notable empirical success. However, the infinite dimensionality of functional data necessitates an effective dimension reduction approach for functional learning tasks, particularly in nonlinear functional regression. In this paper, we introduce a functional deep neural network with an adaptive and discretization-invariant dimension reduction method. Our functional network architecture consists of three parts: first, a kernel embedding step that features an integral transformation with an adaptive smooth kernel; next, a projection step that utilizes eigenfunction bases based on a projection Mercer kernel for the dimension reduction; and finally, a deep ReLU neural network is employed for the prediction. Explicit rates of approximating nonlinear smooth functionals across various input function spaces by our proposed functional network are derived. Additionally, we conduct a generalization analysis for the empirical risk minimization (ERM) algorithm applied to our functional net, by employing a novel two-stage oracle inequality and the established functional approximation results. Ultimately, we conduct numerical experiments on both simulated and real datasets to demonstrate the effectiveness and benefits of our functional net.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑