部分线性模型中非凸惩罚LAD估计：渐近分析与近端算法

中文摘要 AI 辅助

本文研究了部分线性模型中的最小绝对偏差（LAD）回归。我们通过深度神经网络（DNNs）参数化非参数项，并提出了一种惩罚LAD问题用于估计。具体而言，我们的模型面临以下挑战。首先，正则化项可以是非凸且非光滑的，这需要将无穷维变分分析和非光滑分析引入渐近正态性讨论中。其次，我们的网络必须随着样本量的增加而在宽度、稀疏性级别和深度上扩展，从而为理论分析引入了额外的困难。第三，所提出估计量的oracle本身是通过一个超高维、非凸且不连续的优化问题定义的，这本身已经带来了显著的计算和理论挑战。在这些挑战下，我们建立了估计量的一致性、收敛速度和渐近正态性。此外，我们分析了该oracle问题本身及其连续松弛。我们研究了两种形式的近端次梯度方法的收敛性，突显了其结构差异导致了不同的计算子问题。特别是，松弛形式允许显著更便宜的近端更新，反映了统计准确性和计算可行性之间的内在权衡。

英文摘要

This paper investigates the partial linear model by Least Absolute Deviation (LAD) regression. We parameterize the nonparametric term using Deep Neural Networks (DNNs) and formulate a penalized LAD problem for estimation. Specifically, our model exhibits the following challenges. First, the regularization term can be nonconvex and nonsmooth, necessitating the introduction of infinite dimensional variational analysis and nonsmooth analysis into the asymptotic normality discussion. Second, our network must expand (in width, sparsity level and depth) as more samples are observed, thereby introducing additional difficulties for theoretical analysis. Third, the oracle of the proposed estimator is itself defined through a ultra high-dimensional, nonconvex, and discontinuous optimization problem, which already entails substantial computational and theoretical challenges. Under such the challenges, we establish the consistency, convergence rate, and asymptotic normality of the estimator. Furthermore, we analyze the oracle problem itself and its continuous relaxation. We study the convergence of a proximal subgradient method for both formulations, highlighting their structural differences lead to distinct computational subproblems along the iterations. In particular, the relaxed formulation admits significantly cheaper proximal updates, reflecting an inherent trade-off between statistical accuracy and computational tractability.

发表机构

• Department of Applied Mathematics, The Hong Kong Polytechnic University（应用数学系，香港理工大学）
• College of Computing, Data Science, and Society, University of California, Berkeley（计算、数据科学与社会学院，加州大学伯克利分校）

机构由 AI 辅助整理，请以论文原文为准。