通过一阶方法进行最大仿射回归
Max-affine regression via first-order methods
AI总结：
本文研究最大仿射回归，分析随机次高斯观测下 GD 与小批量 SGD 的非渐近线性收敛，并用实验证明 SGD 在无噪声和低样本噪声情形下更优。
摘要：
我们考虑最大仿射模型的回归问题，该模型通过 max 函数组合多个仿射模型，从而产生分段线性模型。最大仿射模型普遍出现在信号处理和统计学应用中，包括多类分类、拍卖问题和凸回归。它还推广了相位恢复以及学习整流线性单元激活函数。我们针对最大仿射回归给出了梯度下降（GD）和小批量随机梯度下降（SGD）的非渐近收敛分析；在该问题中，模型在服从次高斯性和反集中性的随机位置被观测，并带有加性次高斯噪声。在这些假设下，经过适当初始化的 GD 和 SGD 会线性收敛到由相应误差界指定的真实参数邻域。我们提供了数值结果来证实这一理论发现。重要的是，在无噪声情形下，SGD 不仅相比交替最小化和 GD 使用更少观测值并在运行时间上收敛更快，而且在有噪声的低样本情形中也优于它们。
英文摘要：
We consider regression of a max-affine model that produces a piecewise linear model by combining affine models via the max function. The max-affine model ubiquitously arises in applications in signal processing and statistics including multiclass classification, auction problems, and convex regression. It also generalizes phase retrieval and learning rectifier linear unit activation functions. We present a non-asymptotic convergence analysis of gradient descent (GD) and mini-batch stochastic gradient descent (SGD) for max-affine regression when the model is observed at random locations following the sub-Gaussianity and an anti-concentration with additive sub-Gaussian noise. Under these assumptions, a suitably initialized GD and SGD converge linearly to a neighborhood of the ground truth specified by the corresponding error bound. We provide numerical results that corroborate the theoretical finding. Importantly, SGD not only converges faster in run time with fewer observations than alternating minimization and GD in the noiseless scenario but also outperforms them in low-sample scenarios with noise.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
