AI 中文总结
该研究针对布雷格曼回归，推导了依赖残差的曲率相关条件，明确了二次-四次势函数的凸性条件，通过实验揭示了步长范围随四次参数变化的规律，刻画了曲率与势函数、设计矩阵的相互作用。
AI 中文摘要
我们考虑通过最小化以下形式的布雷格曼损失来拟合的线性回归模型：\\(\frac{1}{n}\sum_{i=1}^n \left[ \phi(y_i)-\phi(x_i^\top\theta) -\phi'(x_i^\top\theta) \bigl(y_i-x_i^\top\theta\bigr) \right]\\)，其中\\(y_i\\)是观测响应，\\(x_i^\top\theta\\)是线性预测。即使生成势函数\\(\phi\\)是强凸的，所得的回归目标关于\\(\theta\\)也可能是非凸的，这在生成势函数的凸性与拟合模型的优化几何之间形成了间隙。海森矩阵可表示为加权格拉姆矩阵，其权重取决于势函数的导数和当前残差。该表示给出了局部强凸性、光滑性以及梯度下降的条件线性收敛的简单条件。针对二次-四次势函数，我们推导了精确的标量凸性条件，确定了负曲率区间，并得到了正曲率的局部和全局充分条件。数值实验表明，这些标量条件可能失效，但在评估点处完整海森矩阵仍保持正定；还表明，随着四次参数增大，导致收敛的测试梯度下降步长范围减小。这些结果刻画了反向布雷格曼回归中，依赖残差的曲率如何与生成势函数和设计矩阵相互作用。
英文摘要
We consider linear regression models fitted by minimizing Bregman losses of the form \[ \frac{1}{n}\sum_{i=1}^n \left[ ϕ(y_i)-ϕ(x_i^\topθ) -ϕ'(x_i^\topθ) \bigl(y_i-x_i^\topθ\bigr) \right], \] where \(y_i\) is the observed response and \(x_i^\topθ\) is the linear prediction. Even when the generating potential \(ϕ\) is strongly convex, the resulting regression objective may be nonconvex in \(θ\). This creates a gap between the convexity of the generating potential and the optimization geometry of the fitted model. The Hessian can be written as a weighted Gram matrix whose weights depend on the derivatives of the potential and the current residuals. This representation gives simple conditions for local strong convexity, smoothness, and conditional linear convergence of gradient descent. For the quadratic--quartic potential, we derive an exact scalar convexity condition, identify the interval of negative curvature, and obtain local and global sufficient conditions for positive curvature. Numerical experiments indicate that these scalar conditions may fail while the full Hessian remains positive definite at the evaluated points. They also indicate that the range of tested gradient-descent step sizes leading to convergence decreases as the quartic parameter grows. These results characterize how residual-dependent curvature interacts with the generating potential and the design matrix in reverse Bregman regression.
Comments21 pages, 3 figures, 4 tables
