连接混合线性回归中的分数匹配、最大似然和期望最大化

中文摘要 AI 辅助

我们研究了在混合权重未知的混合线性回归（MLR）中响应的方差保持扩散。我们的分析将分数匹配的统计保证与固定扩散噪声水平下的损失几何和优化信号分离开来。KL散度将通过扩散路径积分的去噪分数匹配目标与似然和终端差异联系起来。在温和的正则条件和终端调度下，所得估计器收敛到MLR的真实参数，其缩放误差收敛到最大似然估计器的高斯极限。在固定的扩散噪声水平尺度下，我们推导出一个分解，将分数匹配损失与交叉熵和期望最大化（EM）算子联系起来。该分解产生了一个与EM相关的低噪声梯度展开，并带有潜在方差的额外修正项。在高噪声极限下，我们进一步描述了在各向同性协方差下该极限损失上的梯度下降。沿着固定的高信噪比射线，分数匹配不平衡梯度和潜在方差项逐点趋于零。数值实验说明了我们的理论发现和统计保证。

英文摘要

We study variance-preserving diffusion of the response in mixed linear regression (MLR) with unknown mixing weights. Our analysis separates the statistical guarantees of score matching from the loss geometry and optimization signal at a fixed diffusion noise level. The KL divergence links the denoising score matching objective integrated over the diffusion path with the likelihood and a terminal discrepancy. Under mild regularity conditions and terminal schedule, the resulting estimator converges up to the ground truth parameters of MLR, and its scaled error converges to the Gaussian limit of the maximum-likelihood estimator. At a fixed scale of the diffusion noise level, we derive a decomposition linking the score matching loss to cross-entropy and Expectation-Maximization (EM) operators. This decomposition yields an EM-related low-noise gradient expansion with additional correction terms of latent variance. In the high-noise limit, we further characterize gradient descent on this limiting loss under isotropic covariance. Along fixed high signal-to-noise ratio rays, the score matching imbalance gradient and the latent-variance term tend to zero pointwise. Numerical experiments illustrate our theoretical findings and statistical guarantees.

发表机构

• Purdue University（普渡大学）

机构由 AI 辅助整理，请以论文原文为准。