用于带逐坐标保证的最小二乘的Hadamard展平与高斯池化草图
Hadamard Flattening and Gaussian Pooling Sketch for Least Squares with Coordinate-wise Guarantee
- Massachusetts Institute of Technology（麻省理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究针对带逐坐标保证的最小二乘问题，提出结合Hadamard展平与高斯池化的新随机变换，将草图行数优化至O(ε⁻²d log d)，解决了现有方法的独立性假设缺陷，实现高效的超约束ℓ₂回归加速。
AI中文摘要：
随机草图与求解算法通过将输入替换为更小的问题来加速超约束ℓ₂回归。标准子空间嵌入保证回归代价几乎被保留，但解的逐坐标精度更微妙：我们希望解向量本身在ℓ_∞范数下接近最优解。具体而言，我们要找到向量x'∈ℝ^d，使得‖x'-x*‖_∞≤(ε/√d)·‖Ax^⋆-b‖₂·‖A†‖_op。Price、Song和Woodruff率先研究该问题，证明了具有O(ε⁻²d^(1+Θ(√log log n/log d)))行的子采样随机Hadamard变换（SRHT）可达到此保证。Song、Ye、Yin和Zhang的后续工作声称将行数改进为O(ε⁻²d log³n)，但其证明依赖的独立性假设并不普遍成立，我们给出了一个明确的反例说明其失效。为达到真正近线性于d的行数，我们引入一种新的快速稠密随机变换，结合随机Hadamard展平、随机置换及平衡不相交高斯池化。在Hadamard与置换阶段的条件下，草图问题变为精确高斯回归，其中噪声与整个草图设计独立；这种条件独立性正是早期论证所缺失的。我们的草图以m=O(ε⁻²d log d)行实现ℓ_∞保证，使用一次Hadamard变换，其填充内部维度N=Õ(n+ε⁻²d³)，且应用高效：草图对(SA,Sb)可在O(Nd log N)=Õ(nd+ε⁻²d⁴)时间内计算。
英文摘要：
Randomized sketch-and-solve algorithms accelerate overconstrained $\ell_2$ regression by replacing the input with a smaller problem. Standard subspace embeddings guarantee that the cost of the regression is nearly preserved, but coordinate-wise accuracy of the solution is more delicate: we want the solution vector itself to be close to the optimal solution in $\ell_\infty$ norm. In particular, we want to find a vector $x'\in \mathbb{R}^d$ such that $\|x'-x^*\|_\infty\leq \fracε{\sqrt d}\cdot \|Ax^\star-b\|_2\cdot \|A^\dagger\|_{\rm op}$. Price, Song and Woodruff initiated the study of this problem and showed that the subsampled randomized Hadamard transform (SRHT) with $O(ε^{-2} d^{1+Θ(\sqrt{\log\log n/\log d})})$ rows achieves this guarantee. A subsequent work of Song, Ye, Yin and Zhang claimed to improve the row count to $O(ε^{-2}d\log^3 n)$. Unfortunately, their proof relies on an independence assumption that does not hold in general, and we exhibit an explicit instance on which it fails. To achieve a truly nearly-linear-in-$d$ row count, we introduce a new fast, dense randomized transform, which combines a randomized Hadamard flattening, a random permutation, and balanced, disjoint Gaussian pooling. Conditioned on the Hadamard-and-permutation stage, the sketched problem becomes an exact Gaussian regression in which the noise is independent of the entire sketched design; this conditional independence is exactly what the earlier argument was missing. Our sketch yields the $\ell_\infty$ guarantee with $m=O(ε^{-2}d\log d)$ rows, uses one Hadamard pass with a padded internal dimension $N=\widetilde{O}(n+ε^{-2}d^3)$, and is efficient to apply: the sketched pair $(SA, Sb)$ can be computed in $O(Nd\log N)=\widetilde{O}(nd+ε^{-2}d^4)$ time.
