当大数据实际上是低秩的，或某些函数生成矩阵的逐元素逼近
When big data actually are low-rank, or entrywise approximation of certain function-generated matrices
AI总结：
本文澄清了“大数据矩阵近似低秩”的误解，为三类函数生成矩阵提供了秩独立于维度且随对数增长的逐元素逼近理论，并扩展到张量列逼近及注意力机制场景。
摘要：
本文关注通过采样两个$m$维变量的光滑函数所生成的矩阵的低秩逼近问题。我们识别了围绕一个断言的若干误解，该断言声称：对于一类特定的解析函数，这样的$n \times n$矩阵允许秩与$m$无关且随$\log(n)$增长的精确逐元素逼近——通俗地称为“大数据矩阵近似低秩”。我们为支持该断言所呈现的数值结果提供了理论解释，描述了三个更窄的函数类别，对于这些类别，函数生成的矩阵可以在逐元素误差为$\varepsilon$阶的情况下被逼近，其秩为$\mathcal{O}(\log(n) \varepsilon^{-2} \log(\varepsilon^{-1}))$，且与维度$m$无关：(i) 两个变量内积的函数，(ii) 变量间欧几里得距离的函数，以及(iii) 平移不变正定核。我们将论证扩展到由多个变量的“高阶内积”函数生成的张量的张量列逼近。我们在(a)增长数据集和(b)变换器神经网络中注意力的低秩逼近背景下讨论我们的结果。
英文摘要：
The article concerns low-rank approximation of matrices generated by sampling a smooth function of two $m$-dimensional variables. We identify several misconceptions surrounding a claim that, for a specific class of analytic functions, such $n \times n$ matrices admit accurate entrywise approximation of rank that is independent of $m$ and grows as $\log(n)$ -- colloquially known as ''big-data matrices are approximately low-rank''. We provide a theoretical explanation of the numerical results presented in support of this claim, describing three narrower classes of functions for which function-generated matrices can be approximated within an entrywise error of order $\varepsilon$ with rank $\mathcal{O}(\log(n) \varepsilon^{-2} \log(\varepsilon^{-1}))$ that is independent of the dimension $m$: (i) functions of the inner product of the two variables, (ii) functions of the Euclidean distance between the variables, and (iii) shift-invariant positive-definite kernels. We extend our argument to tensor-train approximation of tensors generated with functions of the ''higher-order inner product'' of their multiple variables. We discuss our results in the context of low-rank approximation of (a) growing datasets and (b) attention in transformer neural networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
