关于潜在扩散Transformer（DiTs）的统计速率与可证明高效的准则
On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)
AI总结：
该研究在低维线性潜在空间假设下，分析潜在扩散Transformer（DiTs）的统计极限（逼近误差、样本复杂度、分布恢复）与计算极限（前向推理、反向计算难度），提出高效准则与近线性时间加速方法，证明其性能由子空间维度主导，有潜力规避高维数据挑战。
摘要：
我们在低维线性潜在空间假设下，研究潜在扩散Transformer（DiTs）的统计与计算极限。在统计层面，我们研究了DiTs得分函数的通用逼近性与样本复杂度，以及初始数据的分布恢复特性。具体而言，在温和的数据假设下，我们推导了潜在DiTs得分网络的逼近误差界，该误差界关于潜在空间维度呈次线性。此外，我们推导了相应的样本复杂度界，并证明由估计的得分函数生成的数据分布会收敛到原始分布的邻近区域。在计算层面，我们在强指数时间假设（SETH）下，刻画了潜在DiTs前向推理与反向计算的难度。对于前向推理，我们确定了所有潜在DiTs推理算法的高效准则，并通过将效率推进至近线性时间推理来验证我们的理论。对于反向计算，我们利用DiTs训练梯度计算中的低秩结构来实现可能的算法加速。具体而言，我们通过将DiTs梯度转化为一系列误差有界的链式低秩近似，证明这种加速可实现近线性时间的潜在DiTs训练。在低维假设下，我们证明统计速率与计算效率均由子空间的维度主导，这表明潜在DiTs有潜力规避初始数据高维性带来的挑战。
英文摘要：
We investigate the statistical and computational limits of latent Diffusion Transformers (DiTs) under the low-dimensional linear latent space assumption. Statistically, we study the universal approximation and sample complexity of the DiTs score function, as well as the distribution recovery property of the initial data. Specifically, under mild data assumptions, we derive an approximation error bound for the score network of latent DiTs, which is sub-linear in the latent space dimension. Additionally, we derive the corresponding sample complexity bound and show that the data distribution generated from the estimated score function converges toward a proximate area of the original one. Computationally, we characterize the hardness of both forward inference and backward computation of latent DiTs, assuming the Strong Exponential Time Hypothesis (SETH). For forward inference, we identify efficient criteria for all possible latent DiTs inference algorithms and showcase our theory by pushing the efficiency toward almost-linear time inference. For backward computation, we leverage the low-rank structure within the gradient computation of DiTs training for possible algorithmic speedup. Specifically, we show that such speedup achieves almost-linear time latent DiTs training by casting the DiTs gradient as a series of chained low-rank approximations with bounded error. Under the low-dimensional assumption, we show that the statistical rates and the computational efficiency are all dominated by the dimension of the subspace, suggesting that latent DiTs have the potential to bypass the challenges associated with the high dimensionality of initial data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
