垂直分割分布式模型中稀疏协方差估计的信息论界
Information-Theoretic Bounds for Sparse Covariance Estimation in the Vertical-Split Distributed Model
AI总结：
研究垂直分割分布式设置下稀疏协方差矩阵的极小化估计误差，证明稀疏性可降低通信和样本复杂度，并给出匹配的上下界。
摘要：
我们研究了垂直分割（特征分割）设置下分布式协方差矩阵估计的极小化估计误差，其中两个智能体各自观测 $m$ 个独立同分布的子高斯样本的不同坐标，并向中心服务器传输有限比特数。虽然 Rahmani 等人 [2025] 对稠密（无结构）互协方差矩阵建立了近乎紧的界，但我们研究了在互协方差 $C_{21}$ 上施加元素级 $s$-稀疏性是否能降低所需的通信和样本复杂度。与水平分割设置（Braverman 等人 [2016] 表明稀疏性不能降低均值估计的通信成本）相反，我们证明在垂直分割中稀疏性确实有助于互协方差估计。具体地，我们建立了极小化下界，表明每个智能体的通信预算为 $B_k = \Omega(\sigma^4 d_k\\, s' \log(d_1 d_2/s')/\varepsilon^2)$，互协方差估计的样本复杂度为 $m = \Omega(\sigma^4\\, s' \log(d_1 d_2/s')/\varepsilon^2)$，其中 $s' = s \wedge d_{\min}$。对于 $1$-稀疏情况，与稠密率相比，这实现了从 $d_1 d_2$ 到 $\log(d_1 d_2)$ 的指数级改进。我们的下界通过 Fano 方法建立，使用基于 Varshamov--Gilbert 型论证的显式稀疏打包（针对符号部分置换矩阵）并结合 Rahmani 等人 [2025] 的条件强数据处理不等式。我们通过匹配的可实现方案证明了界的紧性，该方案基于覆盖网量化和逐元素硬阈值，在多项式对数因子内达到 $s$-稀疏下界。
英文摘要：
We study the minimax estimation error for distributed covariance matrix estimation in the vertical-split (feature-split) setting, where two agents each observe different coordinates of~$m$ i.i.d.\ sub-Gaussian samples and communicate a limited number of bits to a central server. While \cite{rahmani2025fundamental} established nearly tight bounds for dense (unstructured) cross-covariance matrices, we investigate whether imposing elementwise $s$-sparsity on the cross-covariance $C_{21}$ can reduce the required communication and sample complexity. We show that sparsity can reduce the communication required for cross-covariance estimation in the vertical split at a fixed target distortion. Specifically, for sufficiently large $d_1d_2/s'$ and $0<\varepsilon<σ^2\sqrt{s'}/32$, any scheme achieving expected Frobenius distortion at most $\varepsilon$ must satisfy $B_k = Ω(σ^4 d_k\, s' \log(d_1 d_2/s')/\varepsilon^2)$ and $m = Ω(σ^4\, s' \log(d_1 d_2/s')/\varepsilon^2)$ for cross-covariance estimation, where $s' = s \wedge d_{\min}$. For the $1$-sparse case, our achievable scheme reduces the $d_1d_2$ factor in the dense communication rate to $\log(d_1d_2)$, up to polylogarithmic factors, for the cross-covariance communication component in the matching regime. Our lower bounds are established via Fano's method with an explicit sparse packing using a Varshamov--Gilbert-type argument for signed partial permutation matrices combined with the Conditional Strong Data Processing Inequality of \cite{rahmani2025fundamental}. We show that the communication lower bound is tight up to polylogarithmic factors under the conditions of Remark 4.9, using an achievable scheme based on covering-net quantization and entry-wise hard thresholding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
