面向压缩与隐私的可扩展离散到连续信道模拟
Scalable Discrete-to-Continuous Channel Simulation for Compression and Privacy
浏览论文内容
中文总结 AI 辅助
提出一种固定随机样本数、运行时间与信道无关的离散到连续信道模拟方案，通过潜在置换和指数竞赛实现压缩与隐私应用，可扩展到长块长度。
中文摘要 AI 辅助
信道模拟近来已成为机器学习系统中的有用组件，在这些系统中，需要压缩来自指定概率分布的样本。然而，一般的信道模拟算法常常面临高计算成本、随机停止时间，甚至在最坏情况下可能需要生成无限数量的共享随机样本。我们提出了一种用于离散到连续信道的精确和近似模拟的方案，该方案相反地使用固定数量的随机样本，因此其运行时间与信道和输入无关。与现有的信道模拟方案（从提议分布生成一系列独立样本）不同，我们的方法从每个潜在目标分布生成一个样本，或者可选地生成固定数量的样本。然后，我们在进行样本选择之前对样本应用潜在置换，并使用指数竞赛。我们的方案在生成的样本数量和压缩率之间提供了灵活的权衡。利用极化和多级编码，我们将方法扩展到处理长块长度，时间复杂度为$O(n \log n)$，以便从降低的每符号开销中获益。最后，我们通过随机VQ-VAE的可变速率压缩和通过精确模拟高斯机制进行通信高效的差分私有分布式均值估计来展示应用。
英文摘要
Channel simulation has recently emerged as a useful component in machine learning systems where samples from a prescribed probability distribution are to be compressed. Yet, general channel simulation algorithms often suffer from high computational costs, random stopping times or, in the worst case, can require generating an infinite number of shared random samples. We introduce a scheme for both exact and approximate simulation of discrete-to-continuous channels which conversely uses a fixed number of random samples, and therefore has a runtime independent of the channel and the input. Unlike existing channel simulation schemes which generate a sequence of independent samples from a proposal distribution, our approach generates one sample, or alternatively a fixed number of samples, from each potential target distribution. We then apply a latent permutation to the samples before performing sample selection using an exponential race. Our scheme provides a flexible tradeoff between the number of generated samples and the compression rate. Using polar and multilevel coding, we scale our approach to handle long blocklengths in $O(n \log n)$ time in order to benefit from reduced per-symbol overhead. We conclude by demonstrating applications to variable-rate compression with stochastic VQ-VAEs and communication-efficient differentially private distributed mean estimation via exact simulation of the Gaussian mechanism.
发表机构
- University of Toronto（多伦多大学）
机构由 AI 辅助整理，请以论文原文为准。
