发表机构
Center for Computational Mathematics, Flatiron Institute（计算数学中心，Flatiron研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对均匀量化平稳高斯过程，提出解析近似与蒙特卡洛估计两种熵率计算方法，后者基于隐马尔可夫粒子滤波，实验验证近似精度并对比无损编码器性能。
AI 中文摘要
我们考虑在均匀量化后观测到的平稳高斯过程。所得整数序列的熵率是对量化信号进行无损压缩的基本极限，但它没有闭式表达式，并且经典的高分辨率近似在谱密度在部分频带上很小或消失时会失效，这种情况在平滑或滤波后经常发生。这里我们提出两种计算该熵率的方法。第一种是解析近似，通过将精确抖动恒等式与Kolmogorov-Szegő公式相结合获得；量化噪声功率作为谱密度的下限，因此在经典公式失效的地方，熵率保持有限。第二种是对精确熵率的蒙特卡洛估计。其最小相位谱因子将过程表示为高斯创新的有限滑动平均，使量化序列成为隐马尔可夫过程，其最优（完全自适应）粒子滤波器具有闭式解；根据Shannon-McMillan-Breiman定理，熵率可由滤波器在单个长序列上的对数似然得出。在跨越六个过程族的实验中，对于高达信号标准差的量化步长，在大多数参数范围内，近似值与估计值相差在每样本几毫比特以内，包括经典公式失效的强滤波过程；在最强的滤波点上，差异增长到熵率的几个百分点，而对于更粗糙的步长，应使用估计器。我们还比较了无损编码器与该极限。在步长为标准差的四分之一时，线性预测编码后接熵编码器达到熵率的2%至4%以内，FLAC达到3%至32%以内，而原始样本上的五种通用压缩器则高出12%至89%。
英文摘要
We consider a stationary Gaussian process observed after uniform quantization. The entropy rate of the resulting integer sequence is the fundamental limit on lossless compression of the quantized signal, but it has no closed form, and the classical high-resolution approximation breaks down whenever the spectral density is small or vanishing on part of the band, as happens routinely after smoothing or filtering. Here we present two methods for computing the rate. The first is an analytical approximation obtained by combining an exact dithering identity with the Kolmogorov-Szegő formula; the quantization noise power acts as a floor on the spectral density, so the rate remains finite where the classical formula fails. The second is a Monte Carlo estimate of the exact rate. Its minimum-phase spectral factor represents the process as a finite moving average of Gaussian innovations, making the quantized sequence a hidden Markov process whose optimal (fully adapted) particle filter is available in closed form; by the Shannon-McMillan-Breiman theorem the rate follows from the filter's log-likelihood on a single long sequence. In experiments across six process families, the approximation agrees with the estimate to within a few millibits per sample over most of the parameter range for quantization steps up to the signal standard deviation, including strongly filtered processes on which the classical formula fails; at the most strongly filtered points the discrepancy grows to a few percent of the rate, and for much coarser steps the estimator should be used. We also compare lossless coders against this limit. At a step of one quarter of the standard deviation, linear predictive coding followed by an entropy coder comes within 2 to 4 percent of the entropy rate and FLAC within 3 to 32 percent, whereas five general-purpose compressors on the raw samples remain 12 to 89 percent above it.
CommentsCode, data, and an interactive demonstration at https://github.com/magland/entropy-gaussian-process-paper
