← 研踪 PaperTrailarXiv:2609.18784 · v3
Computing the entropy rate of a quantized stationary Gaussian process
首次提交 2026-09-16 23:04
版本更新 2026-11-10 03:00
雷达观测 2026-11-11 11:04
摘要
The entropy rate of a uniformly quantized stationary Gaussian process is the fundamental limit on its lossless compression. It has no closed form, and the classical high-resolution approximation breaks down whenever the spectral density is small on part of the band, as routine smoothing and filtering produce. Here we present two methods for computing the rate. The first is an analytical approximation, obtained from an exact dithering identity and the Kolmogorov-Szegő formula, in which the quantization noise power acts as a floor on the spectral density. The second is a Monte Carlo estimate of the exact rate: a minimum-phase moving-average representation makes the quantized sequence a hidden Markov process whose optimal particle filter is available in closed form, and the filter's log-likelihood on one long sequence gives the rate. The approximation agrees with the estimate to within a few millibits per sample over most of the parameter range for quantization steps up to the signal standard deviation, including processes on which the classical formula fails; for coarser steps the estimate should be used instead. At a step of a quarter of the standard deviation, linear predictive coding comes within 2 to 4 percent of the rate, FLAC within 3 to 32 percent, and general-purpose compressors remain 12 to 89 percent above it.
作者备注：Code, data, and an interactive demonstration at https://github.com/magland/entropy-gaussian-process-paper
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-10 03:00 · 最新观测版本
Computing the entropy rate of a quantized stationary Gaussian process
The entropy rate of a uniformly quantized stationary Gaussian process is the fundamental limit on its lossless compression. It has no closed form, and the classical high-resolution approximation breaks down whenever the spectral density is small on part of the band, as routine smoothing and filtering produce. Here we present two methods for computing the rate. The first is an analytical approximation, obtained from an exact dithering identity and the Kolmogorov-Szegő formula, in which the quantization noise power acts as a floor on the spectral density. The second is a Monte Carlo estimate of the exact rate: a minimum-phase moving-average representation makes the quantized sequence a hidden Markov process whose optimal particle filter is available in closed form, and the filter's log-likelihood on one long sequence gives the rate. The approximation agrees with the estimate to within a few millibits per sample over most of the parameter range for quantization steps up to the signal standard deviation, including processes on which the classical formula fails; for coarser steps the estimate should be used instead. At a step of a quarter of the standard deviation, linear predictive coding comes within 2 to 4 percent of the rate, FLAC within 3 to 32 percent, and general-purpose compressors remain 12 to 89 percent above it.
查看此版本解读 →查看此版本原文 ↗