A causal rate distortion function with a general fidelity criterion is formulated on abstract alphabets and a coding theorem is derived. Existence of the minimizing kernel is shown using the topology of weak convergence of probability measures. The optimal reconstruction kernel is derived, which is causal, and certain properties of the causal rate distortion function are presented.
In this paper, we consider the mismatched rate-distortion problem, in which the encoding is done using a codebook, and the encoder chooses the minimum-distortion codeword according to a mismatched distortion function that differs from the true one. For the case of discrete memoryless sources, we establish achievable rate-distortion bounds using multi-user coding techniques, namely, superposition coding and expurgated parallel coding. We study examples where these attain the matched rate-distortion trade-off but a standard ensemble with independent codewords fails to do so. On the other hand, i…
Motivated from the fact that universal source coding on countably infinite alphabets is not feasible, this work introduces the notion of almost lossless source coding. Analog to the weak variable-length source coding problem studied by Han (IEEE TIT, 2000, 46, 1217-1226), almost lossless source coding aims at relaxing the lossless block-wise assumption to allow an average per-letter distortion that vanishes asymptotically as the block-length tends to infinity. In this setup, we show on one hand that Shannon entropy characterizes the minimum achievable rate (similarly to the case of finite alph…
In successive refinement of information, the decoder refines its representation of the source progressively as it receives more encoded bits. The rate-distortion region of successive refinement describes the minimum rates required to attain the target distortions at each decoding stage. In this paper, we derive a parametric characterization of the rate-distortion region for successive refinement of abstract sources. Our characterization extends Csiszar's result to successive refinement, and generalizes a result by Tuncel and Rose, applicable for finite alphabet sources, to abstract sources. Th…
In this paper we analyze the joint rate distortion function (RDF), for a tuple of correlated sources taking values in abstract alphabet spaces (i.e., continuous) subject to two individual distortion criteria. First, we derive structural properties of the realizations of the reproduction Random Variables (RVs), which induce the corresponding optimal test channel distributions of the joint RDF. Second, we consider a tuple of correlated multivariate jointly Gaussian RVs, $X_1 : Ω\rightarrow {\mathbb R}^{p_1}, X_2 : Ω\rightarrow {\mathbb R}^{p_2}$ with two square-error fidelity criteria, and we de…
Joint universal lossy coding and identification of stationary mixing sources with general alphabets
Beckman Institute for Advanced Science and Technology, University of Illinois（伊利诺伊大学贝克曼先进科学技术研究所）；Duke University（杜克大学）
我们考虑的问题，联合通用可变速率有损编码和识别的参数类的固定$β$混合源一般（波兰）字母。压缩性能是衡量的拉格朗日，而识别性能是衡量的变化之间的距离真实源和估计源。假设源以足够快的速率混合并满足一定的平滑性和Vapnik-Chervonenkis可学习性条件，则对于有界度量失真，存在用于联合有损压缩和识别的通用方案，其拉格朗日冗余收敛到零，如块长度$n$趋于无穷大时的$\sqrt{V_n \log n /n}$，其中$V_n$是由源的$n$维边缘分布定义的某类决策区域的Vapnik-Chervonenkis维数;此外，对于每个$n$，解码器可以识别活动源的$n$维边缘，直到在变距离中的半径为$O(\sqrt{V_n\log n/n})$的球，最终概率为1。结果补充了几个例子的参数源满足正则性条件。
An effective 'on-the-fly' mechanism for stochastic lossy coding of Markov sources using string matching techniques is proposed in this paper. Earlier work has shown that the rate-distortion bound can be asymptotically achieved by a 'natural type selection' (NTS) mechanism which iteratively encodes asymptotically long source strings (from an unknown source distribution P) and regenerates the codebook according to a maximum likelihood distribution framework, after observing a set of K codewords to 'd-match' (i.e., satisfy the distortion constraint for) a respective set of K source words. This re…
A privacy-utility tradeoff is developed for an arbitrary set of finite-alphabet source distributions. Privacy is quantified using differential privacy (DP), and utility is quantified using expected Hamming distortion maximized over the set of distributions. The family of source distribution sets (source sets) is categorized into three classes, based on different levels of prior knowledge they capture. For source sets whose convex hull includes the uniform distribution, symmetric DP mechanisms are optimal. For source sets whose probability values have a fixed monotonic ordering, asymmetric DP m…
A Constrained BA Algorithm for Rate-Distortion and Distortion-Rate Functions
Tsinghua University（清华大学）；Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China（电子科技大学长三角研究院（湖州））；Huawei Technologies Co. Ltd.（华为技术有限公司）；University of Science and Technology of China（中国科学技术大学）
本文改进 Blahut-Arimoto 算法，通过 Newton 求根动态更新乘子，直接计算给定失真下的 RD 及 DR 函数，并证明其 O(1/n) 收敛率与运算复杂度，实验显示显著加速。
