Alexandre Graell i Amat, Gianluigi Liva, Fabian Steiner
Approaching capacity with low complexity is a very challenging task. In this paper, we review and compare three promising coding solutions to achieve that, which are suitable for future very high-throughput, low-complexity optical communications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We present a new algorithm for dynamic prefix-free coding, based on Shannon coding. We give a simple analysis and prove a better upper bound on the length of the encoding produced than the corresponding bound for dynamic Huffman coding. We show how our algorithm can be modified for efficient length-restricted coding, alphabetic coding and coding with unequal letter costs.
An interference alignment perspective is used to identify the simplest instances (minimum possible number of edges in the alignment graph, no more than 2 interfering messages at any destination) of index coding problems where non-Shannon information inequalities are necessary for capacity characterization. In particular, this includes the first known example of a multiple unicast (one destination per message) index coding problem where non-Shannon information inequalities are shown to be necessary. The simplest multiple unicast example has 7 edges in the alignment graph and 11 messages. The si…
Statements of Shannon's Noiseless Coding Theorem by various authors, including the original, are reviewed and clarified. Traditional statements of the theorem are often unclear as to when it applies. A new notation is introduced and the domain of application is clarified. An examination of the bounds of the Theorem leads to a new symmetric restatement. It is shown that the extended upper bound is an acheivable upper bound, giving symmetry to the theorem.The relation of information entropy to the physical entropy of Gibbs and Boltmann is illustrated. Consequently, the study of Shannon Entropy i…
Homophonic coding is a framework to reversibly convert a message into a sequence with some target distribution. This is a promising tool to generate a codeword with a biased code-symbol distribution, which is required for capacity-achieving communication by asymmetric channels. It is known that asymptotically optimal homophonic coding can be realized by a Fixed-to-Variable (FV) length code using an interval algorithm similar to a random number generator. However, FV codes are not preferable as a component of channel codes since a decoding error propagates to all subsequent codewords. As a solu…
A common complaint about adaptive prefix coding is that it is much slower than static prefix coding. Karpinski and Nekrich recently took an important step towards resolving this: they gave an adaptive Shannon coding algorithm that encodes each character in (O (1)) amortized time and decodes it in (O (\log H)) amortized time, where $H$ is the empirical entropy of the input string $s$. For comparison, Gagie's adaptive Shannon coder and both Knuth's and Vitter's adaptive Huffman coders all use (Θ(H)) amortized time for each character. In this paper we give an adaptive Shannon coder that both enco…
Shannon's secrecy system is studied in a setting, where both the legitimate decoder and the wiretapper have access to side information sequences correlated to the source, but the wiretapper receives both the coded information and the side information via channels that are more noisy than the respective channels of the legitmate decoder, which in turn, also shares a secret key with the encoder. A single--letter characterization is provided for the achievable region in the space of five figures of merit: the equivocation at the wiretapper, the key rate, the distortion of the source reconstructio…
Starting from Shannon's celebrated 1948 channel coding theorem, we trace the evolution of channel coding from Hamming codes to capacity-approaching codes. We focus on the contributions that have led to the most significant improvements in performance vs. complexity for practical applications, particularly on the additive white Gaussian noise (AWGN) channel. We discuss algebraic block codes, and why they did not prove to be the way to get to the Shannon limit. We trace the antecedents of today's capacity-approaching codes: convolutional codes, concatenated codes, and other probabilistic coding …
Many statements from the classic information theory (the theory of Shannon's entropy) have natural counterparts in the algorithmic information theory (in the framework of Kolmogorov complexity). In this paper we discuss one simple instance of the parallelism between Shannon's and Kolmogorov's theories: we prove in the setting of Kolmogorov complexity a version of Wolf's characterization of admissible rates for the fork network.
We study and compare the Shannon capacity region and the stable throughput region for a random access system in which source nodes multicast their messages to multiple destination nodes. Under an erasure channel model which accounts for interference and allows for multipacket reception, we first characterize the Shannon capacity region. We then consider a queueing-theoretic formulation and characterize the stable throughput region for two different transmission policies: a retransmission policy and random linear coding. Our results indicate that for large blocklengths, the random linear coding…
Universal Coding for Shannon Ciphers under Side-Channel Attacks
本文研究侧信道攻击下 Shannon 密码系统的通用编码，针对任意明文分布、密钥噪声观测信道和物理测量设备，推导兼顾可靠解密与信息论安全的可达速率区域。
The ordering of communication channels was first introduced by Shannon. In this paper, we aim to find a characterization of the Shannon ordering. We show that $W'$ contains $W$ if and only if $W$ is the skew-composition of $W'$ with a convex-product channel. This fact is used to derive a characterization of the Shannon ordering that is similar to the Blackwell-Sherman-Stein theorem. Two channels are said to be Shannon-equivalent if each one is contained in the other. We investigate the topologies that can be constructed on the space of Shannon-equivalent channels. We introduce the strong topol…
We consider the storage-retrieval rate tradeoff in private information retrieval (PIR) systems using a Shannon-theoretic approach. Our focus is mostly on the canonical two-message two-database case, for which a coding scheme based on random codebook generation and the binning technique is proposed. This coding scheme reveals a hidden connection between PIR and the classic multiple description source coding problem. We first show that when the retrieval rate is kept optimal, the proposed non-linear scheme can achieve better performance over any linear scheme. Moreover, a non-trivial storage-ret…
We consider transmission of stationary and ergodic sources over non-ergodic composite channels with channel state information at the receiver (CSIR). Previously we introduced alternate capacity definitions to Shannon capacity, including the capacity versus outage and the expected capacity. These generalized definitions relax the constraint of Shannon capacity that all transmitted information must be decoded at the receiver. In this work alternate end-to-end distortion metrics such as the distortion versus outage and the expected distortion are introduced to relax the constraint that a single d…
We introduce the (private) entropy of a directed graph (in a new network coding sense) as well as a number of related concepts. We show that the entropy of a directed graph is identical to its guessing number and can be bounded from below with the number of vertices minus the size of the graph's shortest index code. We show that the Network Coding solvability of each specific multiple unicast network is completely determined by the entropy (as well as by the shortest index code) of the directed graph that occur by identifying each source node with each corresponding target node. Shannon's info…
This paper begins with a discussion of integration over probability types (p-types). After doing that, the paper re-visits 3 mainstay problems of classical (non-quantum) Shannon Information Theory (SIT): source coding without distortion, channel coding, and source coding with distortion. The paper proves well-known, conventional results for each of these 3 problems. However, the proofs given for these results are not conventional. They are based on complex integration techniques (approximations obtained by applying the method of steepest descent to p-type integrals) instead of the usual delta …
Although Shannon mutual information has been widely used, its effective calculation is often difficult for many practical problems, including those in neural population coding. Asymptotic formulas based on Fisher information sometimes provide accurate approximations to the mutual information but this approach is restricted to continuous variables because the calculation of Fisher information requires derivatives with respect to the encoded variables. In this paper, we consider information-theoretic bounds and approximations of the mutual information based on Kullback--Leibler divergence and Ré…
This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen-Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. The paper presents strictly positive lower bounds on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, for the following two codi…
A simple proof for the Shannon coding theorem, using only the Markov inequality, is presented. The technique is useful for didactic purposes, since it does not require many preliminaries and the information density and mutual information follow naturally in the proof. It may also be applicable to situations where typicality is not natural.
Revisiting Shannon's Source Coding Theorem with Distributional Uncertainty under the Nonlinear Expectation Theory
本文基于非线性期望理论，针对分布不确定的信源建立非线性信源编码定理，推广了信息熵概念，为通信网络中信源的分析提供了更全面的理论框架。
We consider the Shannon cipher system in a setting where the secret key is delivered to the legitimate receiver via a channel with limited capacity. For this setting, we characterize the achievable region in the space of three figures of merit: the security (measured in terms of the equivocation), the compressibility of the cryptogram, and the distortion associated with the reconstruction of the plaintext source. Although lossy reconstruction of the plaintext does not rule out the option that the (noisy) decryption key would differ, to a certain extent, from the encryption key, we show, nevert…
A Gallager-Type Redundancy Bound for Binary Shannon-Fano Coding
研究二进制香农 - 法诺编码冗余界，通过构建法诺树等方法，得出依赖最大源概率\(p_1\)的显式七段包络，是首个此类冗余界。
Physics-Aware, Shannon-Optimal Compression via Arithmetic Coding for Distributional Fidelity
本文提出一种基于信息论的方法，利用无损压缩评估数据集的分布一致性，通过物理指导的概率表示提供绝对的信度标准，同时提升压缩效率。
This article serves as a supplement to the recently published call for participation in a Research Topic [1] that is timed to commemorate the 75th anniversary of Shannon's pioneering 1948 paper [2]. Here, we include some citations of key and relevant literature, which reflect our opinions/perspectives on the proposed topic, and serve as guidance to potential submissions.
This paper considers lossy source coding of $n$-dimensional memoryless sources and shows an explicit approximation to the minimum source coding rate required to sustain the probability of exceeding distortion $d$ no greater than $ε$, which is simpler than known dispersion-based approximations. Our approach takes inspiration in the celebrated classical result stating that the Shannon lower bound to rate-distortion function becomes tight in the limit $d \to 0$. We formulate an abstract version of the Shannon lower bound that recovers both the classical Shannon lower bound and the rate-distortion…
Identification Codes and Post-Shannon Communication: Theory, Architectures, and Emerging Applications
本文围绕识别编码展开，它扩展了香农通信范式。介绍其理论基础，双指数增长特性带来新通信架构。探讨基于该范式的新兴架构与应用领域，强调实际用例，还展示了相关系统概念、行业观点及实现示例。
New and novel way for resources allocation in wireless communication has been proposed in this paper. Under this new method, it has been shown that the required power budget becomes independent of the number of served terminals in the downlink. However, the required power depends only of the coverage area, i.e. the channel losses at the cell boarder. Therefore, huge number (theoretically any number) of terminals could be supported concurrently at finite and small downlink power budget. This could be very useful to support the downlink signalling channels in HSPA+, LTE, and 5G. It can be very u…
Foundations of information theory for coding theory
本文通过信息论基础构建，探讨了编码理论中噪声信道的理论极限及最大似然解码原理。
Let $W$ be a binary-input memoryless symmetric (BMS) channel with Shannon capacity $I(W)$ and fix any $α> 0$. We construct, for any sufficiently small $δ> 0$, binary linear codes of block length $O(1/δ^{2+α})$ and rate $I(W)-δ$ that enable reliable communication on $W$ with quasi-linear time encoding and decoding. Shannon's noisy coding theorem established the \emph{existence} of such codes (without efficient constructions or decoding) with block length $O(1/δ^2)$. This quadratic dependence on the gap $δ$ to capacity is known to be best possible. Our result thus yields a constructive version o…
We prove an upper bound on the Shannon capacity of a graph via a linear programming variation. We show that our bound can outperform both the Lovász theta number and the Haemers minimum rank bound. As a by-product, we also obtain a new upper bound on the broadcast rate of Index Coding.
We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories: Shannon entropy versus Kolmogorov complexity, the relation of both to universal coding, Shannon mutual information versus Kolmogorov (`algorithmic') mutual information, probabilistic sufficient statistic versus algorithmic sufficient statistic (related to lossy compression in the Shannon theory versus meaningful information in the Kolmogorov theory), and rate dis…
Security constraint results in \textit{rate-loss} in wiretap channels. In this paper we propose a coding scheme for two user Multiple Access Channel with Wiretap (MAC-WT), where previous messages are used as a key to enhance the secrecy rates of both the users until we achieve the usual capacity region of a Multiple Access Channel (MAC) without the wiretapper (Shannon capacity region). With this scheme all the messages transmitted in the recent past are secure with respect to all the information of the eavesdropper till now. To achieve this goal we introduce secret key buffers at both the user…
This paper applies Information Theoretic analysis to packet-based random multiple access communication systems. A new channel coding approach is proposed for coding within each data packet with built-in support for bursty traffic properties, such as message underflow, and for random access properties, such as packet collision detection. The coding approach does not require joint communication rate determination either among the transmitters or between the transmitters and the receiver. Its performance limitation is characterized by an achievable region defined in terms of communication rates, …
Breaking through the classical Shannon entropy limit: A new frontier through logical semantics
本文首次对具备逻辑演绎推理能力的通信系统进行香农式分析，证明该能力可显著提升通信效率，并主张语义信息理论应纳入演绎推理以充分释放语义潜力。
We compare two different techniques for proving non-Shannon-type information inequalities. The first one is the original Zhang-Yeung's method, commonly referred to as the copy/pasting lemma/trick. The copy lemma was used to derive the first conditional and unconditional non-Shannon-type inequalities. The second technique first appeared in Makarychev et al paper [7] and is based on a coding lemma from Ahlswede and Körner works. We first emphasize the importance of balanced inequalities and provide a simpler proof of a theorem of Chan's for the case of Shannon-type inequalities. We compare the p…
Semantic Huffman Coding using Synonymous Mapping
针对语义通信场景下的信源压缩需求，本文提出基于同义映射与同义集的语义哈夫曼编码，理论上可逼近语义熵，实验显示其在语义无损条件下的压缩效率优于经典哈夫曼编码。
Large Deviation Analysis for the Reverse Shannon Theorem
本文利用 Rényi 散度研究信道模拟的逆向香农定理，刻画了 Rényi 模拟速率，并分别在通信代价速率高于和低于该速率时给出了可靠性函数与线性增长速率，从而确定强逆指数。
We address the problem of simulating an arbitrary binary interactive first-order Markovian protocol over a pair of binary symmetric channels with crossover probability $\varepsilon$. We are interested in the achievable rates of reliable simulation, i.e., in characterizing the smallest possible blowup in communications such that a vanishing error probability (in the protocol length) can be attained. Whereas for general interactive protocols the output of each party may depend on all previous outputs of its counterpart, in a (first-order) Markovian protocol this dependence is limited to the last…
Shannon meets von Neumann: A Minimax Theorem for Channel Coding in the Presence of a Jammer
本文研究了在存在对抗性干扰者的情况下，由一组信道组成的信道编码问题，将其视为编码器-解码器团队与干扰者之间的零和博弈。编码器-解码器团队选择随机的编码和解码策略以最小化传输的平均错误概率，而干扰者则选择状态空间上的分布以最大化此概率。该博弈的极小极大值等同于复合信道的信道编码，我们称之为该问题的香农解。极大极小值对应于寻找具有最大最小可实现错误概率的混合信道。当极小极大和极大极小值相等时，问题被称为具有鞍点或冯·诺依曼解。尽管香农解总是存在的，但冯·诺依曼解可能不存在，因为通信团队的问题具有内在的非凸性。然而，我们证明在大块长极限下，对于所有但有限多个速率，极小极大和极大极小值会趋于相等。我们明确地将此极限值作为速率函数进行表征，并获得了极小极大和极大极小值的紧有限块长界。作为推论，我们得到了在随机码下复合信道的ε-容量的显式表达式——据我们所知，这是首次此类结果。我们的结果展示了复合信道与混合信道之间更深层次的关系，也表明传统的信息论观点，通过香农解表达，最终与通过冯·诺依曼解表达的游戏论观点一致。
A new approach to joint source-channel coding is presented in the context of communicating correlated sources over multiple access channels. Similar to the separation architecture, the joint source-channel coding system architecture in this approach is modular, whereby the source encoding and channel decoding operations are decoupled. However, unlike the separation architecture, the same codeword is used for both source coding and channel coding, which allows the resulting hybrid coding scheme to achieve the performance of the best known joint source-channel coding schemes. Applications of the…
Signal Constellations with Enhanced Energy Efficiency for High-Speed Communication Systems
本文提出SCOPT方法，通过增加归一化信号持续时间提升多维信号星座的最小欧几里得距离，实现比传统方法更高的能效，且结构简单适用于QAM等标准调制方案。
Beyond Shannon Limits: Quantum Communications through Quantum Paths
本文研究了量子路径下可实现的量子容量并建立其上下界，证明了量子路径在超越传统Shannon理论极限的通信速率上相比经典路径具有实质性优势。
Shannon separation theorem lays the foundation for traditional image compression and transmission schemes, which consist of JPEG type image compression methods and the usual channel coding schemes such as Turbo and LDPC codes. One of the advantages of the separate design is that each of the two components, channel coding and source coding can be handled independently without considering the other, which is the base of decades-long technologies.
Developing computationally-efficient codes that approach the Shannon-theoretic limits for communication and compression has long been one of the major goals of information and coding theory. There have been significant advances towards this goal in the last couple of decades, with the emergence of turbo codes, sparse-graph codes, and polar codes. These codes are designed primarily for discrete-alphabet channels and sources. For Gaussian channels and sources, where the alphabet is inherently continuous, Sparse Superposition Codes or Sparse Regression Codes (SPARCs) are a promising class of code…
Semantic Communication: From Philosophical Conceptions Towards a Mathematical Framework
本文提出一种基于哲学概念和抽象层次的语义通信概率模型，将问题正交化为子问题，证明 Shannon 框架为其特例，并推导了物理噪声下语义容量的下界及可达速率。
Neural Coding Is Not Always Semantic: Toward the Standardized Coding Workflow in Semantic Communications
本文针对语义通信中神经编码是否足以理解语义的问题，提出语义编码的标准化定义和基于上下文建模的通用语义表示方案，以最小修改实现人机端到端传输。
Visible light communication (VLC) could provide short-range optical wireless communication together with illumination using LED lighting. However, conventional forward error correction (FEC) codes for reliable communication do not have the features for dimming support and flicker mitigation which are required in VLC for the main functionality of lighting. Therefore, auxiliary coding techniques are usually needed, which eventually reduce the coding efficiency and increase the complexity. In this paper, a polar codes-based FEC coding scheme for dimmable VLC is proposed to increase the coding eff…
In this paper, firstly, the Shannon channel capacity formula is briefly stated, and the relationship between the formula and the signal uncertainty principle is analyzed in order to prepare for deriving the formula which is able to break through the Shannon channel capacity. Then, as a practical example of breaking the Shannon limit, the time-shift non orthogonal multicarrier modulation technology is introduced. After more than twenty years of development, this technique is proved to be a practical modulation technique for digital communication.
An adaptive joint source-channel coding (JSCC) scheme is presented for transmitting correlated sources over discrete-memoryless two-way channels subject to distortion constraints. The proposed JSCC scheme makes use of the previously transmitted and received channel signals as well as the sources' correlation to facilitate coordination between terminals. It is shown that the adaptive scheme strictly subsumes prior lossy coding methods for two-way simultaneous transmission and yields a new adaptive separate source-channel coding result. Two examples are given to show the scheme's advantages.
We investigate binary block-codes (BBC). A BBC is defined as a vector of Boolean functions. We consider BBCs which are generated randomly, and using single-letter distributions. We characterize the vector of dependency spectrums of these BBCs. We use this vector to upper-bound the correlation between the outputs of two distributed BBCs. Finally, the upper-bound is used to show that the large blocklength single-letter coding schemes in the literature are sub-optimal in some multiterminal communication settings.
It is well known that orthogonal coding can be used to approach the Shannon capacity of the power-constrained AWGN channel without a bandwidth constraint. This correspondence describes a semi-orthogonal variation of pulse position modulation that is sequential in nature -- bits can be ``streamed across'' without having to buffer up blocks of bits at the transmitter. ML decoding results in an exponentially small probability of error as a function of tolerated receiver delay and thus eventually a zero probability of error on every transmitted bit. In the high-rate regime, a matching upper bound …
Computer-aided Characterization of Fundamental Limits of Coded Caching with Linear Coding
本文提出一种计算机辅助框架，用于在线性编码约束下表征缓存系统的根本极限，并证明了某些内存-负载权衡点的最优性。
A joint communication and channel state estimation problem is investigated, in which reliable information transmission over a noisy channel, and high-fidelity estimation of the channel state, are simultaneously sought. The tradeoff between the achievable information rate and the estimation distortion is quantified by formulating the problem as a constrained channel coding problem, and the resulting capacity-distortion function characterizes the fundamental limit of the joint communication and channel estimation problem. The analytical results are illustrated through case studies, and further i…
Molecular Arithmetic Coding (MoAC) and Optimized Molecular Prefix Coding (MoPC) for Diffusion-Based Molecular Communication
针对适配分子通信的Huffman编码非最优的问题，提出优化分子前缀编码MoPC、分子算术编码MoAC及兼具唯一可译性的MoAPC，经验证压缩与误码性能更优，为纳米尺度分子通信的信源压缩提供新方案。
With the phenomenal growth of the Internet of Things (IoT), Ultra Reliable Low Latency Communications (URLLC) has potentially been the enabler to guarantee the stringent requirements on latency and reliability. However, how to achieve low latency and ultra-reliability with the random arrival remains open. In this paper, a queue-aware variable-length channel coding is presented over the single URLLC user link, in which the finite blocklength of channel coding is determined based on the random arrival. More particularly, a cross-layer approach is proposed for the URLLC user to establish the opti…
A new channel coding approach was proposed in [1] for random multiple access communication over the discrete-time memoryless channel. The coding approach allows users to choose their communication rates independently without sharing the rate information among each other or with the receiver. The receiver will either decode the message or report a collision depending on whether reliable message recovery is possible. It was shown that, asymptotically as the codeword length goes to infinity, the set of communication rates supporting reliable message recovery can be characterized by an achievable …
Using ideas from Chu and Bode/Fano theories, we characterize the maximum achievable rate over the single-input single-output wireless communication channels under a restriction on the antenna size at the receiver. By employing circuit-theoretic multiport models for radio communication systems, we derive the information-theoretic limits of compact antennas. We first describe an equivalent Chu's antenna circuit under the physical realizability conditions of its reflection coefficient. Such a design allows us to subsequently compute the achievable rate for a given receive antenna size thereby pro…
Computation codes in network information theory are designed for the scenarios where the decoder is not interested in recovering the information sources themselves, but only a function thereof. Körner and Marton showed for distributed source coding that such function decoding can be achieved more efficiently than decoding the full information sources. Compute-and-forward has shown that function decoding, in combination with network coding ideas, is a useful building block for end-to-end communication. In both cases, good computation codes are the key component in the coding schemes. In this wo…
The goal of this paper is to promote the idea that including semantic and goal-oriented aspects in future 6G networks can produce a significant leap forward in terms of system effectiveness and sustainability. Semantic communication goes beyond the common Shannon paradigm of guaranteeing the correct reception of each single transmitted packet, irrespective of the meaning conveyed by the packet. The idea is that, whenever communication occurs to convey meaning or to accomplish a goal, what really matters is the impact that the correct reception/interpretation of a packet is going to have on the…
Network coding is an efficient means to improve the spectrum efficiency of satellite communications. However, its resilience to eavesdropping attacks is not well understood. This paper studies the confidentiality issue in a bidirectional satellite network consisting of two mobile users who want to exchange message via a multibeam satellite using the XOR network coding protocol. We aim to maximize the sum secrecy rate by designing the optimal beamforming vector along with optimizing the return and forward link time allocation. The problem is non-convex, and we find its optimal solution using se…
In this paper an approach to joint source-channel coding (JSCC) named Shannon-Kotel'nikov mappings (S-K mappings) is presented. S-K mappings are continuous, or piecewise continuous direct source-to-channel mappings operating directly on amplitude continuous and discrete time signals. Such mappings include several existing JSCC schemes as special cases. Many existing approaches to analog- or hybrid discrete analog JSCC provide both excellent performance as well as robustness to variations in noise level. This at low delay and relatively low complexity. However, a theory explaining their perform…
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。