In [She82], it is shown that four basic functional properties are enough to characterize plain Kolmogorov complexity, hence obtaining an axiomatic characterization of this notion. In this paper, we try to extend this work, both by looking at alternative axiomatic systems for plain complexity and by considering potential axiomatic systems for other types of complexity. First we show that the axiomatic system given by Shen cannot be weakened (at least in any natural way). We then give an analogue of Shen's axiomatic system for conditional complexity. In a the second part of the paper, we look at prefix-free complexity and try to construct an axiomatic system for it. We show however that the natural analogues of Shen's axiomatic systems fails to characterize prefix-free complexity.
Kolmogorov complexity is the length of the ultimately compressed version of a file (that is, anything which can be put in a computer). Formally, it is the length of a shortest program from which the file can be reconstructed. We discuss the incomputabilty of Kolmogorov complexity, which formal loopholes this leaves us, recent approaches to compute or approximate Kolmogorov complexity, which approaches are problematic and which approaches are viable.
Diverse applications of Kolmogorov complexity to learning [CIKK16], circuit complexity [OPS19], cryptography [LP20], average-case complexity [Hir21], and proof search [Kra22] have been discovered in recent years. Since the running time of algorithms is a key resource in these fields, it is crucial in the corresponding arguments to consider time-bounded variants of Kolmogorov complexity. While fruitful interactions between time-bounded Kolmogorov complexity and different areas of theoretical computer science have been known for quite a while (e.g., [Sip83, Ko91, ABK+06, AF09], to name a few), t…
The paper proposes open problems in classical Kolmogorov complexity. Each problem is presented with background information and thus the article also surveys some recent studies in the area.
It was proved by Brudno that entropy and Kolmogorov complexity for dynamical systems are tightly related. We generalize his results to the case of arbitrary computable amenable group actions. Namely, for an ergodic shift-action, the asymptotic Kolmogorov complexity of a typical point is equal to the Kolmogorov-Sinai entropy of the action. For topological shift actions, the asymptotic Komogorov complexity of every point is bounded from above by the topological entropy, and there is a point attaining this bound.
There is a parallelism between Shannon information theory and algorithmic information theory. In particular, the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of strings (Hammer et al., 1997), as well as for sizes of subgroups and projections of sets (Chan, Yeung, Romashchenko, Shen, Vereshchagin, 1998--2002). This parallelism started with the Kolmogorov-Levin formula (1968) for the complexity of pairs of strings with logarithmic precision. Longpré (1986) proved a version of this formula for space-bounded complexitie…
Romashchenko and Zimand~\cite{rom-zim:c:mutualinfo} have shown that if we partition the set of pairs $(x,y)$ of $n$-bit strings into combinatorial rectangles, then $I(x:y) \geq I(x:y \mid t(x,y)) - O(\log n)$, where $I$ denotes mutual information in the Kolmogorov complexity sense, and $t(x,y)$ is the rectangle containing $(x,y)$. We observe that this inequality can be extended to coverings with rectangles which may overlap. The new inequality essentially states that in case of a covering with combinatorial rectangles, $I(x:y) \geq I(x:y \mid t(x,y)) - \log ρ- O(\log n)$, where $t(x,y)$ is any…
Northeastern University（东北大学）；GE Global Research（通用电气全球研究中心）
Recent developments have linked causal inference with Algorithmic Information Theory, and methods have been developed that utilize Conditional Kolmogorov Complexity to determine causation between two random variables. We present a method for inferring causal direction between continuous variables by using an MDL Binning technique for data discretization and complexity calculation. Our method captures the shape of the data and uses it to determine which variable has more information about the other. Its high predictive performance and robustness is shown on several real world use cases.
It is well known that normality can be described as incompressibility via finite automata. Still the statement and the proof of this result as given by Becher and Heiber (2013) in terms of "lossless finite-state compressors" do not follow the standard scheme of Kolmogorov complexity definition (an automaton is used for compression, not decompression). We modify this approach to make it more similar to the traditional Kolmogorov complexity theory (and simpler) by explicitly defining the notion of automatic Kolmogorov complexity and using its simple properties. Using this characterization and a …
These reminiscences are about the "dark ages" of algorithmic information theory in the USSR. After a great interest in this topic in 1960s and the beginning of 1970s the number of people working in this area in the USSR decreased significantly. At that time L.A. Levin published a bunch of papers that were seminal for the modern algorithmic information theory. Then he left the USSR, and the new wave of interest was triggered by the talk of A.N. Kolmogorov at a Moscow State (Lomonosov) University Mathematical Department (Logic and Algorithms Division) seminar organized by him; several younger re…
Consider a binary string $x$ of length $n$ whose Kolmogorov complexity is $αn$ for some $α<1$. We want to increase the complexity of $x$ by changing a small fraction of bits in $x$. This is always possible: Buhrman, Fortnow, Newman and Vereshchagin (2005) showed that the increase can be at least $δn$ for large $n$ (where $δ$ is some positive number that depends on $α$ and the allowed fraction of changed bits). We consider a related question: what happens with the complexity of $x$ when we randomly change a small fraction of the bits (changing each bit independently with some probability $τ$)? …
The Gaussian width is a fundamental quantity in probability, statistics and geometry, known to underlie the intrinsic difficulty of estimation and hypothesis testing. In this work, we show how the Gaussian width, when localized to any given point of an ellipse, can be controlled by the Kolmogorov width of a set similarly localized. This connection leads to an explicit characterization of the estimation error of least-squares regression as a function of the true regression vector within the ellipse. The rate of error decay varies substantially as a function of location: as a concrete example, i…
We consider the network consisting of three nodes $1, 2, 3$ connected by two open channels $1\rightarrow 2$ and $1\rightarrow 3$. The information present in the node 1 consists of four strings $x,y,z,w$. The nodes $2, 3$ know $x,w$ and need to know $y,z$, respectively. We want to arrange transmission of information over the channels so that both nodes $2$ and $3$ learn what they need and the disclosure of information is as small as possible. By information disclosure we mean the amount of information in the strings transmitted through channels about $x,y,z,w$ (or about $x,w$). We are also inte…
