Kolmogorov complexity is the length of the ultimately compressed version of a file (that is, anything which can be put in a computer). Formally, it is the length of a shortest program from which the file can be reconstructed. We discuss the incomputabilty of Kolmogorov complexity, which formal loopholes this leaves us, recent approaches to compute or approximate Kolmogorov complexity, which approaches are problematic and which approaches are viable.
Diverse applications of Kolmogorov complexity to learning [CIKK16], circuit complexity [OPS19], cryptography [LP20], average-case complexity [Hir21], and proof search [Kra22] have been discovered in recent years. Since the running time of algorithms is a key resource in these fields, it is crucial in the corresponding arguments to consider time-bounded variants of Kolmogorov complexity. While fruitful interactions between time-bounded Kolmogorov complexity and different areas of theoretical computer science have been known for quite a while (e.g., [Sip83, Ko91, ABK+06, AF09], to name a few), t…
The paper proposes open problems in classical Kolmogorov complexity. Each problem is presented with background information and thus the article also surveys some recent studies in the area.
Romashchenko and Zimand~\cite{rom-zim:c:mutualinfo} have shown that if we partition the set of pairs $(x,y)$ of $n$-bit strings into combinatorial rectangles, then $I(x:y) \geq I(x:y \mid t(x,y)) - O(\log n)$, where $I$ denotes mutual information in the Kolmogorov complexity sense, and $t(x,y)$ is the rectangle containing $(x,y)$. We observe that this inequality can be extended to coverings with rectangles which may overlap. The new inequality essentially states that in case of a covering with combinatorial rectangles, $I(x:y) \geq I(x:y \mid t(x,y)) - \log ρ- O(\log n)$, where $t(x,y)$ is any…
It was proved by Brudno that entropy and Kolmogorov complexity for dynamical systems are tightly related. We generalize his results to the case of arbitrary computable amenable group actions. Namely, for an ergodic shift-action, the asymptotic Kolmogorov complexity of a typical point is equal to the Kolmogorov-Sinai entropy of the action. For topological shift actions, the asymptotic Komogorov complexity of every point is bounded from above by the topological entropy, and there is a point attaining this bound.
Northeastern University（东北大学）；GE Global Research（通用电气全球研究中心）
Recent developments have linked causal inference with Algorithmic Information Theory, and methods have been developed that utilize Conditional Kolmogorov Complexity to determine causation between two random variables. We present a method for inferring causal direction between continuous variables by using an MDL Binning technique for data discretization and complexity calculation. Our method captures the shape of the data and uses it to determine which variable has more information about the other. Its high predictive performance and robustness is shown on several real world use cases.
There is a parallelism between Shannon information theory and algorithmic information theory. In particular, the same linear inequalities are true for Shannon entropies of tuples of random variables and Kolmogorov complexities of tuples of strings (Hammer et al., 1997), as well as for sizes of subgroups and projections of sets (Chan, Yeung, Romashchenko, Shen, Vereshchagin, 1998--2002). This parallelism started with the Kolmogorov-Levin formula (1968) for the complexity of pairs of strings with logarithmic precision. Longpré (1986) proved a version of this formula for space-bounded complexitie…
Consider a binary string $x$ of length $n$ whose Kolmogorov complexity is $αn$ for some $α<1$. We want to increase the complexity of $x$ by changing a small fraction of bits in $x$. This is always possible: Buhrman, Fortnow, Newman and Vereshchagin (2005) showed that the increase can be at least $δn$ for large $n$ (where $δ$ is some positive number that depends on $α$ and the allowed fraction of changed bits). We consider a related question: what happens with the complexity of $x$ when we randomly change a small fraction of the bits (changing each bit independently with some probability $τ$)? …
It is well known that normality can be described as incompressibility via finite automata. Still the statement and the proof of this result as given by Becher and Heiber (2013) in terms of "lossless finite-state compressors" do not follow the standard scheme of Kolmogorov complexity definition (an automaton is used for compression, not decompression). We modify this approach to make it more similar to the traditional Kolmogorov complexity theory (and simpler) by explicitly defining the notion of automatic Kolmogorov complexity and using its simple properties. Using this characterization and a …
These reminiscences are about the "dark ages" of algorithmic information theory in the USSR. After a great interest in this topic in 1960s and the beginning of 1970s the number of people working in this area in the USSR decreased significantly. At that time L.A. Levin published a bunch of papers that were seminal for the modern algorithmic information theory. Then he left the USSR, and the new wave of interest was triggered by the talk of A.N. Kolmogorov at a Moscow State (Lomonosov) University Mathematical Department (Logic and Algorithms Division) seminar organized by him; several younger re…
We consider the network consisting of three nodes $1, 2, 3$ connected by two open channels $1\rightarrow 2$ and $1\rightarrow 3$. The information present in the node 1 consists of four strings $x,y,z,w$. The nodes $2, 3$ know $x,w$ and need to know $y,z$, respectively. We want to arrange transmission of information over the channels so that both nodes $2$ and $3$ learn what they need and the disclosure of information is as small as possible. By information disclosure we mean the amount of information in the strings transmitted through channels about $x,y,z,w$ (or about $x,w$). We are also inte…
