Inference, Prediction, and Entropy-Rate Estimation of Continuous-time, Discrete-event Processes
W. M. Keck Science Department of Pitzer, Scripps, and Claremont McKenna College（皮泽学院、斯克里普斯学院和克莱蒙特·麦肯纳学院W.M.凯克科学系）；University of California at Davis（加州大学戴维斯分校）
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network's universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.
The information detection of complex systems from data is currently undergoing a revolution, driven by the emergence of big data and machine learning methodology. Discovering governing equations and quantifying dynamical properties of complex systems are among central challenges. In this work, we devise a nonparametric approach to learn the relative entropy rate from observations of stochastic differential equations with different drift functions.The estimator corresponding to the relative entropy rate then is presented via the Gaussian process kernel theory. Meanwhile, this approach enables t…
We consider the problem of the Bayesian inference of drift and diffusion coefficient functions in a stochastic differential equation given discrete observations of a realisation of its solution. We give conditions for the well-posedness and stable approximations of the posterior measure. These conditions in particular allow for priors with unbounded support. Our proof relies on the explicit construction of transition probability densities using the parametrix method for general parabolic equations. We then study an application of these results in inferring the rates of Birth-and-Death processe…
The existence and exact form of the continuum expression of the discrete nonlogarithmic $q$-entropy is an important open problem in generalized thermostatistics, since its possible lack implies that nonlogarithmic $q$-entropy is irrelevant for the continuous classical systems. In this work, we show how the discrete nonlogarithmic $q$-entropy in fact converges in the continuous limit and the negative of the $q$-entropy with continuous variables is demonstrated to lead to the (Csisz{á}r type) $q$-relative entropy just as the relation between the continuous Boltzmann-Gibbs expression and the Kull…
Several years ago, it has been discussed that non-logarithmic entropies such as the Tsallis q-entropy cannot be applied to systems with continuous variables. Now, in their recent paper [Phys. Rev. E 97, 012104 (2018)], Oikonomou and Bagci have modified the form of the q-entropy for discrete variables in such a way that its continuum limit exists. Here, it is shown that this modification violates the expandability property of entropy, and their work is actually a supporting evidence for the absence of the q-entropy for systems with continuous variables.
It has been known for some time that the usual $q$-entropy $S_q^{(n)}$ cannot be shown to converge to the continuous case. In [Phys. Rev. E 97 (2018) 012104], we have shown that the discrete $q$-entropy $\widetilde{S}_q^{(n)}$ converges to the continuous case when the total number of states are properly taken into account in terms of a convergence factor. Ou and Abe [Phys. Rev. E 97, (2018) 066101, arXiv:1801.03035] noted that this form of the discrete $q$-entropy does not conform to the Shannon-Khinchin expandability axiom. As a reply, we note that the fulfillment or not of the expandability …
Discrete time crystals are periodically driven systems that display spontaneous symmetry breaking of time translation invariance in the form of indefinite subharmonic oscillations. We introduce a thermodynamically consistent model for a discrete time crystal and analyze it using the framework of stochastic thermodynamics. In particular, we evaluate the rate of energy dissipation of this many body system of interacting noisy subharmonic oscillators in contact with a heat bath. Our model displays the novel phenomena of subharmonic synchronization, which corresponds to collective subharmonic osci…
In this letter we show that the Shore--Johnson axioms for Maximum Entropy Principle in statistical estimation theory account for a considerably wider class of entropic functional than previously thought. Apart from a formal side of the proof, we substantiate our point by analyzing the effect of weak correlations and discuss two pertinent examples: $2$-qubit quantum system and strongly interacting nuclear systems.
The nature of the different zero-temperature phases in discrete two-dimensional spin glasses: Entropy, universality, chaos and cascades in the renormalization group flow
Maximum Entropy is a powerful concept that entails a sharp separation between relevant and irrelevant variables. It is typically invoked in inference, once an assumption is made on what the relevant variables are, in order to estimate a model from data, that affords predictions on all other (dependent) variables. Conversely, maximum entropy can be invoked to retrieve the relevant variables (sufficient statistics) directly from the data, once a model is identified by Bayesian model selection. We explore this approach in the case of spin models with interactions of arbitrary order, and we discus…
