We have discussed dynamical properties of the Tsallis entropy and the generalized Fisher information in nonextensive systems described by the Langevin model subjected to additive and multiplicative noise. Analytical expressions for the time-dependent Tsallis entropy and generalized Fisher information have been obtained with the use of the $q$-moment approach to the Fokker-Planck equation developed in a previous study [H. Hasegawa, Phys. Rev. E {\bf 77}, 031133 (2008)]. Model calculations of the information entropies in response to an applied pulse and sinusoidal inputs have been presented.
The form invariance of pseudoadditivity is shown to determine the structure of nonextensive entropies. Nonextensive entropy is defined as the appropriate expectation value of nonextensive information content, similar to the definition of Shannon entropy. Information content in a nonextensive system is obtained uniquely from generalized axioms by replacing the usual additivity with pseudoadditivity. The satisfaction of the form invariance of the pseudoadditivity of nonextensive entropy and its information content is found to require the normalization of nonextensive entropies. The proposed prin…
We discuss the generalized von Neumann (Tsallis) entropy and the generalized Fisher information (GFI) in nonextensive quantum systems, by using the interpolation approximation (IA) which has been shown to yield good results for the quantal distributions within $O(q-1)$ and in high- and low-temperature limits, $q$ being the entropic index [H. Hasegawa, Phys. Rev. E 80 (2009) 011126]. Three types of GFIs which have been proposed so far in the nonextensive statistics, are discussed from the viewpoint of their metric properties and the Cramér-Rao theorem. Numerical calculations of the $q$- and tem…
The form invariance of the statement of the maximum entropy principle and the metric structure in quantum density matrix theory, when generalized to nonextensive situations, is shown here to determine the structure of the nonextensive entropies. This limits the range of the nonextensivity parameter to so as to preserve the concavity of the entropies. The Tsallis entropy is thereby found to be appropriately renormalized.
In nonextensive statistical mechanics, two kinds of definitions have been considered for expectation valu of a physical quantity: one is the ordinary definition and the other is the normalized q-expectation value employing the escort distribution. Since both of them lead to the maximum-Tsallis-entropy distributions of a similar type, it is of crucial importance to determine which the correct physical one is. A point is that the definition of expectation value is indivisibly connected to the form of generalized relative entropy. Studying the properties of the relative entropies associated with …
The Tsallis entropy and Fisher information entropy (matrix) are very important quantities expressing information measures in nonextensive systems. Stationary and dynamical properties of the information entropies have been investigated in the $N$-unit coupled Langevin model subjected to additive and multiplicative white noise, which is one of typical nonextensive systems. We have made detailed, analytical and numerical study on the dependence of the stationary-state entropies on additive and multiplicative noise, external inputs, couplings and number of constitutive elements ($N$). By solving t…
In this paper, we show that 1) additive energy is not appropriate for discussing the validity of Tsallis or Rényi statistics for nonextensive systems at meta-equilibrium; 2) $N$-body systems with nonadditive energy or entropy should be described by generalized statistics whose nature is prescribed by the existence of thermodynamic stationarity. 3) the equivalence of Tsallis and Rényi entropies is in general not true.
We have calculated the Tsallis entropy and Fisher information matrix (entropy) of spatially-correlated nonextensive systems, by using an analytic non-Gaussian distribution obtained by the maximum entropy method. Effects of the correlated variability on the Fisher information matrix are shown to be different from those on the Tsallis entropy. The Fisher information is increased (decreased) by a positive (negative) correlation, whereas the Tsallis entropy is decreased with increasing an absolute magnitude of the correlation independently of its sign. This fact arises from the difference in their…
Maximum entropy principles in nonextensive statistical physics are revisited as an application of the Tsallis relative entropy defined for non-negative matrices in the framework of matrix analysis. In addtition, some matrix trace inequalities related to the Tsallis relative entropy are studied.
We propose a new way of defining entropy of a system, which gives a general form which may be nonextensive as Tsallis entropy, but is linearly dependent on component entropies, like Renyi entropy, which is extensive. This entropy has a conceptually novel but simple origin and is mathematically easy to define by a very simple expression, though the probability distribution resulting from optimizing it gives rather complex, which is compared numerically with the other entropies. It may, therefore, appear as the right candidate in a physical situation where the probability distribution does not s…
We show that the non-additivity relation of the Tsallis entropies in nonextensive statistical mechanics has a simple physical interpretation for systems with fluctuating temperature or fluctuating energy dissipation rate. We also show that there is a distinguished dependence of the entropic index q on the spatial scale that makes the Tsallis entropies quasi-additive. Quasi-additivity implies that q is a strictly monotonously decreasing function of the spatial scale r, as indeed observed in various experiments.
The joint eigenvalue distributions of random-matrix ensembles are derived by applying the principle maximum entropy to the Renyi, Abe and Kaniadakis entropies. While the Renyi entropy produces essentially the same matrix-element distributions as the previously obtained expression by using the Tsallis entropy, and the Abe entropy does not lead to a closed form expression, the Kaniadakis entropy leads to a new generalized form of the Wigner surmise that describes a transition of the spacing distribution from chaos to order. This expression is compared with the corresponding expression obtained b…
We study the similarities and differences between different models concerning subdiffusion. More particularly, we calculate first passage time (FPT) distributions for subdiffusion, derived from Greens' functions of nonlinear equations obtained from Sharma-Mittal's, Tsallis's and Gauss's nonadditive entropies. Then we compare these with FPT distributions calculated from a fractional model using a subdiffusion equation with a fractional time derivative. All of Greens' functions give us exactly the same standard relation $<(Δx)^2> =2 D_αt^α$ which characterizes subdiffusion ($0<α<1$), but general…
We introduce a nonextensive entropic measure $S_χ$ that grows like $N^χ$, where $N$ is the size of the system under consideration. This kind of nonextensivity arises in a natural way in some $N$-body systems endowed with long-range interactions described by $r^{-α}$ interparticle potentials. The power law (weakly nonextensive) behavior exhibited by $S_χ$ is intermediate between (1) the linear (extensive) regime characterizing the standard Boltzmann-Gibbs entropy and the (2) the exponential law (strongly nonextensive) behavior associated with the Tsallis generalized $q$-entropies. The functiona…
The present paper studies continuity of generalized entropy functions and relative entropies defined using the notion of a deformed logarithmic function. In particular, two distinct definitions of relative entropy are discussed. As an application, all considered entropies are shown to satisfy Lesche's stability condition. The entropies of Tsallis' nonextensive thermostatistics are taken as examples.
We present two extended forms of Fisher information that fit well in the context of nonextensive thermostatistics. We show that there exists an interplay between these generalized Fisher information, the generalized $q$-Gaussian distributions and the $q$-entropies. The minimum of the generalized Fisher information among distributions with a fixed moment, or with a fixed $q$-entropy is attained, in both cases, by a generalized $q$-Gaussian distribution. This complements the fact that the $q$-Gaussians maximize the $q$-entropies subject to a moment constraint, and yields new variational characte…
The cornerstones of Boltzmann-Gibbs and nonextensive statistical mechanics respectively are the entropies $S_{BG} \equiv -k \sum_{i=1}^W p_i \ln p_i $ and $S_{q}\equiv k (1-\sum_{i=1}^Wp_i^{q})/(q-1) (q\in{\mathbb R} ; S_1=S_{BG})$. Through them we revisit the concept of additivity, and illustrate the (not always clearly perceived) fact that (thermodynamical) extensivity has a well defined sense {\it only} if we specify the composition law that is being assumed for the subsystems (say $A$ and $B$). If the composition law is {\it not} explicitly indicated, it is {\it tacitly} assumed that $A$ a…
We comment on some open questions and theoretical peculiarities in Tsallis nonextensive statistical mechanics. It is shown that the theoretical basis of the successful Tsallis' generalized exponential distribution shows some worrying properties with the conventional normalization and the escort probability. These theoretical difficulties may be avoided by introducing an so called incomplete normalization allowing to deduce Tsallis' generalized distribution in a more convincing and consistent way.
Gibbs-Boltzmann entropy leads to systems that have a strong dependence on initial conditions. In reality, most materials behave quite independently of initial conditions. Nonextensive entropy or Tsallis entropy leads to nonextensive statistical mechanics. In this paper, we calculate the Tsallis form of Clausius inequality and then determine the upper bound for extracting work from the small system in the nonextensive statistical mechanics with mutual information. In the following, we extract mutual information and adjust Maxwell's demon with quantum feedback control.
This is a study of the information evolution of complex systems by geometrical consideration. We look at chaotic systems evolving in fractal phase space. The entropy change in time due to the fractal geometry is assimilated to the information growth through the scale refinement. Due to the incompleteness of the state number counting at any scale on fractal support, the incomplete normalization $\sum_ip_i^q=1$ is applied throughout the paper, where $q$ is the fractal dimension divided by the dimension of the smooth Euclidean space in which the fractal structure of the phase space is embedded. I…
In this paper we extend our recent results [Physica A340 (2004)110] on q-nonextensive statistics with non-Tsallis entropies. In particular, we combine an axiomatics of Renyi with the q-deformed version of Khinchin axioms to obtain the entropy which accounts both for systems with embedded self-similarity and q-nonextensivity. We find that this entropy can be uniquely solved in terms of a one-parameter family of information measures. The corresponding entropy maximizer is expressible via a special function known under the name of the Lambert W-function. We analyze the corresponding "high" and "l…
We revisit some topics of classical thermostatistics from the perspective of the nonextensive optimal Lagrange multipliers (OLM), a recently introduced technique for dealing with the maximization of Tsallis' information measure. It is shown that Equipartition and Virial theorems can be reproduced by Tsallis' nonextensive formalism independently of the value of the nonextensivity index.
Statistical mechanics is generalized on the basis of an information theory for inexact or incomplete probability distributions. A parameterized normalization is proposed and leads to a nonextensive entropy. The resulting incomplete statistical mechanics is proved to have the same theoretical characteristics as Tsallis one based on the conventional normalization.
For a random variable we can define a variational relationship with practical physical meaning as dI=dbar(x)-bar(dx), where I is called as uncertainty measurement. With the help of a generalized definition of expectation, bar(x)=sum_(i)g(p_i)x_i, and the expression of dI, we can find the concrete forms of the maximizable entropies for any given probability distribution function, where g(p_i) may have different forms for different statistics which includes the extensive and nonextensive statistics.
We perform a throughout numerical study of the average sensitivity to initial conditions and entropy production for two symplectically coupled standard maps focusing on the control-parameter region close to regularity. Although the system is ultimately strongly chaotic (positive Lyapunov exponents), it first stays lengthily in weak-chaotic regions (zero Lyapunov exponents). We argue that the nonextensive generalization of the classical formalism is an adequate tool in order to get nontrivial information about this complex phenomenon. Within this context we analyze the relation between the powe…
We consider the problem of guessing the realization of a random variable but under more general Tsallis' non-extensive entropic framework rather than the classical Maxwell-Boltzman-Gibbs-Shannon framework. We consider both the conditional guessing problem in the presence of some related side information, and the unconditional one where no such side-information is available. For both types of the problem, the non-extensive moment bounds of the required number of guesses are derived; here we use the $q$-normalized expectation in place of the usual (linear) expectation to define the non-extensive…
The original canonical ensemble formalism for the nonextensive entropy thermostatistics is reconsidered. It is shown that the unambiguous connection of the statistical mechanics with the equilibrium thermodynamics is provided if the entropic parameter $1/(q-1)$ is an extensive variable of the state. Based on a particular example of the perfect gas, it is proved that the Tsallis thermostatistics meets all the requirements of equilibrium thermodynamics in the thermodynamic limit. In particular, the entropy of the system is extensive and the temperature is intensive. However, for finite systems b…
A chain rule and a subadditivity for the entropy of type $β$, which is one of the nonadditive entropies, were derived by Z.Daróczy. In this paper, we study the further relations among Tsallis type entropies which are typical nonadditive entropies. The chain rule is generalized by showing it for Tsallis relative entropy and the nonadditive entropy. We show some inequalities related to Tsallis entropies, especially the strong subadditivity for Tsallis type entropies and the subadditivity for the nonadditive entropies. The subadditivity and the strong subadditivity naturally lead to define Tsalli…
The Shannon-Khinchin axioms for the ordinary information entropy are generalized in a natural way to the nonextensive systems based on the concept of nonextensive conditional entropy, and a complete proof of the uniqueness theorem for the Tsallis entropy is presented.
The nonextensive statistics based on the $q$-entropy $S_q=-\frac{\sum_{i=1}^v(p_i-p_i^q)}{1-q}$ has been so far applied to systems in which the $q$ value is uniformly distributed. For the systems containing different $q$'s, the applicability of the theory is still a matter of investigation. The difficulty is that the class of systems to which the theory can be applied is actually limited by the usual nonadditivity rule of entropy which is no more valid when the systems contain non uniform distribution of $q$ values. In this paper, within the framework of the so called incomplete information th…
The application of information theory approach (both in its extensive and nonextensive versions) to high energy multiparticle processes is discussed and confronted with experimental data on e+e- annihilation processes, pp and \bar{p}p scatterings and heavy ion collisions.
We introduce a class of information measures based on group entropies, allowing us to describe the information-theoretical properties of complex systems. These entropic measures are nonadditive, and are mathematically deduced from a series of natural requirements. In particular, we introduce an extensivity postulate as a natural requirement for an information measure to be meaningful. The information measures proposed are suitably defined for describing universality classes of complex systems, each characterized by a specific phase space growth rate function.
We present an overview of information theory approach (both in its extensive and nonextensive versions) applied to high energy multiparticle production processes. It will be illustrated by analysis of single particle distributions measured in proton-proton, proton-antiproton and nuclear collisions. We shall demonstrate the particular role played by the nonextensivity parameter q in such analysis as summarizing our knowledge on the fluctuations existed in hadronizing system.
Two information entropies, $S_{IB}$ based on the local bonding energy configurations and $S_{IS}$ based on the local spin configurations, are applied in the square-lattice $\pm {J}$ Ising systems with Monte Carlo simulations. With $S_{IB}$ and $S_{IS}$, the spin glass states can be distinguished from paramagnetic states clearly. The results reveal that the nature of spin glass states is of ordering in bonding energy and disordering in spin orientations. The consistence of $S_{IB}$ and the thermodynamic entropy is found.
Some general considerations on the notion of entropy in physics are presented. An attempt is made to clarify the question of the differentiation between physical entropy (the Clausius-Boltzmann one) and quantities called entropies associated to Information Theory, which are in fact generating functionals for the derivation of probability distributions and not thermodynamic functions of state. The role of them in the construction of the so-called Unconventional Statistical Mechanics, and the meaning and use of the latter, is discussed. Particular attention is given to the situation involving fa…
We investigate the limiting cases of Tsallis statistics. The viewpoint adopted is not the standard information-theoretic one, where one derives the distribution from a given measure of information. Instead the mechanical approach recently proposed in [M. Campisi, G.B. Bagci, Phys. Lett. A (2006), doi:10.1016/j.physleta.2006.09.081], is adopted, where the distribution is given and one looks for the associated physical entropy. We show that, not only the canonical ensemble is recovered in the limit of $q$ tending to one, as one expects, but also the microcanonical ensemble is recovered in the li…
By using the $q$-Gaussian distribution derived by the maximum entropy method for spatially-correlated $N$-unit nonextensive systems, we have calculated the generalized Fisher information matrix of $g_{θ_n θ_m}$ for $(θ_1, θ_2, θ_3) = (μ_q, σ_q^2$, $s$), where $μ_q$, $σ_q^2$ and $s$ denote the mean, variance and degree of spatial correlation, respectively, for a given entropic index $q$. It has been shown from the Cramér-Rao theorem that (1) an accuracy of an unbiased estimate of $μ_q$ is improved (degraded) by a negative (positive) correlation $s$, (2) that of $σ_q^2$ is worsen with increasing…
PAE cannot be made a basis for either a generalized statistical mechanics or a generalized information theory. Either statistical independence must be waived, or the expression of the averaged conditional probability as the difference between the marginal and joint entropies must be relinquished. The same inequality, relating the PAE to the Rényi entropy, when applied to the mean code length produces an expression that it is without bound as the order of the code length approaches infinity. Since the mean code length associated with the Rényi entropy is finite and can be made to come as close …
Systems with a long-term stationary state that possess as a spatio-temporally fluctuation quantity $β$ can be described by a superposition of several statistics, a "super statistics". We consider first, the Gamma, log-normal and $F$-distributions of $β$. It is assumed that they depend only on $p_l$, the probability associated with the microscopic configuration of the system. For each of the three $β-$distributions we calculate the Boltzmann factors and show that they coincide for small variance of the fluctuations. For the Gamma distribution it is possible to calculate the entropy in a closed …
We show that thermodynamic uncertainties (TU) it preserve their form in passing from Boltzmann-Gibbs' statistics to Tsallis' one provided that we express these TU in terms of the appropriate variable conjugate to the temperature in a nonextensive context.
The property that power means are monotonically increasing functions of their order is shown to be the basis of the second laws not only for processes involving heat conduction but also for processes involving deformations. In an $L$-potentail equilibration the final state will be one of maximum entropy, while in an entropy equilibrium the final state will be one of minimum $L$. A metric space is connected with the power means, and the distance between means of different order is related to the Carnot efficiency. In the ideal classical gas limit, the average change in the entropy is shown to b…
We show that using the most parsimonious version of Frieden and Soffer's extreme information principle (EPI) with a Fisher measure constructed with escort probabilities, the concomitant solutions obey a type of Naudts' duality for nonextensive ensembles. We also determine in closed form the general (normalized) probability distribution that minimizes Fisher's information.
We geometrically study the Legendre duality relation that plays an important role in statistical physics with the standard or generalized entropies. For this purpose, we introduce dualistic structure defined by information geometry, and discuss concepts arising in generalized thermostatistics, such as relative entropies, escort distributions and modified expectations. Further, a possible generalization of these concepts in a certain direction is also considered. Finally, as an application of such a geometric viewpoint, we briefly demonstrate several new results on a behavior of the solution to…
The entropy of Boltzmann-Gibbs, as proved by Shannon and Khinchin, is based on four axioms, where the fourth one concerns additivity. The group theoretic entropies make use of formal group theory to replace this axiom with a more general composability axiom. As has been pointed out before, generalised entropies crucially depend on the number of allowed number degrees of freedom $N$. The functional form of group entropies is restricted (though not uniquely determined) by assuming extensivity on the equal probability ensemble, which leads to classes of functionals corresponding to sub-exponentia…
In the present paper, we would like to draw attention to a possible generalized Fisher information that fits well in the formalism of nonextensive thermostatistics. This generalized Fisher information is defined for densities on $\mathbb{R}^{n}.$ Just as the maximum Rényi or Tsallis entropy subject to an elliptic moment constraint is a generalized q-Gaussian, we show that the minimization of the generalized Fisher information also leads a generalized q-Gaussian. This yields a generalized Cramér-Rao inequality. In addition, we show that the generalized Fisher information naturally pops up in a …
In this manuscript we present a comparative study about the determination of the relaxation (\textit{i.e.}, independence) time scales obtained from the correlation function, the mutual information, and a criterion based on the evaluation of a nonextensive generalisation of mutual entropy. Our results show that, for systems with a small degree of complexity, standard mutual information and the criterion based on its nonextensive generalisation provide the same scale, whereas for systems with a higher complex dynamics the standard mutual information presents a time scale consistently smaller.
Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [Lieb, Yngvason, Proc. Roy. Soc.(2013)]. In particular, we obtain a direct relation between the Clausius entropy and the Shannon entropy, or its generalisation to quantum systems, the von Neumann entropy. More generally, we find that …
As an application of generalised statistical mechanics, it is studied a possible route toward a consistent generalised information theory in terms of a family of non-extensive, non-parametric entropies $H^\pm_D(P)$. Unlike other proposals based on non-extensive entropies with a parameter dependence, our scheme is asymptotically equivalent to the one formulated by Shannon, while it differs in regions where the density of states is reasonably small, which leads to information distributions constrained to their background. Two basic concepts are discussed to this aim. First, we prove two effectiv…
Correlations reduce the configurational entropies of liquids below their ideal gas limits. By means of first principles molecular dynamics simulations, we obtain accurate pair correlation functions of liquid metals, then subtract the mutual information content of these correlations from the ideal gas entropies to predict the absolute entropies over a broad range of temperatures. We apply this method to liquid aluminum and copper and demonstrate good agreement with experimental measurements, then we apply it to predict the entropy of a liquid aluminum-copper alloy. Corrections due to electronic…
We define the class of multivariate group entropies as a novel set of information - theoretical measures, which extends significantly the family of group entropies. We propose new examples related to the "super-exponential" universality class of complex systems; in particular, we introduce a general entropy, representing a suitable information measure for this class. We also show that the group-theoretical structure associated with our multivariate entropies can be used to define a large family of exactly solvable discrete dynamical models. The natural mathematical framework allowing us to for…
We implement a Wang-Landau sampling technique in quantum Monte Carlo (QMC) for the purpose of calculating the Renyi entanglement entropies and associated mutual information. The algorithm converges an estimate for an analogue to the density of states for Stochastic Series Expansion QMC allowing a direct calculation of Renyi entropies without explicit thermodynamic integration. We benchmark results for the mutual information on two-dimensional (2D) isotropic and anisotropic Heisenberg models, 2D transverse field Ising model, and 3D Heisenberg model, confirming a critical scaling of the mutual i…
Complex systems that are characterized by strong correlations and fat-tailed distribution functions have been argued to be incompatible within the framework of Boltzmann-Gibbs entropy. As an alternative, so-called generalized entropies were proposed and intensively studied. Here we show that this incompatibility is a misconception. For a broad class of processes, Boltzmann entropy the log multiplicity remains the valid entropy concept, however, for non-i.i.d., non-multinomial, and non-ergodic processes, Boltzmann entropy is not of Shannon form. The correct form of Boltzmann entropy can be show…
Relative entropy is a measure of distinguishability for quantum states, and plays a central role in quantum information theory. The family of Renyi entropies generalizes to Renyi relative entropies that include as special cases most entropy measures used in quantum information theory. We construct a Euclidean path-integral approach to Renyi relative entropies in conformal field theory, then compute the fidelity and the relative entropy of states in one spatial dimension at zero and finite temperature using a replica trick. In contrast to the entanglement entropy, the relative entropy is free o…
In a recent paper Hawking has argued that there is no information loss in black holes in asymptotically AdS spacetimes. We remind that there are several types of information (entropy) in statistical physics -- fine grained (microscopic) and coarse grained (macroscopic) ones which behave differently under unitary evolution. We suggest that the coarse grained information of the rest of the Universe is lost while fine grained information is preserved. A possibility to develop in quantum gravity an analogue of the Bogoliubov derivation of the irreversible Boltzmann and Navier - Stokes equations fr…
The LMC complexity, an indicator of complexity based on a probabilistic description, is revisited. A straightforward approach allows us to establish the time evolution of this indicator in a near-equilibrium situation and gives us a new insight for interpreting the LMC complexity for a general non equilibrium system. Its relationship with the Renyi entropies is also explained. One of the advantages of this indicator is that its calculation does not require a considerable computational effort in many cases of physical and biological interest.
Rényi and Tsallis information entropies for a harmonic position-dependent mass
University of Education, Winneba（温纳巴教育学院）；Ecole Polytechnique d’Abomey Calavi (EPAC-UAC)（阿博美-卡拉维高等理工学院）；Université d’Abomey-Calavi (UAC)（阿博美-卡拉维大学）；Institut de Mathématiques et de Sciences Physiques (IMSP)（数学与物理科学研究所）；University of Ghana（加纳大学）；African Institute for Mathematical Sciences (AIMS)（非洲数学科学研究所）
We propose a unifying picture where the notion of generalized entropy is related to information theory by means of a group-theoretical approach. The group structure comes from the requirement that an entropy be well defined with respect to the composition of independent systems, in the context of a recently proposed generalization of the Shannon-Khinchin axioms. We associate to each member of a large class of entropies a generalized information measure, satisfying the additivity property on a set of independent systems as a consequence of the underlying group law. At the same time, we also sho…
Quantum entanglement of pure states of a bipartite system is defined as the amount of local or marginal ({\em i.e.}referring to the subsystems) entropy. For mixed states this identification vanishes, since the global loss of information about the state makes it impossible to distinguish between quantum and classical correlations. Here we show how the joint knowledge of the global and marginal degrees of information of a quantum state, quantified by the purities or in general by information entropies, provides an accurate characterization of its entanglement. In particular, for Gaussian states …
