Using an approximation by a set-valued dynamical system, this paper studies a class of non Markovian and non homogeneous stochastic processes on a finite state space. It provides an unified approach to simulated annealing type processes. It permits to study new models of vertex reinforced random walks and new models of learning in games including Markovian fictitious play.
In this paper, we consider the linearly reinforced and the once-reinforced random walk models in the transient phase on trees. We show the large deviations for the upper tails for both models. We also show the exponential decay for the lower tail in the once-reinforced random walk model. On the other hand, the lower tail is in polynomial decay for the linearly reinforced random walk model.
This work introduces the notion of edge oriented reinforced random walk which proposes in a general framework an alternative understanding of the annealed law of random walks in random environment.
We study once-reinforced random walk (ORRW) on $\mathbb Z$. For this model, we derive limit results on all moments of its range using Tauberian theory.
In this short paper, we consider the Once-reinforced random walk with reinforcement parameter $a$ on trees with bounded degree which are transient for the simple random walk. On each of these trees, we prove that there exists an explicit critical parameter $a_0$ such that the Once-reinforced random walk is almost surely recurrent if $a>a_0$ and almost surely transient if $a<a_0$. This provides the first examples of phase transition for the Once-reinforced random walk.
We consider Reinforced Random Walks where transition probabilities are a function of the proportion of times the walk has traversed an edge. We give conditions for recurrence or transience. A phase transition is observed, similar to Pemantle \cite{Pem000} on trees.
We prove that any vertex-reinforced random walk on the integer lattice with non-decreasing reinforcement sequence $w$ satisfying $w(k) = o(k^α)$ for some $α< 1/2$ is recurrent. This improves on previous results of Volkov (2006) and Schapira (2012).
We consider a vertex reinforced random walk on the integer lattice with sub-linear reinforcement. Under some assumptions on the regular variation of the weight function, we characterize whether the walk gets stuck on a finite interval. When this happens, we estimate the size of the localization set. In particular, we show that, for any odd number $N$ larger than or equal to 5, there exists a vertex reinforced random walk which localizes with positive probability on exactly $N$ consecutive sites.
We prove that vertex-reinforced random walk on the integers with weight of order k to the power alpha, for alpha in [0, 1/2), is recurrent. This confirms a conjecture of Volkov for alpha<1/2. The conjecture for alpha in [1/2, 1) remains open.
We consider a generalized version of a directionally reinforced random walk, which was originally introduced by Mauldin, Monticino, and von Weizsäcker in \cite{drw}. Our main result is a stable limit theorem for the position of the random walk in higher dimensions. This extends a result of Horváth and Shao \cite{limits} that was previously obtained in dimension one only (however, in a more stringent functional form).
We continue the investigation of the localization phenomenon for a Vertex Reinforced Random Walk on the integer lattice. We provide some partial results towards a full characterization of the weights for which localization on 5 sites occurs with positive probability, and make some conjecture concerning the almost sure behavior.
We study Vertex-Reinforced-Random-Walk on the complete graph with weights of the form $w(n)=n^α$, with $α>1$. Unlike for the Edge-Reinforced-Random-Walk, which in this case localizes a.s. on 2 sites, here we observe various phase transitions, and in particular localization on arbitrary large sets is possible, provided $α$ is close enough to 1. Our proof relies on stochastic approximation techniques. At the end of the paper, we also prove a general result ensuring that any strongly reinforced VRRW on any bounded degree graph localizes a.s. on a finite subgraph.
We introduce random interlacements for transient vertex-reinforced jump processes on a general graph $G$. Using increasing finite subgraphs $G_n$ of $G$ with wired boundary conditions, we show convergence of the vertex-reinforced jump process on $G_n$ observed in a finite window to the random interlacement observed in the same window.
In this article we establish for the superdiffusive regime $p \in (1/2,1)$ that the fluctuations of a general step-reinforced random walk around $a_n \hat{W}$, where $(a_n)_{n \in \mathbb{N}}$ is a non-negative sequence of order $n^p$ and $\hat{W}$ is a non-degenerate random variable, is Gaussian. This extends a known result by Kubota and Takei for the elephant random walk to the more general setting of step-reinforced random walks. Further, we provide an application of the asymptotic normality of $\hat{S}$ around $a_n \hat{W}$ to reinforced empirical processes as studied recently by Bertoin, …
We prove that the only nearest neighbor jump process with local dependence on the occupation times satisfying the partial exchangeability property is the vertex reinforced jump process, under some technical conditions. This result gives a counterpart to the characterization of edge reinforced random walk given by Rolles.
We consider a linearly edge-reinforced random walk on a class of two-dimensional graphs with constant initial weights. The graphs are obtained from $\mathbb{Z}^2$ by replacing every edge by a sufficiently large, but fixed number of edges in series. We prove that the linearly edge-reinforced random walk on these graphs is recurrent. Furthermore, we derive bounds for the probability that the edge-reinforced random walk hits the boundary of a large box before returning to its starting point.
Consider a vertex-reinforced jump process defined on a regular tree, where each vertex has exactly $b$ children, with $b \ge 3$. We prove the strong law of large numbers and the central limit theorem for the distance of the process from the root. Notice that it is still unknown if vertex-reinforced jump process is transient on the binary tree.
We consider several variants of a class of random walks whose increment distributions depend on the average value of the process over its most recent $N$ steps. We investigate the speed of the process, and in particular, the limiting speed as the "history window" $N\to\infty$.
We prove that Vertex Reinforced Random Walk on $\mathbb{Z}$ with weight of order $k^α$, with $α\in [0,1/2)$, is either almost surely recurrent or almost surely transient. This improves a previous result of Volkov who showed that the set of sites which are visited infinitely often was a.s. either empty or infinite.
We describe and analyze how reinforced random walks can eventually localize, i.e. only visit finitely many sites. After introducing vertex and edge self-interacting walks on a discrete graph in a general setting, and stating the main results and conjectures so far on the topic, we present martingale techniques that provide an alternative proof of the a.s. localization of vertex-reinforced random walks (VRRWs) on the integers on finitely many sites and, with positive probability, on five consecutive sites, initially proved by Pemantle and Volkov (1999). Next we introduce the continuous time-lin…
We prove that the edge-reinforced random walk on the ladder ${\mathbb{Z}\times\{1,2\}}$ with initial weights $a>3/4$ is recurrent. The proof uses a known representation of the edge-reinforced random walk on a finite piece of the ladder as a random walk in a random environment. This environment is given by a marginal of a multicomponent Gibbsian process. A transfer operator technique and entropy estimates from statistical mechanics are used to analyze this Gibbsian process. Furthermore, we prove spatially exponentially fast decreasing bounds for normalized local times of the edge-reinforced ran…
We study the asymptotic behaviour of once-reinforced biased random walk (ORbRW) on Galton-Watson trees. Here the underlying (unreinforced) random walk has a bias towards or away from the root. We prove that in the setting of multiplicative once-reinforcement the ORbRW can be recurrent even when the underlying biased random walk is ballistic. We also prove that, on Galton-Watson trees without leaves, the speed is positive in the transient regime. Finally, we prove that, on regular trees, the speed of the ORbRW is monotone decreasing in the reinforcement parameter when the underlying random walk…
The motivation for this paper is the study of the phase transition for recurrence/transience of a class of self-interacting random walks on trees, which includes the once-reinforced random walk. For this purpose, we define a quantity, that we call the branching-ruin number of a tree, which provides (in the spirit of Furstenberg, 1970, and Lyons, 1990) a natural way to measure trees with polynomial growth. We prove that the branching-ruin number of a tree is equal to the critical parameter for the recurrence/transience of the once-reinforced random walk. We define a sharp and effective (i.e. co…
This article studies vertex reinforced random walks that are non-backtracking (denoted VRNBW), i.e. U-turns forbidden. With this last property and for a strong reinforcement, the emergence of a path may occur with positive probability. These walks are thus useful to model the path formation phenomenon, observed for example in ant colonies. This study is carried out in two steps. First, a large class of reinforced random walks is introduced and results on the asymptotic behavior of these processes are proved. Second, these results are applied to VRNBWs on complete graphs and for reinforced weig…
Given a random walk $(S_n)$ with typical step distributed according to some fixed law and a fixed parameter $p \in (0,1)$, the associated positively step-reinforced random walk is a discrete-time process which performs at each step, with probability $1-p$, the same step as $(S_n)$ while with probability $p$, it repeats one of the steps it performed previously chosen uniformly at random. The negatively step-reinforced random walk follows the same dynamics but when a step is repeated its sign is also changed. In this work, we shall prove functional limit theorems for the triplet of a random walk…
We analyze the Brownian Motion limit of a prototypical unit step reinforced random-walk on the half line. A reinforced random walk is one which changes the weight of any edge (or vertex) visited to increase the frequency of return visits. The generating function for the discrete case is first derived for the joint probability distribution of $S_N$ (the location of the walker at the $N$^{th}$ step) and $A_N$ the maximum location the walker achieved in $N$ steps. Then the bulk of the analysis concerns the statistics of the limiting Brownian walker, and of its "environment", both parametrized by …
We revisit an unpublished paper of Vervoort (2002) on the once reinforced random walk, and prove that this process is recurrent on any graph of the form $\mathbb{Z}\times Γ$, with $Γ$ a finite graph, for sufficiently large reinforcement parameter. We also obtain a shape theorem for the set of visited sites, and show that the fluctuations around this shape are of polynomial order. The proof involves sharp general estimates on the time spent on subgraphs of the ambiant graph which might be of independent interest.
We give a short proof of Theorem 2.1 from [MR07], stating that the linearly edge reinforced random walk (ERRW) on a locally finite graph is recurrent if and only if it returns to its starting point almost surely. This result was proved in [MR07] by means of the much stronger property that the law of the ERRW is a mixture of Markov chains. Our proof only uses this latter property on finite graphs, in which case it is a consequence of De Finetti's theorem on exchangeability.
We study the behaviour of a class of edge-reinforced random walks {on $\mathbb{Z}_+$}, with heterogeneous initial weights, where each edge weight can be updated only when the edge is traversed from left to right. We provide a description for different behaviours of this process and describe phase transitions that arise as trade-offs between the strength of the reinforcement and that of the initial weights. Our result aims to complete the ones given by Davis~\cite{Davis89, Davis90}, Takeshima~\cite{Takeshima00, Takeshima01} and Vervoort~\cite{Vervoort00}.
We characterize non-decreasing weight functions for which the associated one-dimensional vertex reinforced random walk (VRRW) localizes on 4 sites. A phase transition appears for weights of order $n\log \log n$: for weights growing faster than this rate, the VRRW localizes almost surely on at most 4 sites whereas for weights growing slower, the VRRW cannot localize on less than 5 sites. When $w$ is of order $n\log \log n$, the VRRW localizes almost surely on either 4 or 5 sites, both events happening with positive probability.
Sabot and Zeng have discovered two martingales, one of which played a key role in their investigation of the vertex-reinforced jump process. Starting from the related supersymmetric hyperbolic sigma model, we give an alternative derivation of these two martingales. They turn out to be the first two instances in an infinite hierarchy of martingales, derived from a generating function.
We study linearly edge-reinforced random walks on $\mathbb{Z}_+$, where each edge $\{x,x+1\}$ has the initial weight $x^α \vee 1$, and each time an edge is traversed, its weight is increased by $Δ$. It is known that the walk is recurrent if and only if $α\leq 1$. The aim of this paper is to study the almost sure behavior of the walk in the recurrent regime. For $α<1$ and $Δ>0$, we obtain a limit theorem which is a counterpart of the law of the iterated logarithm for simple random walks. This reveals that the speed of the walk with $Δ>0$ is much slower than $Δ=0$. In the critical case $α=1$, ou…
We give an alternative proof of the fact that the vertex reinforced jump process on Galton- Watson tree has a phase transition between recurrence and transience as a function of c, the initial local time, see [3]. Further, applying the techniques in [1], we show a phase transition between positive speed and null speed for the discrete time process associated in the transient regime.
The aim of our work is to study vertex-reinforced jump processes with super-linear weight function $w(t) = t^α$ , for some $α>1$. On any complete graph $G = (V, E)$, we prove that there is one vertex $v \in V$ such that the total time spent at $v$ almost surely tends to infinity while the total time spent at the remaining vertices is bounded.
In this work, we discuss the smoothly amnesia-reinforced multidimensional elephant random walk (MARW). The scaling limit of the MARW is shown to exist in the diffusive, critical and superdiffusive regimes. We also establish the almost sure convergence in all of the three regimes. The quadratic strong law is displayed in the diffusive regime as well as in the critical regime. The mean square convergence towards a non-Gaussian random variable is established in the superdiffusive regime. Similar results for the barycenter process are also derived. Finally, the last two Sections are devoted to a d…
In this paper we introduce a new simple but powerful general technique for the study of edge- and vertex-reinforced processes with super-linear reinforcement, based on the use of order statistics for the number of edge, respectively of vertex, traversals. The technique relies on upper bound estimates for the number of edge traversals, proved in a different context by Cotar and Limic [Ann. Appl. Probab. (2009)] for finite graphs with edge reinforcement. We apply our new method both to edge- and to vertex-reinforced random walks with super-linear reinforcement on arbitrary infinite connected gra…
Vertex-Reinforced Random Walk (VRRW), defined by Pemantle (1988a), is a random process in a continuously changing environment which is more likely to visit states it has visited before. We consider VRRW on arbitrary graphs and show that on almost all of them, VRRW visits only finitely many vertices with a positive probability. We conjecture that on all graphs of bounded degree, this happens a.s., and provide a proof only for trees of this type. We distinguish between several different patterns of localization and explicitly describe the long-run structure of VRRW, which depends on whether a gr…
Vertex-reinforced random walk (VRRW), defined by Pemantle in 1988, is a random process that takes values in the vertex set of a graph G, which is more likely to visit vertices it has visited before. Pemantle and Volkov considered the case when the underlying graph is the one-dimensional integer lattice Z. They proved that the range is almost surely finite and that with positive probability the range contains exactly five points. They conjectured that this second event holds with probability 1. The proof of this conjecture is the main purpose of this paper.
Consider a negatively drifted one dimensional Brownian motion starting at positive initial position, its first hitting time to 0 has the inverse Gaussian law. Moreover, conditionally on this hitting time, the Brownian motion up to that time has the law of a 3- dimensional Bessel bridge. In this paper, we give a generalization of this result to a family of Brownian motions with interacting drifts, indexed by the vertices of a conductance network. The hitting times are equal in law to the inverse of a random potential that appears in the analysis of a self-interacting process called the Vertex R…
We describe a model for $m$ vertex reinforced interacting random walks on complete graphs with $d\geq 2$ vertices. The transition probability of a random walk to a given vertex depends exponentially on the proportion of visits made by all walks to that vertex. The individual proportion of visits is modulated by a strength parameter that can be set equal to any real number. This model covers a large variety of interactions including different vertex repulsion and attraction strengths between any two random walks as well as self-reinforced interactions. We show that the process of empirical vert…
We consider diffusivity of random walks with transition probabilities depending on the number of consecutive traversals of the last traversed edge, the so called senile reinforced random walk (SeRW). In one dimension, the walk is known to be sub-diffusive with identity reinforcement function. We perturb the model by introducing a small probability $δ$ of escaping the last traversed edge at each step. The perturbed SeRW model is diffusive for any $δ>0 $, with enhanced diffusivity ($\gg O(δ^2)$) in the small $δ$ regime. We further study stochastically perturbed SeRW models by having the last edg…
