The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform
Jean-marie Monnez - One of the best experts on this subject based on the ideXlab platform.
-
Almost Sure Convergence of a stochastic approximation process in a convex set
International Journal of Apllied Mathematics, 2007Co-Authors: Abdelkrim Bennar, Jean-marie MonnezAbstract:We prove two Almost Sure Convergence theorems of a stochastic approximation process in a convex set and give two applications
-
Almost Sure Convergence of stochastic gradient processes with matrix step sizes
Statistics & Probability Letters, 2006Co-Authors: Jean-marie MonnezAbstract:We consider a stochastic gradient process, which is a special case of stochastic approximation process, where the positive real step size an is replaced by a random matrix An: Xn+1=Xn-An[backward difference]g(Xn)-AnVn. We give two theorems of Almost Sure Convergence in the case where the equation [backward difference]g=0 has a set of solutions.
-
Almost Sure Convergence of stochastic gradient processes with matrix step sizes
Statistics & Probability Letters, 2006Co-Authors: Jean-marie MonnezAbstract:Abstract We consider a stochastic gradient process, which is a special case of stochastic approximation process, where the positive real step size a n is replaced by a random matrix A n : X n + 1 = X n - A n ∇ g ( X n ) - A n V n . We give two theorems of Almost Sure Convergence in the case where the equation ∇ g = 0 has a set of solutions.
Matthias Meiners - One of the best experts on this subject based on the ideXlab platform.
-
exponential rate of Almost Sure Convergence of intrinsic martingales in supercritical branching random walks
Journal of Applied Probability, 2010Co-Authors: Alexander Iksanov, Matthias MeinersAbstract:We provide sufficient conditions which enSure that the intrinsic martingale in the supercritical branching random walk converges exponentially fast to its limit. We include in particular the case of Galton-Watson processes so that our results can be seen as a generalization of a result given in the classical treatise by Asmussen and Hering (1983). As an auxiliary tool, we prove ultimate versions of two results concerning the exponential renewal meaSures which may be of interest in themselves and which correct, generalize, and simplify some earlier works.
-
exponential rate of Almost Sure Convergence of intrinsic martingales in supercritical branching random walks
arXiv: Probability, 2009Co-Authors: Alexander Iksanov, Matthias MeinersAbstract:We provide sufficient conditions which enSure that the intrinsic martingale in the supercritical branching random walk converges exponentially fast to its limit. The case of Galton-Watson processes is particularly included so that our results can be seen as a generalization of a result given in the classical treatise by Asmussen and Hering. As an auxiliary tool, we prove ultimate versions of two results concerning the exponential renewal meaSures which may be interesting on its own and which correct, generalize and simplify some earlier works.
Alexander Iksanov - One of the best experts on this subject based on the ideXlab platform.
-
exponential rate of Almost Sure Convergence of intrinsic martingales in supercritical branching random walks
Journal of Applied Probability, 2010Co-Authors: Alexander Iksanov, Matthias MeinersAbstract:We provide sufficient conditions which enSure that the intrinsic martingale in the supercritical branching random walk converges exponentially fast to its limit. We include in particular the case of Galton-Watson processes so that our results can be seen as a generalization of a result given in the classical treatise by Asmussen and Hering (1983). As an auxiliary tool, we prove ultimate versions of two results concerning the exponential renewal meaSures which may be of interest in themselves and which correct, generalize, and simplify some earlier works.
-
exponential rate of Almost Sure Convergence of intrinsic martingales in supercritical branching random walks
arXiv: Probability, 2009Co-Authors: Alexander Iksanov, Matthias MeinersAbstract:We provide sufficient conditions which enSure that the intrinsic martingale in the supercritical branching random walk converges exponentially fast to its limit. The case of Galton-Watson processes is particularly included so that our results can be seen as a generalization of a result given in the classical treatise by Asmussen and Hering. As an auxiliary tool, we prove ultimate versions of two results concerning the exponential renewal meaSures which may be interesting on its own and which correct, generalize and simplify some earlier works.
Volkan Cevher - One of the best experts on this subject based on the ideXlab platform.
-
on the Almost Sure Convergence of stochastic gradient descent in non convex problems
Neural Information Processing Systems, 2020Co-Authors: Panayotis Mertikopoulos, Nadav Hallak, Ali Kavis, Volkan CevherAbstract:This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's Convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with probability 1 under a very broad range of step-size schedules. Subsequently, going beyond existing positive probability guarantees, we show that SGD avoids strict saddle points/manifolds with probability 1 for the entire spectrum of step-size policies considered. Finally, we prove that the algorithm's rate of Convergence to Hurwicz minimizers is O(1/n p) if the method is employed with a Θ(1/n p) step-size. This provides an important guideline for tuning the algorithm's step-size as it suggests that a cool-down phase with a vanishing step-size could lead to faster Convergence; we demonstrate this heuristic using ResNet architectures on CIFAR.
-
on the Almost Sure Convergence of stochastic gradient descent in non convex problems
arXiv: Optimization and Control, 2020Co-Authors: Panayotis Mertikopoulos, Nadav Hallak, Ali Kavis, Volkan CevherAbstract:This paper analyzes the trajectories of stochastic gradient descent (SGD) to help understand the algorithm's Convergence properties in non-convex problems. We first show that the sequence of iterates generated by SGD remains bounded and converges with probability $1$ under a very broad range of step-size schedules. Subsequently, going beyond existing positive probability guarantees, we show that SGD avoids strict saddle points/manifolds with probability $1$ for the entire spectrum of step-size policies considered. Finally, we prove that the algorithm's rate of Convergence to Hurwicz minimizers is $\mathcal{O}(1/n^{p})$ if the method is employed with a $\Theta(1/n^p)$ step-size schedule. This provides an important guideline for tuning the algorithm's step-size as it suggests that a cool-down phase with a vanishing step-size could lead to faster Convergence; we demonstrate this heuristic using ResNet architectures on CIFAR.
Zhan Shi - One of the best experts on this subject based on the ideXlab platform.
-
Almost Sure Convergence for stochastically biased random walks on trees
Probability Theory and Related Fields, 2012Co-Authors: Gabriel Faraud, Zhan ShiAbstract:We are interested in the biased random walk on a supercritical Galton–Watson tree in the sense of Lyons (Ann. Probab. 18:931–958, 1990) and Lyons, Pemantle and Peres (Probab. Theory Relat. Fields 106:249–264, 1996), and study a phenomenon of slow movement. In order to observe such a slow movement, the bias needs to be random; the resulting random walk is then a tree-valued random walk in random environment. We investigate the recurrent case, and prove, under suitable general integrability assumptions, that upon the system’s non-extinction, the maximal displacement of the walk in the first n steps, divided by (log n)3, converges Almost Surely to a known positive constant.
-
Almost Sure Convergence for stochastically biased random walks on trees
arXiv: Probability, 2010Co-Authors: Gabriel Faraud, Zhan ShiAbstract:We are interested in the biased random walk on a supercritical Galton--Watson tree in the sense of Lyons, Pemantle and Peres, and study a phenomenon of slow movement. In order to observe such a slow movement, the bias needs to be random; the resulting random walk is then a tree-valued random walk in random environment. We investigate the recurrent case, and prove, under suitable general integrability assumptions, that upon the system's non-extinction, the maximal displacement of the walk in the first n steps, divided by (log n)^3, converges Almost Surely to a known positive constant.