The Experts below are selected from a list of 303 Experts worldwide ranked by ideXlab platform
Ma Xuezhe - One of the best experts on this subject based on the ideXlab platform.
-
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization
2021Co-Authors: Ma XuezheAbstract:In this paper, we introduce Apollo, a Quasi-Newton Method for nonconvex stochastic optimization, which dynamically incorporates the curvature of the loss function by approximating the Hessian via a diagonal matrix. Importantly, the update and storage of the diagonal approximation of Hessian is as efficient as adaptive first-order optimization Methods with linear complexity for both time and memory. To handle nonconvexity, we replace the Hessian with its rectified absolute value, which is guaranteed to be positive-definite. Experiments on three tasks of vision and language show that Apollo achieves significant improvements over other stochastic optimization Methods, including SGD and variants of Adam, in term of both convergence speed and generalization performance. The implementation of the algorithm is available at https://github.com/XuezheMax/apollo.Comment: Added convergence analysis, more baseline Methods, and more discussions and extensions. 29 pages (plus appendix), 6 figures, 7 table
-
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization
2021Co-Authors: Ma XuezheAbstract:In this paper, we introduce Apollo, a Quasi-Newton Method for nonconvex stochastic optimization, which dynamically incorporates the curvature of the loss function by approximating the Hessian via a diagonal matrix. Importantly, the update and storage of the diagonal approximation of Hessian is as efficient as adaptive first-order optimization Methods with linear complexity for both time and memory. To handle nonconvexity, we replace the Hessian with its rectified absolute value, which is guaranteed to be positive-definite. Experiments on three tasks of vision and language show that Apollo achieves significant improvements over other stochastic optimization Methods, including SGD and variants of Adam, in term of both convergence speed and generalization performance. The implementation of the algorithm is available at https://github.com/XuezheMax/apollo.Comment: Fixed errors in convergence analysis. 29 pages (plus appendix), 6 figures, 7 table
-
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization
2020Co-Authors: Ma XuezheAbstract:In this paper, we introduce Apollo, a Quasi-Newton Method for nonconvex stochastic optimization, which dynamically incorporates the curvature of the loss function by approximating the Hessian via a diagonal matrix. Importantly, the update and storage of the diagonal approximation of Hessian is as efficient as adaptive first-order optimization Methods with linear complexity for both time and memory. To handle nonconvexity, we replace the Hessian with its rectified absolute value, which is guaranteed to be positive-definite. Experiments on three tasks of vision and language show that Apollo achieves significant improvements over other stochastic optimization Methods, including SGD and variants of Adam, in term of both convergence speed and generalization performance. The implementation of the algorithm is available at https://github.com/XuezheMax/apollo.Comment: Draft version. Work in progress. 15 pages (plus appendix), 4 figures, 4 tables. Fixed typos in the first version of preprin
Delphine Picart - One of the best experts on this subject based on the ideXlab platform.
-
parameter identification in multistage population dynamics model
Nonlinear Analysis-real World Applications, 2011Co-Authors: Bedreddine Ainseba, Delphine PicartAbstract:Abstract We present a numerical analysis to solve a parameter identification problem. We identify the demographical parameters of a multistage population dynamics model (Ainseba et al., 2011 [12] ). Our nonlinear optimization problem with constraints is solved by a Quasi-Newton Method. The convergence proof of this numerical Method is performed here. Some numerical applications of it are also given at the end of the paper.
Marvin L Cohen - One of the best experts on this subject based on the ideXlab platform.
-
relaxation of crystals with the quasi newton Method
Journal of Computational Physics, 1997Co-Authors: Bernd G Pfrommer, Michel Cote, Steven G Louie, Marvin L CohenAbstract:A Quasi-Newton Method is used to simultaneously relax the internal coordinates and lattice parameters of crystals under pressure. The symmetry of the crystal structure is preserved during the relaxation. From the inverse of the Hessian matrix, elastic properties, and some optical phonon frequencies at the Brillouin zone center can be estimated. The efficiency of the Method is demonstrated for silicon test systems.
Joos Vandewalle - One of the best experts on this subject based on the ideXlab platform.
-
A globally convergent algorithm for solving a broad class of nonlinear resistive circuits
IEEE International Symposium on Circuits and Systems, 1Co-Authors: Lieven Vandenberghe, Joos VandewalleAbstract:A globally convergent algorithm for solving sets of nonlinear equations was applied to resistive circuits. Criteria for convergence of the algorithm coincide with very broad sufficient conditions for the existence of solutions. Some of the features of a simple implementation of the algorithm are discussed briefly. The combination of the basic Method with a Quasi-Newton Method and the exploitation of linear structure leads to an efficient and reliable algorithm. >
Qinglai Guo - One of the best experts on this subject based on the ideXlab platform.
-
dynamic economic dispatch using lagrangian relaxation with multiplier updates based on a quasi newton Method
IEEE Transactions on Power Systems, 2013Co-Authors: Boming Zhang, Hongbin Sun, Qinglai GuoAbstract:To accommodate large-scale integration of renewable generation, look-ahead power dispatch, which is essentially a type of dynamic economic dispatch (DED), is widely used in power system operation. DED can be reduced to a two-stage dual problem via Lagrangian relaxation. The major challenge to solving this dual problem robustly and efficiently is the multiplier updating strategy. To solve the dual problems of quadratic DEDs, this paper proposes multiplier updating based on a Quasi-Newton Method, together with an initialization strategy for the multipliers and the approximation matrix. Comparative numerical tests are carried out to verify the performance of this technique in terms of convergence, computational efficiency, and robustness.
