The Experts below are selected from a list of 91104 Experts worldwide ranked by ideXlab platform
Peter Korosec - One of the best experts on this subject based on the ideXlab platform.
-
understanding exploration and exploitation powers of meta heuristic Stochastic Optimization algorithms through statistical analysis
Genetic and Evolutionary Computation Conference, 2019Co-Authors: Tome Eftimov, Peter KorosecAbstract:Understanding of exploration and exploitation powers of meta-heuristic Stochastic Optimization algorithms is very important for algorithm developers. For this reason, we have recently proposed an approach for making a statistical comparison of meta-heuristic Stochastic Optimization algorithms according to the distribution of the solutions in the search space, which is also presented in this paper. Its main contribution is the support to identify exploration and exploitation powers of the compared algorithms. This is especially important when dealing with multimodal search spaces, which consist of many local optima with similar values, and large-scale continuous Optimization problems, where it is hard to understand the reasons for the differences in performances. Experimental results showed that our recently proposed approach gives very promising results.
-
a novel statistical approach for comparing meta heuristic Stochastic Optimization algorithms according to the distribution of solutions in the search space
Information Sciences, 2019Co-Authors: Tome Eftimov, Peter KorosecAbstract:Abstract In this paper a novel statistical approach for comparing meta-heuristic Stochastic Optimization algorithms according to the distribution of the solutions in the search space is introduced, known as extended Deep Statistical Comparison. This approach is an extension of the recently proposed Deep Statistical Comparison approach used for comparing meta-heuristic Stochastic Optimization algorithms according to the solutions values. Its main contribution is that the algorithms are compared not only according to obtained solutions values, but also according to the distribution of the obtained solutions in the search space. The information it provides can additionally help to identify exploitation and exploration powers of the compared algorithms. This is important when dealing with a multimodal search space, where there are a lot of local optima with similar values. The benchmark results show that our proposed approach gives promising results and can be used for a statistical comparison of meta-heuristic Stochastic Optimization algorithms according to solutions values and their distribution in the search space.
Tome Eftimov - One of the best experts on this subject based on the ideXlab platform.
-
understanding exploration and exploitation powers of meta heuristic Stochastic Optimization algorithms through statistical analysis
Genetic and Evolutionary Computation Conference, 2019Co-Authors: Tome Eftimov, Peter KorosecAbstract:Understanding of exploration and exploitation powers of meta-heuristic Stochastic Optimization algorithms is very important for algorithm developers. For this reason, we have recently proposed an approach for making a statistical comparison of meta-heuristic Stochastic Optimization algorithms according to the distribution of the solutions in the search space, which is also presented in this paper. Its main contribution is the support to identify exploration and exploitation powers of the compared algorithms. This is especially important when dealing with multimodal search spaces, which consist of many local optima with similar values, and large-scale continuous Optimization problems, where it is hard to understand the reasons for the differences in performances. Experimental results showed that our recently proposed approach gives very promising results.
-
a novel statistical approach for comparing meta heuristic Stochastic Optimization algorithms according to the distribution of solutions in the search space
Information Sciences, 2019Co-Authors: Tome Eftimov, Peter KorosecAbstract:Abstract In this paper a novel statistical approach for comparing meta-heuristic Stochastic Optimization algorithms according to the distribution of the solutions in the search space is introduced, known as extended Deep Statistical Comparison. This approach is an extension of the recently proposed Deep Statistical Comparison approach used for comparing meta-heuristic Stochastic Optimization algorithms according to the solutions values. Its main contribution is that the algorithms are compared not only according to obtained solutions values, but also according to the distribution of the obtained solutions in the search space. The information it provides can additionally help to identify exploitation and exploration powers of the compared algorithms. This is important when dealing with a multimodal search space, where there are a lot of local optima with similar values. The benchmark results show that our proposed approach gives promising results and can be used for a statistical comparison of meta-heuristic Stochastic Optimization algorithms according to solutions values and their distribution in the search space.
Junshan Zhang - One of the best experts on this subject based on the ideXlab platform.
-
Stochastic Optimization based economic dispatch and interruptible load management with increased wind penetration
IEEE Transactions on Smart Grid, 2016Co-Authors: Lei Yang, Vijay Vittal, Junshan ZhangAbstract:In this paper, Stochastic Optimization of economic dispatch (ED) and interruptible load management is investigated using short-term distributional forecast of wind farm generation. Specifically, using the statistical information of wind farm generation extracted from historical data, a Markov chain-based distributional forecast model for wind farm generation is developed in a rigorous Optimization framework, in which the diurnal nonstationarity and the seasonality of wind generation are accounted for by constructing multiple finite-state Markov chains for each epoch of 3 h and for each individual month. Based on this distributional forecast model, the joint Optimization of ED and interruptible load management is cast as a Stochastic Optimization problem. Additionally, a robust ED is formulated using an uncertainty set constructed based on the proposed distributional forecast, aiming to minimize the system cost for worst cases. The proposed Stochastic ED is compared with four other ED schemes: 1) the robust ED; 2) deterministic ED using the persistence wind generation forecast model; 3) scenario-based Stochastic ED; and 4) deterministic ED, in which perfect wind generation forecasts are used. Numerical studies, using the IEEE Reliability Test System-1996 and realistic wind measurement data from an actual wind farm, demonstrate the significant benefits obtained by leveraging the Markov chain-based distributional forecast and the interruptible load management.
-
Stochastic Optimization based economic dispatch and interruptible load management with distributional forecast of wind farm generation
Conference on Decision and Control, 2014Co-Authors: Lei Yang, Vijay Vittal, Junshan ZhangAbstract:We study Stochastic Optimization of economic dispatch (ED) and interruptible load management using short-term distributional forecast of wind farm generation. Specifically, we develop a Markov-chain-based distributional forecast model for wind farm generation based on spatial and temporal characteristics of the wind turbine power output in a wind farm. Built on the distributional forecast model, the joint Optimization of ED and interruptible load management is cast as a Stochastic Optimization problem. The proposed Stochastic ED problem is compared with the deterministic ED problem using the persistence wind generation forecast model, and also with the scenario-based ED formulation which uses all possible wind generation states. Numerical studies, using a modified IEEE RTS 24-bus system and realistic wind measurement data from an actual wind farm, demonstrate the significant benefits obtained by leveraging the Markov-chain-based distributional forecast and the interruptible load management.
Lei Yang - One of the best experts on this subject based on the ideXlab platform.
-
Stochastic Optimization based economic dispatch and interruptible load management with increased wind penetration
IEEE Transactions on Smart Grid, 2016Co-Authors: Lei Yang, Vijay Vittal, Junshan ZhangAbstract:In this paper, Stochastic Optimization of economic dispatch (ED) and interruptible load management is investigated using short-term distributional forecast of wind farm generation. Specifically, using the statistical information of wind farm generation extracted from historical data, a Markov chain-based distributional forecast model for wind farm generation is developed in a rigorous Optimization framework, in which the diurnal nonstationarity and the seasonality of wind generation are accounted for by constructing multiple finite-state Markov chains for each epoch of 3 h and for each individual month. Based on this distributional forecast model, the joint Optimization of ED and interruptible load management is cast as a Stochastic Optimization problem. Additionally, a robust ED is formulated using an uncertainty set constructed based on the proposed distributional forecast, aiming to minimize the system cost for worst cases. The proposed Stochastic ED is compared with four other ED schemes: 1) the robust ED; 2) deterministic ED using the persistence wind generation forecast model; 3) scenario-based Stochastic ED; and 4) deterministic ED, in which perfect wind generation forecasts are used. Numerical studies, using the IEEE Reliability Test System-1996 and realistic wind measurement data from an actual wind farm, demonstrate the significant benefits obtained by leveraging the Markov chain-based distributional forecast and the interruptible load management.
-
Stochastic Optimization based economic dispatch and interruptible load management with distributional forecast of wind farm generation
Conference on Decision and Control, 2014Co-Authors: Lei Yang, Vijay Vittal, Junshan ZhangAbstract:We study Stochastic Optimization of economic dispatch (ED) and interruptible load management using short-term distributional forecast of wind farm generation. Specifically, we develop a Markov-chain-based distributional forecast model for wind farm generation based on spatial and temporal characteristics of the wind turbine power output in a wind farm. Built on the distributional forecast model, the joint Optimization of ED and interruptible load management is cast as a Stochastic Optimization problem. The proposed Stochastic ED problem is compared with the deterministic ED problem using the persistence wind generation forecast model, and also with the scenario-based ED formulation which uses all possible wind generation states. Numerical studies, using a modified IEEE RTS 24-bus system and realistic wind measurement data from an actual wind farm, demonstrate the significant benefits obtained by leveraging the Markov-chain-based distributional forecast and the interruptible load management.
Ma Xuezhe - One of the best experts on this subject based on the ideXlab platform.
-
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization
2021Co-Authors: Ma XuezheAbstract:In this paper, we introduce Apollo, a quasi-Newton method for nonconvex Stochastic Optimization, which dynamically incorporates the curvature of the loss function by approximating the Hessian via a diagonal matrix. Importantly, the update and storage of the diagonal approximation of Hessian is as efficient as adaptive first-order Optimization methods with linear complexity for both time and memory. To handle nonconvexity, we replace the Hessian with its rectified absolute value, which is guaranteed to be positive-definite. Experiments on three tasks of vision and language show that Apollo achieves significant improvements over other Stochastic Optimization methods, including SGD and variants of Adam, in term of both convergence speed and generalization performance. The implementation of the algorithm is available at https://github.com/XuezheMax/apollo.Comment: Added convergence analysis, more baseline methods, and more discussions and extensions. 29 pages (plus appendix), 6 figures, 7 table
-
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization
2021Co-Authors: Ma XuezheAbstract:In this paper, we introduce Apollo, a quasi-Newton method for nonconvex Stochastic Optimization, which dynamically incorporates the curvature of the loss function by approximating the Hessian via a diagonal matrix. Importantly, the update and storage of the diagonal approximation of Hessian is as efficient as adaptive first-order Optimization methods with linear complexity for both time and memory. To handle nonconvexity, we replace the Hessian with its rectified absolute value, which is guaranteed to be positive-definite. Experiments on three tasks of vision and language show that Apollo achieves significant improvements over other Stochastic Optimization methods, including SGD and variants of Adam, in term of both convergence speed and generalization performance. The implementation of the algorithm is available at https://github.com/XuezheMax/apollo.Comment: Fixed errors in convergence analysis. 29 pages (plus appendix), 6 figures, 7 table
-
Apollo: An Adaptive Parameter-wise Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization
2020Co-Authors: Ma XuezheAbstract:In this paper, we introduce Apollo, a quasi-Newton method for nonconvex Stochastic Optimization, which dynamically incorporates the curvature of the loss function by approximating the Hessian via a diagonal matrix. Importantly, the update and storage of the diagonal approximation of Hessian is as efficient as adaptive first-order Optimization methods with linear complexity for both time and memory. To handle nonconvexity, we replace the Hessian with its rectified absolute value, which is guaranteed to be positive-definite. Experiments on three tasks of vision and language show that Apollo achieves significant improvements over other Stochastic Optimization methods, including SGD and variants of Adam, in term of both convergence speed and generalization performance. The implementation of the algorithm is available at https://github.com/XuezheMax/apollo.Comment: Draft version. Work in progress. 15 pages (plus appendix), 4 figures, 4 tables. Fixed typos in the first version of preprin
