The Experts below are selected from a list of 53841 Experts worldwide ranked by ideXlab platform
Donya Yazdani - One of the best experts on this subject based on the ideXlab platform.
-
when hypermutations and ageing enable artificial immune systems to outperform evolutionary algorithms
Theoretical Computer Science, 2020Co-Authors: Dogan Corus, Pietro S Oliveto, Donya YazdaniAbstract:Abstract We present a time complexity analysis of the Opt-IA artificial immune system (AIS). We first highlight the power and limitations of its distinguishing operators (i.e., hypermutations with mutation potential and ageing) by analysing them in isolation. Recent work has shown that ageing combined with local mutations can help escape local optima on a dynamic optimisation Benchmark Function. We generalise this result by rigorously proving that, compared to evolutionary algorithms (EAs), ageing leads to impressive speed-ups on the standard Benchmark Function both when using local and global mutations. Unless the stop at first constructive mutation (FCM) mechanism is applied, we show that hypermutations require exponential expected runtime to optimise any Function with a polynomial number of optima. If instead FCM is used, the expected runtime is at most a linear factor larger than the upper bound achieved for any random local search algorithm using the artificial fitness levels method. Nevertheless, we prove that algorithms using hypermutations can be considerably faster than EAs at escaping local optima. An analysis of the complete Opt-IA reveals that it is efficient on the previously considered Functions and highlights problems where the use of the full algorithm is crucial. We complete the picture by presenting a class of Functions for which Opt-IA fails with overwhelming probability while standard EAs are efficient.
-
on the runtime analysis of the opt ia artificial immune system
Genetic and Evolutionary Computation Conference, 2017Co-Authors: Dogan Corus, Pietro S Oliveto, Donya YazdaniAbstract:We present a time complexity analysis of the Opt-IA artificial immune system (AIS). We first highlight the power and limitations of its distinguishing operators (i.e., hypermutations with mutation potential and ageing) by analysing them in isolation. Recent work has shown that ageing combined with local mutations can help escape local optima on a dynamic optimisation Benchmark Function. We generalise this result by rigorously proving that ageing leads to considerable speed-ups (compared to evolutionary algorithms (EAs)) on the standard C liff Benchmark Function both when using local and global mutations. Unless the stop at first constructive mutation (FCM) mechanism is applied, we show that hypermutations require exponential expected runtime to optimise any Function with a polynomial number of optima. If instead FCM is used, the expected runtime is at most a linear factor larger than the upper bound achieved for any random local search algorithm using the artificial fitness levels method. Nevertheless, we prove that algorithms using hypermutations can be considerably faster than EAs at escaping local optima. An analysis of the complete Opt-IA reveals that it is efficient on the previously considered Functions and highlights problems where the use of the full algorithm is crucial.
Doerr Benjamin - One of the best experts on this subject based on the ideXlab platform.
-
Does Comma Selection Help To Cope With Local Optima
'Association for Computing Machinery (ACM)', 2021Co-Authors: Doerr BenjaminAbstract:One hope when using non-elitism in evolutionary computation is that the ability to abandon the current-best solution aids leaving local optima. To improve our understanding of this mechanism, we perform a rigorous runtime analysis of a basic non-elitist evolutionary algorithm (EA), the $(\mu,\lambda)$ EA, on the most basic Benchmark Function with a local optimum, the jump Function. We prove that for all reasonable values of the parameters and the problem, the expected runtime of the $(\mu,\lambda)$~EA is, apart from lower order terms, at least as large as the expected runtime of its elitist counterpart, the $(\mu+\lambda)$~EA (for which we conduct the first runtime analysis on jump Functions to allow this comparison). Consequently, the ability of the $(\mu,\lambda)$~EA to leave local optima to inferior solutions does not lead to a runtime advantage. We complement this lower bound with an upper bound that, for broad ranges of the parameters, is identical to our lower bound apart from lower order terms. This is the first runtime result for a non-elitist algorithm on a multi-modal problem that is tight apart from lower order terms.Comment: 36 pages. Full version of a paper that appeared at GECCO 202
-
A Tight Runtime Analysis for the (µ + λ) EA
'Springer Science and Business Media LLC', 2021Co-Authors: Antipov Denis, Doerr BenjaminAbstract:International audienceDespite significant progress in the theory of evolutionary algorithms, the theoretical understanding of evolutionary algorithms which use non-trivial populations remains challenging and only few rigorous results exist. Already for the most basic problem, the determination of the asymptotic runtime of the $(\mu+\lambda)$ evolutionary algorithm on the simple OneMax Benchmark Function, only the special cases $\mu=1$ and $\lambda=1$ have been solved.In this work, we analyze this long-standing problem and show the asymptotically tight result that the runtime $T$, the number of iterations until the optimum is found, satisfies \[E[T] = \Theta\bigg(\frac{n\log n}{\lambda}+\frac{n}{\lambda / \mu} + \frac{n\log^+\log^+ \lambda/ \mu}{\log^+ \lambda / \mu}\bigg),\] where $\log^+ x := \max\{1, \log x\}$ for all $x > 0$.The same methods allow to improve the previous-best $O(\frac{n \log n}{\lambda} + n \log \lambda)$ runtime guarantee for the $(\lambda+\lambda)$~EA with fair parent selection to a tight $\Theta(\frac{n \log n}{\lambda} + n)$ runtime result
-
Fast Mutation in Crossover-based Algorithms
HAL CCSD, 2021Co-Authors: Antipov Denis, Buzdalov Maxim, Doerr BenjaminAbstract:The heavy-tailed mutation operator proposed in Doerr, Le, Makhmara, and Nguyen (GECCO 2017), called \emph{fast mutation} to agree with the previously used language, so far was proven to be advantageous only in mutation-based algorithms. There, it can relieve the algorithm designer from finding the optimal mutation rate and nevertheless obtain a performance close to the one that the optimal mutation rate gives.In this first runtime analysis of a crossover-based algorithm using a heavy-tailed choice of the mutation rate, we show an even stronger impact. For the $(1+(\lambda,\lambda))$ genetic algorithm optimizing the OneMax Benchmark Function, we show that with a heavy-tailed mutation rate a linear runtime can be achieved. This is asymptotically faster than what can be obtained with any static mutation rate, and is asymptotically equivalent to the runtime of the self-adjusting version of the parameters choice of the $(1+(\lambda,\lambda))$ genetic algorithm. This result is complemented by an empirical study which shows the effectiveness of the fast mutation also on random satisfiable Max-3SAT instances
-
Does Comma Selection Help To Cope With Local Optima?
HAL CCSD, 2021Co-Authors: Doerr BenjaminAbstract:One hope when using non-elitism in evolutionary computation is that the ability to abandon the current-best solution aids leaving local optima. To improve our understanding of this mechanism, we perform a rigorous runtime analysis of a basic non-elitist evolutionary algorithm (EA), the (µ, λ) EA, on the most basic Benchmark Function with a local optimum, the jump Function. We prove that for all reasonable values of the parameters and the problem, the expected runtime of the (µ, λ) EA is, apart from lower order terms, at least as large as the expected runtime of its elitist counterpart, the (µ + λ) EA (for which we conduct the first runtime analysis on jump Functions to allow this comparison). Consequently, the ability of the (µ, λ) EA to leave local optima to inferior solutions does not lead to a runtime advantage. We complement this lower bound with an upper bound that, for broad ranges of the parameters, is identical to our lower bound apart from lower order terms. This is the first runtime result for a non-elitist algorithm on a multi-modal problem that is tight apart from lower order terms
-
Fast Mutation in Crossover-based Algorithms
2021Co-Authors: Antipov Denis, Buzdalov Maxim, Doerr BenjaminAbstract:The heavy-tailed mutation operator proposed in Doerr, Le, Makhmara, and Nguyen (GECCO 2017), called \emph{fast mutation} to agree with the previously used language, so far was proven to be advantageous only in mutation-based algorithms. There, it can relieve the algorithm designer from finding the optimal mutation rate and nevertheless obtain a performance close to the one that the optimal mutation rate gives. In this first runtime analysis of a crossover-based algorithm using a heavy-tailed choice of the mutation rate, we show an even stronger impact. For the $(1+(\lambda,\lambda))$ genetic algorithm optimizing the OneMax Benchmark Function, we show that with a heavy-tailed mutation rate a linear runtime can be achieved. This is asymptotically faster than what can be obtained with any static mutation rate, and is asymptotically equivalent to the runtime of the self-adjusting version of the parameters choice of the $(1+(\lambda,\lambda))$ genetic algorithm. This result is complemented by an empirical study which shows the effectiveness of the fast mutation also on random satisfiable Max-3SAT instances.Comment: This is a version of the same paper presented at GECCO 2020 completed with the proofs which were missing because of the page limi
Matthew B Blaschko - One of the best experts on this subject based on the ideXlab platform.
-
additive tree structured conditional parameter spaces in bayesian optimization a novel covariance Function and a fast implementation
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021Co-Authors: Matthew B BlaschkoAbstract:Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box Functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are usually built on trees. In this work, we generalize the additive assumption to tree-structured Functions and propose an additive tree-structured covariance Function, showing improved sample-efficiency, wider applicability and greater flexibility. Furthermore, by incorporating the structure information of parameter spaces and the additive assumption in the BO loop, we develop a parallel algorithm to optimize the acquisition Function and this optimization can be performed in a low dimensional space. We demonstrate our method on an optimization Benchmark Function, on a neural network compression problem and on pruning pre-trained VGG16 and ResNet50 models. Experimental results show our approach significantly outperforms the current state of the art for conditional parameter optimization including SMAC, TPE and Jenatton et al. (2017).
-
additive tree structured covariance Function for conditional parameter spaces in bayesian optimization
International Conference on Artificial Intelligence and Statistics, 2020Co-Authors: Matthew B BlaschkoAbstract:Bayesian optimization (BO) is a sample-efficient global optimization algorithm for black-box Functions which are expensive to evaluate. Existing literature on model based optimization in conditional parameter spaces are usually built on trees. In this work, we generalize the additive assumption to tree-structured Functions and propose an additive tree-structured covariance Function, showing improved sample-efficiency, wider applicability and greater flexibility. Furthermore, by incorporating the structure information of parameter spaces and the additive assumption in the BO loop, we develop a parallel algorithm to optimize the acquisition Function and this optimization can be performed in a low dimensional space. We demonstrate our method on an optimization Benchmark Function, as well as on a neural network model compression problem, and experimental results show our approach significantly outperforms the current state of the art for conditional parameter optimization including SMAC, TPE and Jenatton et al. (2017).
Dogan Corus - One of the best experts on this subject based on the ideXlab platform.
-
when hypermutations and ageing enable artificial immune systems to outperform evolutionary algorithms
Theoretical Computer Science, 2020Co-Authors: Dogan Corus, Pietro S Oliveto, Donya YazdaniAbstract:Abstract We present a time complexity analysis of the Opt-IA artificial immune system (AIS). We first highlight the power and limitations of its distinguishing operators (i.e., hypermutations with mutation potential and ageing) by analysing them in isolation. Recent work has shown that ageing combined with local mutations can help escape local optima on a dynamic optimisation Benchmark Function. We generalise this result by rigorously proving that, compared to evolutionary algorithms (EAs), ageing leads to impressive speed-ups on the standard Benchmark Function both when using local and global mutations. Unless the stop at first constructive mutation (FCM) mechanism is applied, we show that hypermutations require exponential expected runtime to optimise any Function with a polynomial number of optima. If instead FCM is used, the expected runtime is at most a linear factor larger than the upper bound achieved for any random local search algorithm using the artificial fitness levels method. Nevertheless, we prove that algorithms using hypermutations can be considerably faster than EAs at escaping local optima. An analysis of the complete Opt-IA reveals that it is efficient on the previously considered Functions and highlights problems where the use of the full algorithm is crucial. We complete the picture by presenting a class of Functions for which Opt-IA fails with overwhelming probability while standard EAs are efficient.
-
on the runtime analysis of the opt ia artificial immune system
Genetic and Evolutionary Computation Conference, 2017Co-Authors: Dogan Corus, Pietro S Oliveto, Donya YazdaniAbstract:We present a time complexity analysis of the Opt-IA artificial immune system (AIS). We first highlight the power and limitations of its distinguishing operators (i.e., hypermutations with mutation potential and ageing) by analysing them in isolation. Recent work has shown that ageing combined with local mutations can help escape local optima on a dynamic optimisation Benchmark Function. We generalise this result by rigorously proving that ageing leads to considerable speed-ups (compared to evolutionary algorithms (EAs)) on the standard C liff Benchmark Function both when using local and global mutations. Unless the stop at first constructive mutation (FCM) mechanism is applied, we show that hypermutations require exponential expected runtime to optimise any Function with a polynomial number of optima. If instead FCM is used, the expected runtime is at most a linear factor larger than the upper bound achieved for any random local search algorithm using the artificial fitness levels method. Nevertheless, we prove that algorithms using hypermutations can be considerably faster than EAs at escaping local optima. An analysis of the complete Opt-IA reveals that it is efficient on the previously considered Functions and highlights problems where the use of the full algorithm is crucial.
Chaohua Dai - One of the best experts on this subject based on the ideXlab platform.
-
Seeker optimization algorithm for global optimization: A case study on optimal modelling of proton exchange membrane fuel cell (PEMFC)
International Journal of Electrical Power & Energy Systems, 2011Co-Authors: Chaohua Dai, Weirong Chen, Zhan-li Cheng, Zhi-ling Jiang, Junbo JiaAbstract:In order to optimize the proton exchange membrane fuel cell (PEMFC) model parameters, a novel approach based on seeker optimization algorithm (SOA) is proposed. The SOA is based on the concept of simulating human searching behaviors, where the choice of search direction is based on the empirical gradient by evaluating the response to the position changes and the decision of step length is based on uncertainty reasoning by using a simple Fuzzy rule. In this study, after evaluated on Benchmark Function optimization, the SOA is applied to optimal modelling of the PEMFC by using a fuel cell test system in Fuel Cell Application Centre (FAC) at the Temasek Polytechnic, and compared with several state-of-the-art versions of differential evolution (DE) and particle swarm optimization (PSO) algorithms. The simulation results show that the proposed approach is superior to other compared algorithms, and the PEMFC model with optimized parameters by SOA fitted experimental data well. Hence, SOA is an effective and reliable technique for optimizing the parameters of PEMFC model, and can be helpful for system analysis, optimization design and real-time control of the PEMFCs.
-
Seeker Optimization Algorithm for Optimal Reactive Power Dispatch
IEEE Transactions on Power Systems, 2009Co-Authors: Chaohua Dai, Weirong Chen, Yunfang Zhu, Xuexia ZhangAbstract:Optimal reactive power dispatch problem in power systems has thrown a growing influence on secure and economical operation of power systems. However, this issue is well known as a nonlinear, multimodal and mixed-variable problem. In the last decades, computation intelligence-based techniques, such as genetic algorithms (GAs), differential evolution (DE) algorithms and particle swarm optimization (PSO) algorithms, etc., have often been used for this aim. In this work, a seeker optimization algorithm (SOA)-based reactive power dispatch method is proposed. The SOA is based on the concept of simulating the act of human searching, where the search direction is based on the empirical gradient by evaluating the response to the position changes and the step length is based on uncertainty reasoning by using a simple Fuzzy rule. In this study, the algorithm's performance is evaluated on Benchmark Function optimization. Then, the SOA is applied to optimal reactive power dispatch on standard IEEE 57- and 118-bus power systems, and compared with conventional nonlinear programming method, two versions of GAs, three versions of DE algorithms and four versions of PSO algorithms. The simulation results show that the proposed approach is superior to the other listed algorithms and can be efficiently used for optimal reactive power dispatch.