The Experts below are selected from a list of 232431 Experts worldwide ranked by ideXlab platform
Heike Trautmann - One of the best experts on this subject based on the ideXlab platform.
-
automated Algorithm Selection on continuous black box problems by combining exploratory landscape analysis and machine learning
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Heike TrautmannAbstract:In this article, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in a...
-
automated Algorithm Selection on continuous black box problems by combining exploratory landscape analysis and machine learning
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Heike TrautmannAbstract:In this article, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in automatically constructing Algorithm Selection models in continuous black-box optimization problems. Focusing on Algorithm performance results of the COCO platform of several years, we construct a representative set of high-performing complementary solvers and present an Algorithm Selection model that, compared to the portfolio's single best solver, on average requires less than half of the resources for solving a given problem. Therefore, there is a huge gain in efficiency compared to classical ensemble methods combined with an increased insight into problem characteristics and Algorithm properties by using informative features. The model acts on the assumption that the function set of the Black-Box Optimization Benchmark is representative enough for practical applications. The model allows for selecting the best suited optimization Algorithm within the considered set for unseen problems prior to the optimization itself based on a small sample of function evaluations. Note that such a sample can even be reused for the initial population of an evolutionary (optimization) Algorithm so that even the feature costs become negligible.
-
automated Algorithm Selection survey and perspectives
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Holger H. Hoos, Frank Neumann, Heike TrautmannAbstract:It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different Algorithms. This is particularly pronou...
-
Automated Algorithm Selection: Survey and Perspectives
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Holger H. Hoos, Frank Neumann, Heike TrautmannAbstract:It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different Algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single Algorithm defines the state of the art; instead, there is a set of Algorithms with complementary strengths. This performance complementarity can be exploited in various ways, one of which is based on the idea of selecting, from a set of given Algorithms, for each problem instance to be solved the one expected to perform best. The task of automatically selecting an Algorithm from a given set is known as the per-instance Algorithm Selection problem and has been intensely studied over the past 15 years, leading to major improvements in the state of the art in solving a growing number of discrete combinatorial problems, including propositional satisfiability and AI planning. Per-instance Algorithm Selection also shows much promise for boosting performance in solving continuous and mixed discrete/continuous optimisation problems. This survey provides an overview of research in automated Algorithm Selection, ranging from early and seminal works to recent and promising application areas. Different from earlier work, it covers applications to discrete and continuous problems, and discusses Algorithm Selection in context with conceptually related approaches, such as Algorithm configuration, scheduling, or portfolio Selection. Since informative and cheaply computable problem instance features provide the basis for effective per-instance Algorithm Selection systems, we also provide an overview of such features for discrete and continuous problems. Finally, we provide perspectives on future work in the area and discuss a number of open research challenges.
-
automated Algorithm Selection on continuous black box problems by combining exploratory landscape analysis and machine learning
arXiv: Machine Learning, 2017Co-Authors: Pascal Kerschke, Heike TrautmannAbstract:In this paper, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in automatically constructing Algorithm Selection models in continuous black-box optimization problems. Focussing on Algorithm performance results of the COCO platform of several years, we construct a representative set of high-performing complementary solvers and present an Algorithm Selection model that - compared to the portfolio's single best solver - on average requires less than half of the resources for solving a given problem. Therefore, there is a huge gain in efficiency compared to classical ensemble methods combined with an increased insight into problem characteristics and Algorithm properties by using informative features. Acting on the assumption that the function set of the Black-Box Optimization Benchmark is representative enough for practical applications the model allows for selecting the best suited optimization Algorithm within the considered set for unseen problems prior to the optimization itself based on a small sample of function evaluations. Note that such a sample can even be reused for the initial population of an evolutionary (optimization) Algorithm so that even the feature costs become negligible.
Pascal Kerschke - One of the best experts on this subject based on the ideXlab platform.
-
Making a case for (Hyper-)parameter tuning as benchmark problems
2019Co-Authors: Carola Doerr, Johann Dréo, Pascal KerschkeAbstract:One of the biggest challenges in evolutionary computation concerns the Selection and configuration of a best-suitable heuristic for a given problem. While in the past both of these problems have primarily been addressed by building on experts' experience, the last decade has witnessed a significant shift towards automated decision making, which capitalizes on techniques proposed in the machine learning literature. A key success factor in automated Algorithm Selection and configuration are good training sets, whose performance data can be leveraged to build accurate performance prediction models. With the long-term goal to build landscape-aware parameter control mechanisms for iterative optimization heuristics, we consider in this discussion paper the question how well the 24 functions from the BBOB test bed cover the characteristics of (hyper-)parameter tuning problems. To this end, we perform a preliminary landscape analysis of two hyper-parameter Selection problems, and compare their feature values with those of the BBOB functions. While we do see a good fit for one of the tuning problems, our findings also indicate that some parameter tuning problems might not be very well represented by the BBOB functions. This raises the question if one can nevertheless deduce reliable performance-prediction models for hyper-parameter tuning problems from the BBOB test bed, or whether for this specific target the BBOB benchmark should be adjusted, by adding or replacing some of its functions. Independently of the aspect of training automated Algorithm Selection and configuration techniques, hyper-parameter tuning problems offer a plethora of problems which might be worthwhile to study in the context of benchmarking iterative optimization heuristics.
-
automated Algorithm Selection on continuous black box problems by combining exploratory landscape analysis and machine learning
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Heike TrautmannAbstract:In this article, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in a...
-
automated Algorithm Selection on continuous black box problems by combining exploratory landscape analysis and machine learning
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Heike TrautmannAbstract:In this article, we build upon previous work on designing informative and efficient Exploratory Landscape Analysis features for characterizing problems' landscapes and show their effectiveness in automatically constructing Algorithm Selection models in continuous black-box optimization problems. Focusing on Algorithm performance results of the COCO platform of several years, we construct a representative set of high-performing complementary solvers and present an Algorithm Selection model that, compared to the portfolio's single best solver, on average requires less than half of the resources for solving a given problem. Therefore, there is a huge gain in efficiency compared to classical ensemble methods combined with an increased insight into problem characteristics and Algorithm properties by using informative features. The model acts on the assumption that the function set of the Black-Box Optimization Benchmark is representative enough for practical applications. The model allows for selecting the best suited optimization Algorithm within the considered set for unseen problems prior to the optimization itself based on a small sample of function evaluations. Note that such a sample can even be reused for the initial population of an evolutionary (optimization) Algorithm so that even the feature costs become negligible.
-
automated Algorithm Selection survey and perspectives
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Holger H. Hoos, Frank Neumann, Heike TrautmannAbstract:It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different Algorithms. This is particularly pronou...
-
Automated Algorithm Selection: Survey and Perspectives
Evolutionary Computation, 2019Co-Authors: Pascal Kerschke, Holger H. Hoos, Frank Neumann, Heike TrautmannAbstract:It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different Algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single Algorithm defines the state of the art; instead, there is a set of Algorithms with complementary strengths. This performance complementarity can be exploited in various ways, one of which is based on the idea of selecting, from a set of given Algorithms, for each problem instance to be solved the one expected to perform best. The task of automatically selecting an Algorithm from a given set is known as the per-instance Algorithm Selection problem and has been intensely studied over the past 15 years, leading to major improvements in the state of the art in solving a growing number of discrete combinatorial problems, including propositional satisfiability and AI planning. Per-instance Algorithm Selection also shows much promise for boosting performance in solving continuous and mixed discrete/continuous optimisation problems. This survey provides an overview of research in automated Algorithm Selection, ranging from early and seminal works to recent and promising application areas. Different from earlier work, it covers applications to discrete and continuous problems, and discusses Algorithm Selection in context with conceptually related approaches, such as Algorithm configuration, scheduling, or portfolio Selection. Since informative and cheaply computable problem instance features provide the basis for effective per-instance Algorithm Selection systems, we also provide an overview of such features for discrete and continuous problems. Finally, we provide perspectives on future work in the area and discuss a number of open research challenges.
Ellen Vitercik - One of the best experts on this subject based on the ideXlab platform.
-
dispersion for data driven Algorithm design online learning and private optimization
Foundations of Computer Science, 2018Co-Authors: Mariaflorina Balcan, Travis Dick, Ellen VitercikAbstract:A crucial problem in modern data science is data-driven Algorithm design, where the goal is to choose the best Algorithm, or Algorithm parameters, for a specific application domain. In practice, we often optimize over a parametric Algorithm family, searching for parameters with high performance on a collection of typical problem instances. While effective in practice, these procedures generally have not come with provable guarantees. A recent line of work initiated by a seminal paper of Gupta and Roughgarden (2017) analyzes application-specific Algorithm Selection from a theoretical perspective. We progress this research direction in several important settings. We provide upper and lower bounds on regret for Algorithm Selection in online settings, where problems arrive sequentially and we must choose parameters online. We also consider differentially private Algorithm Selection, where the goal is to find good parameters for a set of problems without divulging too much sensitive information contained therein. We analyze several important parameterized families of Algorithms, including SDP-rounding schemes for problems formulated as integer quadratic programs as well as greedy techniques for several canonical subset Selection problems. The cost function that measures an Algorithm's performance is often a volatile piecewise Lipschitz function of its parameters, since a small change to the parameters can lead to a cascade of different decisions made by the Algorithm. We present general techniques for optimizing the sum or average of piecewise Lipschitz functions when the underlying functions satisfy a sufficient and general condition called dispersion. Intuitively, a set of piecewise Lipschitz functions is dispersed if no small region contains many of the functions' discontinuities. Using dispersion, we improve over the best-known online learning regret bounds for a variety problems, prove regret bounds for problems not previously studied, and provide matching regret lower bounds. In the private optimization setting, we show how to optimize performance while preserving privacy for several important problems, providing matching upper and lower bounds on performance loss due to privacy preservation. Though Algorithm Selection is our primary motivation, we believe the notion of dispersion may be of independent interest. Therefore, we present our results for the more general problem of optimizing piecewise Lipschitz functions. Finally, we uncover dispersion in domains beyond Algorithm Selection, namely, auction design and pricing, providing online and privacy guarantees for these problems as well.
-
dispersion for data driven Algorithm design online learning and private optimization
arXiv: Learning, 2017Co-Authors: Mariaflorina Balcan, Travis Dick, Ellen VitercikAbstract:Data-driven Algorithm design, that is, choosing the best Algorithm for a specific application, is a crucial problem in modern data science. Practitioners often optimize over a parameterized Algorithm family, tuning parameters based on problems from their domain. These procedures have historically come with no guarantees, though a recent line of work studies Algorithm Selection from a theoretical perspective. We advance the foundations of this field in several directions: we analyze online Algorithm Selection, where problems arrive one-by-one and the goal is to minimize regret, and private Algorithm Selection, where the goal is to find good parameters over a set of problems without revealing sensitive information contained therein. We study important Algorithm families, including SDP-rounding schemes for problems formulated as integer quadratic programs, and greedy techniques for canonical subset Selection problems. In these cases, the Algorithm's performance is a volatile and piecewise Lipschitz function of its parameters, since tweaking the parameters can completely change the Algorithm's behavior. We give a sufficient and general condition, dispersion, defining a family of piecewise Lipschitz functions that can be optimized online and privately, which includes the functions measuring the performance of the Algorithms we study. Intuitively, a set of piecewise Lipschitz functions is dispersed if no small region contains many of the functions' discontinuities. We present general techniques for online and private optimization of the sum of dispersed piecewise Lipschitz functions. We improve over the best-known regret bounds for a variety of problems, prove regret bounds for problems not previously studied, and give matching lower bounds. We also give matching upper and lower bounds on the utility loss due to privacy. Moreover, we uncover dispersion in auction design and pricing problems.
A Diazsanchez - One of the best experts on this subject based on the ideXlab platform.
-
euclid preparation iii galaxy cluster detection in the wide photometric survey performance and Algorithm Selection
Astronomy and Astrophysics, 2019Co-Authors: R Adam, M Vannier, S Maurogordato, A Biviano, C Adami, B Ascaso, Fabio Bellagamba, C Benoist, A Cappi, A DiazsanchezAbstract:Galaxy cluster counts in bins of mass and redshift have been shown to be a competitive probe to test cosmological models. This method requires an efficient blind detection of clusters from surveys with a well-known Selection function and robust mass estimates, which is particularly challenging at high redshift. The Euclid wide survey will cover 15 000 deg2 of the sky, avoiding contamination by light from our Galaxy and our solar system in the optical and near-infrared bands, down to magnitude 24 in the H-band. The resulting data will make it possible to detect a large number of galaxy clusters spanning a wide-range of masses up to redshift ∼2 and possibly higher. This paper presents the final results of the Euclid Cluster Finder Challenge (CFC), fourth in a series of similar challenges. The objective of these challenges was to select the cluster detection Algorithms that best meet the requirements of the Euclid mission. The final CFC included six independent detection Algorithms, based on different techniques, such as photometric redshift tomography, optimal filtering, hierarchical approach, wavelet and friend-of-friends Algorithms. These Algorithms were blindly applied to a mock galaxy catalog with representative Euclid-like properties. The relative performance of the Algorithms was assessed by matching the resulting detections to known clusters in the simulations down to masses of M₂₀₀ ∼ 10^(13.25) M⊙. Several matching procedures were tested, thus making it possible to estimate the associated systematic effects on completeness to 80% completeness for a mean purity of 80% down to masses of 10¹⁴ M⊙ and up to redshift z = 2. Based on these results, two Algorithms were selected to be implemented in the Euclid pipeline, the Adaptive Matched Identifier of Clustered Objects (AMICO) code, based on matched filtering, and the PZWav code, based on an adaptive wavelet approach.
Hugh Mcnamara - One of the best experts on this subject based on the ideXlab platform.
-
Algorithm Selection for error resilience in scientific computing
Pacific Rim International Symposium on Dependable Computing, 2014Co-Authors: Joseph Callenessloan, Hugh McnamaraAbstract:With process scaling and the adoption of post-cmos technologies, reliability and power are becoming a significant concern for future computing systems, especially highly parallel systems. Previous approaches have investigated augmenting applications with additional logic to detect and correct errors efficiently. In this research, we investigate the impact of different Algorithmic designs on error resilience and propose an approach for Algorithm Selection for a class of equations, i.e. partial differential equations (PDEs), that are at the core of many scientific computing applications, which drive HPC systems. Many different schemes have been devised for the approximation of PDE systems, each with different accuracy, stability, and performance properties. In this research, there are two primary questions that we address: (1) Does numerical stability translate to error resilience? and (2) How do we design schemes to improve error resilience? If an Algorithm's error resilience is correlated with its numerical stability properties, this may allow us to design more resilient applications by leveraging well established information on numerical stability. Even with a clear translation of numerical stability to error resilience properties, the question of designing these Algorithms still remains however, due to the variety of implementations, schemes, and largely input specific nature of the design. In this research, we propose one approach for automated design using machine-learning. We observe that intelligent Selection of the Algorithm or a given problem, improves robustness by 20%-50%, on average, over the traditional Selection of Algorithms, without the addition of any other detection/correction logic.
-
Algorithm Selection for error resilience in scientific computing
Pacific Rim International Symposium on Dependable Computing, 2014Co-Authors: Joseph Callenessloan, Hugh McnamaraAbstract:With process scaling and the adoption of post-cmos technologies, reliability and power are becoming a significant concern for future computing systems, especially highly parallel systems. Previous approaches have investigated augmenting applications with additional logic to detect and correct errors efficiently. In this research, we investigate the impact of different Algorithmic designs on error resilience and propose an approach for Algorithm Selection for a class of equations, i.e. partial differential equations (PDEs), that are at the core of many scientific computing applications, which drive HPC systems. Many different schemes have been devised for the approximation of PDE systems, each with different accuracy, stability, and performance properties. In this research, there are two primary questions that we address: (1) Does numerical stability translate to error resilience? and (2) How do we design schemes to improve error resilience? If an Algorithm's error resilience is correlated with its numerical stability properties, this may allow us to design more resilient applications by leveraging well established information on numerical stability. Even with a clear translation of numerical stability to error resilience properties, the question of designing these Algorithms still remains however, due to the variety of implementations, schemes, and largely input specific nature of the design. In this research, we propose one approach for automated design using machine-learning. We observe that intelligent Selection of the Algorithm or a given problem, improves robustness by 20%-50%, on average, over the traditional Selection of Algorithms, without the addition of any other detection/correction logic.