The Experts below are selected from a list of 273 Experts worldwide ranked by ideXlab platform
Jean-michel Marin - One of the best experts on this subject based on the ideXlab platform.
-
Bounding rare event probabilities in computer experiments
Computational Statistics and Data Analysis, 2014Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:Bounding probabilities of rare events in the context of computer experiments is an important concern in reliability studies. These rare events depend on the output of a physical model with random input variables. Since the model is only known through an expensive black Box Function, standard efficient Monte Carlo methods designed for rare events cannot be used. That is why a strategy based on importance sampling methods is proposed. This strategy relies on Kriging meta-modeling and manages to achieve sharp upper confidence bounds on the rare events probabilities. The variability due to the Kriging meta-modeling step is properly taken into account. The proposed methodology is applied to an artificial example and compared with more standard Bayesian bounds. Eventually, a challenging real case is analyzed. It consists in finding an upper bound for the probability that the trajectory of an airborne load will collide with the aircraft that released it.
-
Maximin Design on non-hypercube domain and Kernel Interpolation
Statistics and Computing, 2012Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-Box Function to be approximated, an exploratory design has to be used. In this context, two dispersion criteria are usually considered: the MINIMAX and the MAXIMIN ones. In the case of a hypercube domain, a standard strategy consists of taking the MAXIMIN design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-Box Function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the MAXIMIN criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine MAXIMIN designs in any bounded connected domain. We prove the convergence of the different schemes. Finally, the methodology is applied on a challenging real example where the black-blox Function describes the behaviour of an aircraft engine.
-
Estimation of rare events probabilities in computer experiments
2011Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:We are interested in estimating probabilities of rare events in the context of computer experiments. These rare events depend on the output of a physical model with random input variables. Since the model is only known through an expensive black Box Function, standard efficient Monte Carlo estimates of rare events probabilities can not be used. We then propose two strategies to deal with this difficulty: a Bayesian estimate and an importance sampling method. Both proposals rely on Kriging metamodeling and are able to achieve sharp upper confidence bounds on the rare events probabilities. The variability due to the Kriging metamodeling step is properly taking into account. The proposed methodologies are applied to a toy example and a real case study which consists of finding an upper bound of the probability that the trajectory of an airborne load collides the aircraft that has released it.
-
Maximin design on non hypercube domain and kernel interpolation
arXiv: Computation, 2010Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-Box Function to be approximated, an exploratory design have to be used. In this context, two dispersion criteria are usually considered: the minimax and the maximin ones. In the case of a hypercube domain, a standard strategy consists of taking the maximin design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-Box Function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the maximin criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine maximin designs in any bounded connected domain. We prove the convergence of the different schemes.
Yves Auffray - One of the best experts on this subject based on the ideXlab platform.
-
Bounding rare event probabilities in computer experiments
Computational Statistics and Data Analysis, 2014Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:Bounding probabilities of rare events in the context of computer experiments is an important concern in reliability studies. These rare events depend on the output of a physical model with random input variables. Since the model is only known through an expensive black Box Function, standard efficient Monte Carlo methods designed for rare events cannot be used. That is why a strategy based on importance sampling methods is proposed. This strategy relies on Kriging meta-modeling and manages to achieve sharp upper confidence bounds on the rare events probabilities. The variability due to the Kriging meta-modeling step is properly taken into account. The proposed methodology is applied to an artificial example and compared with more standard Bayesian bounds. Eventually, a challenging real case is analyzed. It consists in finding an upper bound for the probability that the trajectory of an airborne load will collide with the aircraft that released it.
-
Maximin Design on non-hypercube domain and Kernel Interpolation
Statistics and Computing, 2012Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-Box Function to be approximated, an exploratory design has to be used. In this context, two dispersion criteria are usually considered: the MINIMAX and the MAXIMIN ones. In the case of a hypercube domain, a standard strategy consists of taking the MAXIMIN design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-Box Function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the MAXIMIN criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine MAXIMIN designs in any bounded connected domain. We prove the convergence of the different schemes. Finally, the methodology is applied on a challenging real example where the black-blox Function describes the behaviour of an aircraft engine.
-
Estimation of rare events probabilities in computer experiments
2011Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:We are interested in estimating probabilities of rare events in the context of computer experiments. These rare events depend on the output of a physical model with random input variables. Since the model is only known through an expensive black Box Function, standard efficient Monte Carlo estimates of rare events probabilities can not be used. We then propose two strategies to deal with this difficulty: a Bayesian estimate and an importance sampling method. Both proposals rely on Kriging metamodeling and are able to achieve sharp upper confidence bounds on the rare events probabilities. The variability due to the Kriging metamodeling step is properly taking into account. The proposed methodologies are applied to a toy example and a real case study which consists of finding an upper bound of the probability that the trajectory of an airborne load collides the aircraft that has released it.
-
Maximin design on non hypercube domain and kernel interpolation
arXiv: Computation, 2010Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-Box Function to be approximated, an exploratory design have to be used. In this context, two dispersion criteria are usually considered: the minimax and the maximin ones. In the case of a hypercube domain, a standard strategy consists of taking the maximin design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-Box Function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the maximin criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine maximin designs in any bounded connected domain. We prove the convergence of the different schemes.
Rodolphe Jenatton - One of the best experts on this subject based on the ideXlab platform.
-
Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
arXiv: Machine Learning, 2019Co-Authors: Valerio Perrone, Matthias Seeger, Cédric Archambeau, Huibin Shen, Rodolphe JenattonAbstract:Bayesian optimization (BO) is a successful methodology to optimize black-Box Functions that are expensive to evaluate. While traditional methods optimize each black-Box Function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-Box Functions. In this work, we introduce a method to automatically design the BO search space by relying on evaluations of previous black-Box Functions. We depart from the common practice of defining a set of arbitrary search ranges a priori by considering search space geometries that are learned from historical data. This simple, yet effective strategy can be used to endow many existing BO methods with transfer learning properties. Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-Box optimization problems. In particular, the proposed approach combined with random search results in a parameter-free, easy-to-implement, robust hyperparameter optimization strategy. We hope it will constitute a natural baseline for further research attempting to warm-start BO.
-
NeurIPS - Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
2019Co-Authors: Valerio Perrone, Matthias Seeger, Cédric Archambeau, Huibin Shen, Rodolphe JenattonAbstract:Bayesian optimization (BO) is a successful methodology to optimize black-Box Functions that are expensive to evaluate. While traditional methods optimize each black-Box Function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-Box Functions. In this work, we introduce a method to automatically design the BO search space by relying on evaluations of previous black-Box Functions. We depart from the common practice of defining a set of arbitrary search ranges a priori by considering search space geometries that are learnt from historical data. This simple, yet effective strategy can be used to endow many existing BO methods with transfer learning properties. Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-Box optimization problems. In particular, the proposed approach combined with random search results in a parameter-free, easy-to-implement, robust hyperparameter optimization strategy. We hope it will constitute a natural baseline for further research attempting to warm-start BO.
-
NeurIPS - Scalable Hyperparameter Transfer Learning
2018Co-Authors: Valerio Perrone, Rodolphe Jenatton, Matthias Seeger, Cédric ArchambeauAbstract:Bayesian optimization (BO) is a model-based approach for gradient-free black-Box Function optimization, such as hyperparameter optimization. Typically, BO relies on conventional Gaussian process (GP) regression, whose algorithmic complexity is cubic in the number of evaluations. As a result, GP-based BO cannot leverage large numbers of past Function evaluations, for example, to warm-start related BO runs. We propose a multi-task adaptive Bayesian linear regression model for transfer learning in BO, whose complexity is linear in the Function evaluations: one Bayesian linear regression model is associated to each black-Box Function optimization problem (or task), while transfer learning is achieved by coupling the models through a shared deep neural net. Experiments show that the neural net learns a representation suitable for warm-starting the black-Box optimization problems and that BO runs can be accelerated when the target black-Box Function (e.g., validation loss) is learned together with other related signals (e.g., training loss). The proposed method was found to be at least one order of magnitude faster that methods recently published in the literature.
Pierre Barbillon - One of the best experts on this subject based on the ideXlab platform.
-
Bounding rare event probabilities in computer experiments
Computational Statistics and Data Analysis, 2014Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:Bounding probabilities of rare events in the context of computer experiments is an important concern in reliability studies. These rare events depend on the output of a physical model with random input variables. Since the model is only known through an expensive black Box Function, standard efficient Monte Carlo methods designed for rare events cannot be used. That is why a strategy based on importance sampling methods is proposed. This strategy relies on Kriging meta-modeling and manages to achieve sharp upper confidence bounds on the rare events probabilities. The variability due to the Kriging meta-modeling step is properly taken into account. The proposed methodology is applied to an artificial example and compared with more standard Bayesian bounds. Eventually, a challenging real case is analyzed. It consists in finding an upper bound for the probability that the trajectory of an airborne load will collide with the aircraft that released it.
-
Maximin Design on non-hypercube domain and Kernel Interpolation
Statistics and Computing, 2012Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-Box Function to be approximated, an exploratory design has to be used. In this context, two dispersion criteria are usually considered: the MINIMAX and the MAXIMIN ones. In the case of a hypercube domain, a standard strategy consists of taking the MAXIMIN design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-Box Function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the MAXIMIN criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine MAXIMIN designs in any bounded connected domain. We prove the convergence of the different schemes. Finally, the methodology is applied on a challenging real example where the black-blox Function describes the behaviour of an aircraft engine.
-
Estimation of rare events probabilities in computer experiments
2011Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:We are interested in estimating probabilities of rare events in the context of computer experiments. These rare events depend on the output of a physical model with random input variables. Since the model is only known through an expensive black Box Function, standard efficient Monte Carlo estimates of rare events probabilities can not be used. We then propose two strategies to deal with this difficulty: a Bayesian estimate and an importance sampling method. Both proposals rely on Kriging metamodeling and are able to achieve sharp upper confidence bounds on the rare events probabilities. The variability due to the Kriging metamodeling step is properly taking into account. The proposed methodologies are applied to a toy example and a real case study which consists of finding an upper bound of the probability that the trajectory of an airborne load collides the aircraft that has released it.
-
Maximin design on non hypercube domain and kernel interpolation
arXiv: Computation, 2010Co-Authors: Yves Auffray, Pierre Barbillon, Jean-michel MarinAbstract:In the paradigm of computer experiments, the choice of an experimental design is an important issue. When no information is available about the black-Box Function to be approximated, an exploratory design have to be used. In this context, two dispersion criteria are usually considered: the minimax and the maximin ones. In the case of a hypercube domain, a standard strategy consists of taking the maximin design within the class of Latin hypercube designs. However, in a non hypercube context, it does not make sense to use the Latin hypercube strategy. Moreover, whatever the design is, the black-Box Function is typically approximated thanks to kernel interpolation. Here, we first provide a theoretical justification to the maximin criterion with respect to kernel interpolations. Then, we propose simulated annealing algorithms to determine maximin designs in any bounded connected domain. We prove the convergence of the different schemes.
Valerio Perrone - One of the best experts on this subject based on the ideXlab platform.
-
Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
arXiv: Machine Learning, 2019Co-Authors: Valerio Perrone, Matthias Seeger, Cédric Archambeau, Huibin Shen, Rodolphe JenattonAbstract:Bayesian optimization (BO) is a successful methodology to optimize black-Box Functions that are expensive to evaluate. While traditional methods optimize each black-Box Function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-Box Functions. In this work, we introduce a method to automatically design the BO search space by relying on evaluations of previous black-Box Functions. We depart from the common practice of defining a set of arbitrary search ranges a priori by considering search space geometries that are learned from historical data. This simple, yet effective strategy can be used to endow many existing BO methods with transfer learning properties. Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-Box optimization problems. In particular, the proposed approach combined with random search results in a parameter-free, easy-to-implement, robust hyperparameter optimization strategy. We hope it will constitute a natural baseline for further research attempting to warm-start BO.
-
NeurIPS - Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning
2019Co-Authors: Valerio Perrone, Matthias Seeger, Cédric Archambeau, Huibin Shen, Rodolphe JenattonAbstract:Bayesian optimization (BO) is a successful methodology to optimize black-Box Functions that are expensive to evaluate. While traditional methods optimize each black-Box Function in isolation, there has been recent interest in speeding up BO by transferring knowledge across multiple related black-Box Functions. In this work, we introduce a method to automatically design the BO search space by relying on evaluations of previous black-Box Functions. We depart from the common practice of defining a set of arbitrary search ranges a priori by considering search space geometries that are learnt from historical data. This simple, yet effective strategy can be used to endow many existing BO methods with transfer learning properties. Despite its simplicity, we show that our approach considerably boosts BO by reducing the size of the search space, thus accelerating the optimization of a variety of black-Box optimization problems. In particular, the proposed approach combined with random search results in a parameter-free, easy-to-implement, robust hyperparameter optimization strategy. We hope it will constitute a natural baseline for further research attempting to warm-start BO.
-
NeurIPS - Scalable Hyperparameter Transfer Learning
2018Co-Authors: Valerio Perrone, Rodolphe Jenatton, Matthias Seeger, Cédric ArchambeauAbstract:Bayesian optimization (BO) is a model-based approach for gradient-free black-Box Function optimization, such as hyperparameter optimization. Typically, BO relies on conventional Gaussian process (GP) regression, whose algorithmic complexity is cubic in the number of evaluations. As a result, GP-based BO cannot leverage large numbers of past Function evaluations, for example, to warm-start related BO runs. We propose a multi-task adaptive Bayesian linear regression model for transfer learning in BO, whose complexity is linear in the Function evaluations: one Bayesian linear regression model is associated to each black-Box Function optimization problem (or task), while transfer learning is achieved by coupling the models through a shared deep neural net. Experiments show that the neural net learns a representation suitable for warm-starting the black-Box optimization problems and that BO runs can be accelerated when the target black-Box Function (e.g., validation loss) is learned together with other related signals (e.g., training loss). The proposed method was found to be at least one order of magnitude faster that methods recently published in the literature.