The Experts below are selected from a list of 103041 Experts worldwide ranked by ideXlab platform
Jack P C Kleijnen - One of the best experts on this subject based on the ideXlab platform.
-
efficient global optimization for black box simulation via sequential intrinsic kriging
Journal of the Operational Research Society, 2018Co-Authors: Ehsan Mehdad, Jack P C KleijnenAbstract:Efficient Global Optimization (EGO) is a popular method that searches sequentially for the global optimum of a simulated system. EGO treats the simulation model as a black-box, and balances local and global searches. In deterministic simulation, EGO uses ordinary Kriging (OK), which is a special case of universal Kriging (UK). In our EGO variant we use intrinsic Kriging (IK), which eliminates the need to estimate the parameters that quantify the trend in UK. In random simulation, EGO uses stochastic Kriging (SK), but we use stochastic IK (SIK). Moreover, in random simulation, EGO needs to select the number of replications per simulated Input Combination, accounting for the heteroscedastic variances of the simulation outputs. A popular selection method uses optimal computer budget allocation (OCBA), which allocates the available total number of replications over simulated Combinations. We derive a new allocation algorithm. We perform several numerical experiments with deterministic simulations and random simulations. These experiments suggest that (1) in deterministic simulations, EGO with IK outperforms classic EGO; (2) in random simulations, EGO with SIK and our allocation rule does not differ significantly from EGO with SK combined with the OCBA allocation rule.
-
Stochastic Intrinsic Kriging for Simulation Metamodelling
SSRN Electronic Journal, 2014Co-Authors: Ehsan Mehdad, Jack P C KleijnenAbstract:Kriging provides metamodels for deterministic and random simulation models. Actually, there are several types of Kriging; the classic type is so-called universal Kriging, which includes ordinary Kriging. These classic types require estimation of the trend in the Input-output data of the underlying simulation model; this estimation deteriorates the Kriging metamodel. We therefore consider so-called intrinsic Kriging originating in geostatistics, and derive intrinsic Kriging for deterministic and random simulations. Moreover, for random simulations we derive experimental designs that specify the number of replications that varies with the Input Combination of the simulation model. To compare the performance of intrinsic Kriging and classic Kriging, we use several numerical experiments with deterministic simulations and random simulations. These experiments show that intrinsic Kriging gives better metamodels, in most experiments.
-
Monotonicity-Preserving Bootstrapped Kriging Metamodels for Expensive Simulations
Research Papers in Economics, 2013Co-Authors: Jack P C Kleijnen, W.c.m. Van BeersAbstract:Kriging (Gaussian process, spatial correlation) metamodels approximate the Input/Output (I/O) functions implied by the underlying simulation models; such metamodels serve sensitivity analysis and optimization, especially for computationally expensive simulations. In practice, simulation analysts often know that the I/O function is monotonic. To obtain a Kriging metamodel that preserves this known shape, this article uses bootstrapping (or resampling). Parametric bootstrapping assuming normality may be used in deterministic simulation, but this article focuses on stochastic simulation (including discrete-event simulation) using distribution-free bootstrapping. In stochastic simulation, the analysts should simulate each Input Combination several times to obtain a more reliable average output per Input Combination. Nevertheless, this average still shows sampling variation, so the Kriging metamodel does not need to interpolate the average outputs. Bootstrapping provides a simple method for computing a noninterpolating Kriging model. This method may use standard Kriging software, such as the free Matlab toolbox called DACE. The method is illustrated through the M/M/1 simulation model with as outputs either the estimated mean or the estimated 90% quantile; both outputs are monotonic functions of the traffic rate, and have nonnormal distributions. The empirical results demonstrate that monotonicity-preserving bootstrapped Kriging may give higher probability of covering the true simulation output, without lengthening the confidence interval.
-
Convex and Monotonic Bootstrapped Kriging
2012Co-Authors: Jack P C Kleijnen, Ehsan Mehdad, W.c.m. Van BeersAbstract:Abstract: Distribution-free bootstrapping of the replicated responses of a given discreteevent simulation model gives bootstrapped Kriging (Gaussian process) metamodels; we require these metamodels to be either convex or monotonic. To illustrate monotonic Kriging, we use an M/M/1 queueing simulation with as output either the mean or the 90% quantile of the transient-state waiting times, and as Input the traffic rate. In this example, monotonic bootstrapped Kriging enables better sensitivity analysis than classic Kriging; i.e., bootstrapping gives lower MSE and confidence intervals with higher coverage and the same length. To illustrate convex Kriging, we start with simulationoptimization of an (s, S) inventory model, but we next switch to a Monte Carlo experiment with a second-order polynomial inspired by this inventory simulation. We could not find truly convex Kriging metamodels, either classic or bootstrapped; nevertheless, our bootstrapped "nearly convex" Kriging does give a confidence interval for the optimal Input Combination.
-
Statistical testing of optimality conditions in multiresponse simulation-based optimization
European Journal of Operational Research, 2009Co-Authors: B.w.m. Bettonvil, Enrique Castillo, Jack P C KleijnenAbstract:This article studies simulation-based optimization with multiple outputs. It assumes that the simulation model has one random objective function and must satisfy given constraints on the other random outputs. It presents a statistical procedure for testing whether a specific Input Combination (proposed by some optimization heuristic) satisfies the Karush–Kuhn–Tucker (KKT) first-order optimality conditions. The article focuses on “expensive” simulations, which have small sample sizes. The article applies the classic t test to check whether the specific Input Combination is feasible, and whether any constraints are binding; next, it applies bootstrapping (resampling) to test the estimated gradients in the KKT conditions. The new methodology is applied to three examples, which gives encouraging empirical results.
Ebru Angün - One of the best experts on this subject based on the ideXlab platform.
-
Response surface methodology's steepest ascent and step size revisited: Correction
European Journal of Operational Research, 2006Co-Authors: Jack P C Kleijnen, Dick Den Hertog, Ebru AngünAbstract:Abstract This Short Communication corrects Table 1 and Fig. 2 , Fig. 3 that were published in a recent article by the same authors, in this journal. The article discussed response surface methodology (RSM), which searches for the Input Combination maximizing the output of a real or simulated system. RSM uses steepest ascent (SA), which is scale-dependent. The article derived scale-independent ‘adapted’ SA (ASA). The two search directions were explored in Monte Carlo experiments. Unfortunately, the canonical and the non-canonical cases were mixed up. This Communication still shows that—in general—ASA gives a better search direction than SA.
-
Response Surface Methodology's Steepest Ascent and Step Size Revisited
European Journal of Operational Research, 2004Co-Authors: Jack P C Kleijnen, Dick Den Hertog, Ebru AngünAbstract:Response Surface Methodology (RSM) searches for the Input Combination maximizing the output of a real system or its simulation.RSM is a heuristic that locally fits first-order polynomials, and estimates the corresponding steepest ascent (SA) paths.However, SA is scale-dependent; and its step size is selected intuitively.To tackle these two problems, this paper derives novel techniques combining mathematical statistics and mathematical programming.Technique 1 called 'adapted' SA (ASA) accounts for the covariances between the components of the estimated local gradient.ASA is scale-independent.The step-size problem is solved tentatively.Technique 2 does follow the SA direction, but with a step size inspired by ASA.Mathematical properties of the two techniques are derived and interpreted; numerical examples illustrate these properties.The search directions of the two techniques are explored in Monte Carlo experiments.These experiments show that - in general - ASA gives a better search direction than SA.
Dick Den Hertog - One of the best experts on this subject based on the ideXlab platform.
-
Response surface methodology's steepest ascent and step size revisited: Correction
European Journal of Operational Research, 2006Co-Authors: Jack P C Kleijnen, Dick Den Hertog, Ebru AngünAbstract:Abstract This Short Communication corrects Table 1 and Fig. 2 , Fig. 3 that were published in a recent article by the same authors, in this journal. The article discussed response surface methodology (RSM), which searches for the Input Combination maximizing the output of a real or simulated system. RSM uses steepest ascent (SA), which is scale-dependent. The article derived scale-independent ‘adapted’ SA (ASA). The two search directions were explored in Monte Carlo experiments. Unfortunately, the canonical and the non-canonical cases were mixed up. This Communication still shows that—in general—ASA gives a better search direction than SA.
-
Response Surface Methodology's Steepest Ascent and Step Size Revisited
European Journal of Operational Research, 2004Co-Authors: Jack P C Kleijnen, Dick Den Hertog, Ebru AngünAbstract:Response Surface Methodology (RSM) searches for the Input Combination maximizing the output of a real system or its simulation.RSM is a heuristic that locally fits first-order polynomials, and estimates the corresponding steepest ascent (SA) paths.However, SA is scale-dependent; and its step size is selected intuitively.To tackle these two problems, this paper derives novel techniques combining mathematical statistics and mathematical programming.Technique 1 called 'adapted' SA (ASA) accounts for the covariances between the components of the estimated local gradient.ASA is scale-independent.The step-size problem is solved tentatively.Technique 2 does follow the SA direction, but with a step size inspired by ASA.Mathematical properties of the two techniques are derived and interpreted; numerical examples illustrate these properties.The search directions of the two techniques are explored in Monte Carlo experiments.These experiments show that - in general - ASA gives a better search direction than SA.
-
Stochastics and Statistics Response surface methodologys steepest ascent and step size revisited
2004Co-Authors: Jack P C Kleijnen, Dick Den Hertog, Ebru AngAbstract:Response surface methodology (RSM) searches for the Input Combination maximizing the output of a real system or its simulation. RSM is a heuristic that locally fits first-order polynomials, and estimates the corresponding steepest ascent (SA) paths. However, SA is scale-dependent; and its step size is selected intuitively. To tackle these two problems, this paper derives novel techniques combining mathematical statistics and mathematical programming. Technique 1, called adapted SA (ASA), accounts for the covariances between the components of the estimated local gradient. ASA is scale-independent. The step-size problem is solved tentatively. Technique 2 does follow the SA direction, but with a step size inspired by ASA. Mathematical properties of the two techniques are derived and interpreted; numerical examples illustrate these properties. The search directions of the two techniques are explored in Monte Carlo experiments. These experiments show that––in general––ASA gives a better search direction than SA. 2003 Elsevier B.V. All rights reserved.
Zaher Mundher Yaseen - One of the best experts on this subject based on the ideXlab platform.
-
application of novel data mining algorithms in prediction of discharge and end depth in trapezoidal sections
Computers and Electronics in Agriculture, 2020Co-Authors: Payam Khosravinia, Ozgur Kisi, Mohammad Reza Nikpour, Zaher Mundher YaseenAbstract:Abstract Flow measurement in irrigation and drainage networks and water conveyance channels have particular importance. Direct methods of flow measurement are costly, time-consuming and are generally associated with losses of energy in flow. In this study, estimation of discharge and end depth of free overfall flows in trapezoidal channels section were investigated. For this purpose, data-driven techniques including dynamic evolving neural-fuzzy inference system (DENFIS), multivariate adaptive regression spline (MARS) and M5 model tree (M5Tree) were developed. 189 laboratory data experiments, six different scenarios based on geometric variables including side slope (m), bed width (B), bed slope (S0), and hydraulic variables including critical depth (Yc), critical slope (Sc) and end depth (YE) or discharge (Q) were applied. The model’s performance was evaluated thorough several statistical indicators and graphical presentations. The accuracy of all three models were apparent in estimation of the discharge and the end depth for most of the scenarios. The results showed that the DENFIS model for the Input Combination of all variables (Yc, YE, B, S0, m, Sc) with the maximum values of R2 and Nash-Sutcliffe efficiency coefficient (NSE) that were equal to 0.976 and 0.975, respectively, and the minimum values of RMSE, MAE, PBIAS and RSR, that were equal to 0.0015, 0.0989, −1.5906, and 0.1574, respectively, showed the highest estimation accuracy. Regarding the end depth estimation, DENFIS model for the Input Combination including the variables Yc, Q, Sc, m, B with the highest values of R2 and NSE equal to 0.993 and 0.992 respectively, and the lowest values of RMSE, MAE, PBIAS and RSR equal to 0.0028, 0.1628, 0.7383 and 0.0883, respectively, had a better performance compared to other MARS and M5Tree. The results of this study suggest DENFIS as a suitable and powerful model for estimation of discharge in irrigation and drainage networks.
-
The Feasibility of Integrative Radial Basis M5Tree Predictive Model for River Suspended Sediment Load Simulation
Water Resources Management, 2019Co-Authors: Hai Tao, Behrooz Keshtegar, Zaher Mundher YaseenAbstract:Accurate suspended sediment transport prediction is highly significant for multiple river engineering sustainability. Conceptually evidenced, sediment load transport is highly stochastic, spatial distributed and redundant pattern due to the incorporation of various hydrological and morphological variables such as river flow discharge and sediment physical properties. The motivation of this study is to explore the feasibility of newly intelligent model called Radial basis M5 model tree (RM5Tree) for suspended sediment load (St) prediction for daily scale information at Trenton hydrological station, Delaware River. Numerous Input Combination attributes are formulated based on the preceding information of sediment and river flow discharge. The prediction accuracy “based statistical and graphical visualizations” of the proposed model validated against numerous well-established predictive models including response surface method (RSM), artificial neural network (ANN) and classical M5Tree based models. The investigated Input Combinations behaved differently from one case to another. The optimum Input Combination attributes are included two months lead times of sediment and discharge information to predict one step ahead St. The attained results of the proposed RM5Tree model exhibited a remarkable prediction accuracy with minimal values of root mean square error (RMSE≈2091 ton/day) and coefficient of determination (R2≈0.86). This presenting a percentage of enhancement in the prediction accuracies by (51.6, 53.1 and 26.3) over (RSM, ANN and M5Tree) optimal models over the testing phase.
Hsuan-yu Lin - One of the best experts on this subject based on the ideXlab platform.
-
Development of a support‐vector‐machine‐based model for daily pan evaporation estimation
Hydrological Processes, 2012Co-Authors: Gwo-fong Lin, Hsuan-yu LinAbstract:Evaporation estimation is an important issue in water resources management. In this article, a four-season model with optimal Input Combination is proposed to estimate the daily evaporation. First, the model based on support vector machine (SVM) coupled with an Input determination process is used to determine the optimal Combination of Input variables. Second, a comparison of the SVM-based model with the model based on back-propagation network (BPN) is made to demonstrate the superiority of the SVM-based model. In addition, season data are used to construct the SVM-based four-season model to further improve the daily evaporation estimation. An application is conducted to demonstrate the performance of the proposed model. Results show that the SVM-based model can select the optimal Input Combination with physical mechanism. The SVM-based model is more appropriate than the BPN-based model because of its higher accuracy, robustness and efficiency. Moreover, the improvement due to the use of the four-season model increases from 3.22% to 15.30% for RMSE and from 4.84% to 91.16% for CE, respectively. In conclusion, the SVM-based model coupled with the proposed Input determination process should be used to select Input variables. The proposed four-season SVM-based model with optimal Input Combination is recommended as an alternative to the existing models. The proposed modelling technique is expected to be useful to improve the daily evaporation estimation. Copyright © 2012 John Wiley & Sons, Ltd.