Objective Test

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 915816 Experts worldwide ranked by ideXlab platform

Yusuke Nojima - One of the best experts on this subject based on the ideXlab platform.

  • performance comparison of nsga ii and nsga iii on various many Objective Test problems
    Congress on Evolutionary Computation, 2016
    Co-Authors: Hisao Ishibuchi, Ryo Imada, Yu Setoguchi, Yusuke Nojima
    Abstract:

    Recently NSGA-III has been frequently used for performance comparison of newly proposed evolutionary many-Objective optimization algorithms. That is, NSGA-III has been used as a benchmark algorithm for evolutionary many-Objective optimization. However, unfortunately, its source code is not available from the authors of the NSGA-III paper. This leads to an undesirable situation where a different implementation is used in a different study. Moreover, comparison is usually performed on DTLZ and WFG Test problems. As a result, the performance of NSGA-III on a wide variety of many-Objective Test problems is still unclear whereas it has been frequently used for performance comparison in the literature. In this paper, we evaluate the performance of NSGA-III in comparison with NSGA-II on four totally different types of many-Objective Test problems with 3–10 Objectives: DTLZ1-4 problems, their maximization variants, distance minimization problems, and knapsack problems. We use two different implementations of NSGA-II and NSGA-III. We show through computational experiments that NSGA-III does not always outperform NSGA-II even for ten-Objective problems. That is, their comparison results depend not only on the number of Objectives but also on the type of Test problems. The choice of Test problems has a larger effect on their comparison results than the number of Objectives in our computational experiments. We also demonstrate that totally different results are obtained from different implementations of NSGA-III for some Test problems.

  • Many-Objective and many-variable Test problems for visual examination of multiObjective search
    2013 IEEE Congress on Evolutionary Computation, 2013
    Co-Authors: Hisao Ishibuchi, Masakazu Yamane, Naoya Akedo, Yusuke Nojima
    Abstract:

    In the development of evolutionary multiObjective optimization (EMO) algorithms, it is important to implement a good balancing mechanism between the convergence of solutions towards the Pareto front and their diversity over the Pareto front. When an EMO algorithm is applied to a two-Objective problem, the balance can be easily visualized by showing all solutions at each generation in the two-dimensional Objective space. However, such a visual examination of the multiObjective search is difficult for many-Objective problems with four or more Objectives. The use of many-Objective Test problems with two decision variables has been proposed in some studies to visually examine the search behavior of EMO algorithms. Such Test problems are defined by a number of points in a two-dimensional decision space where the distance minimization from each point is an Objective. Thus the number of Objectives is the same as the number of points. The search behavior of EMO algorithms can be visually examined in the two-dimensional decision space. In this paper, we propose the use of many-Objective Test problems for visual examination of the search behavior in a high-dimensional decision space. More specifically, our m-Objective Test problem with n variables is generated by specifying m points on a plane in an n-dimensional decision space. We examine the behavior of EMO algorithms through computational experiments on such an m-Objective n-variable Test problem. Our experimental results show that the number of variables has a large effect on the search behavior of EMO algorithms with respect to the diversity of solutions.

  • a many Objective Test problem for visually examining diversity maintenance behavior in a decision space
    Genetic and Evolutionary Computation Conference, 2011
    Co-Authors: Hisao Ishibuchi, Naoya Akedo, Yusuke Nojima
    Abstract:

    Recently distance minimization problems in a two-dimensional decision space have been utilized as many-Objective Test problems to visually examine the behavior of evolutionary multi-Objective optimization (EMO) algorithms. Such a Test problem is usually defined by a single polygon where the distance from a solution to each vertex is minimized in the decision space. We can easily generate different Test problems from different polygons. We can also easily generate Test problems with multiple equivalent Pareto optimal regions using multiple polygons of the same shape and the same size. Whereas these Test problems have a number of advantages, they have no clear relevance to real-world situations since they are artificially generated unrealistic Test problems. In this paper, we generate a distance minimization problem from a real-world map. Our Test problem has four Objectives, which are to minimize the distances to the nearest elementary school, junior high school, railway station, and convenience store. Using our Test problem, we examine the behavior of well-known and frequently-used EMO algorithms in terms of their diversity maintenance ability in the two-dimensional decision space.

  • many Objective Test problems to visually examine the behavior of multiObjective evolution in a decision space
    Parallel Problem Solving from Nature, 2010
    Co-Authors: Hisao Ishibuchi, Yasuhiro Hitotsuyanagi, Noritaka Tsukamoto, Yusuke Nojima
    Abstract:

    Many-Objective optimization is a hot issue in the EMO (evolutionary multiObjective optimization) community. Since almost all solutions in the current population are non-dominated with each other in many-Objective EMO algorithms, we may need a different fitness evaluation scheme from the case of two and three Objectives. One difficulty in the design of many-Objective EMO algorithms is that we cannot visually observe the behavior of multiObjective evolution in the Objective space with four or more Objectives. In this paper, we propose the use of many-Objective Test problems in a two- or three-dimensional decision space to visually examine the behavior of multiObjective evolution. Such a visual examination helps us to understand the characteristic features of EMO algorithms for many-Objective optimization. Good understanding of existing EMO algorithms may facilitates their modification and the development of new EMO algorithms for many-Objective optimization.

Mark Harman - One of the best experts on this subject based on the ideXlab platform.

  • empirical evaluation of pareto efficient multi Objective regression Test case prioritisation
    International Symposium on Software Testing and Analysis, 2015
    Co-Authors: Michael G Epitropakis, Shin Yoo, Mark Harman, Edmund K Burke
    Abstract:

    The aim of Test case prioritisation is to determine an ordering of Test cases that maximises the likelihood of early fault revelation. Previous prioritisation techniques have tended to be single Objective, for which the additional greedy algorithm is the current state-of-the-art. Unlike Test suite minimisation, multi Objective Test case prioritisation has not been thoroughly evaluated. This paper presents an extensive empirical study of the effectiveness of multi Objective Test case prioritisation, evaluating it on multiple versions of five widely-used benchmark programs and a much larger real world system of over 1 million lines of code. The paper also presents a lossless coverage compaction algorithm that dramatically scales the performance of all algorithms studied by between 2 and 4 orders of magnitude, making prioritisation practical for even very demanding problems.

  • highly scalable multi Objective Test suite minimisation using graphics cards
    Symposium on Search Based Software Engineering, 2011
    Co-Authors: Shin Yoo, Mark Harman
    Abstract:

    Despite claims of "embarrassing parallelism" for many optimisation algorithms, there has been very little work on exploiting parallelism as a route for SBSE scalability. This is an important oversight because scalability is so often a critical success factor for Software Engineering work. This paper shows how relatively inexpensive General Purpose computing on Graphical Processing Units (GPGPU) can be used to run suitably adapted optimisation algorithms, opening up the possibility of cheap scalability. The paper develops a search based optimisation approach for multi Objective regression Test optimisation, evaluating it on benchmark problems as well as larger real world problems. The results indicate that speed-ups of over 25x are possible using widely available standard GPUs. It is also encouraging that the results reveal a statistically strong correlation between larger problem instances and the degree of speed up achieved. This is the first time that GPGPU has been used for SBSE scalability.

  • Using hybrid algorithm for Pareto efficient multi-Objective Test suite minimisation
    Journal of Systems and Software, 2010
    Co-Authors: Shin Yoo, Mark Harman
    Abstract:

    Test suite minimisation techniques seek to reduce the effort required for regression Testing by selecting a subset of Test suites. In previous work, the problem has been considered as a single-Objective optimisation problem. However, real world regression Testing can be a complex process in which multiple Testing criteria and constraints are involved. This paper presents the concept of Pareto efficiency for the Test suite minimisation problem. The Pareto-efficient approach is inherently capable of dealing with multiple Objectives, providing the decision maker with a group of solutions that are not dominated by each other. The paper illustrates the benefits of Pareto efficient multi-Objective Test suite minimisation with empirical studies of two and three Objective formulations, in which multiple Objectives such as coverage and past fault-detection history are considered. The paper utilises a hybrid, multi-Objective genetic algorithm that combines the efficient approximation of the greedy approach with the capability of population based genetic algorithm to produce higher-quality Pareto fronts.

  • pareto efficient multi Objective Test case selection
    International Symposium on Software Testing and Analysis, 2007
    Co-Authors: Shin Yoo, Mark Harman
    Abstract:

    Previous work has treated Test case selection as a single Objective optimisation problem. This paper introduces the concept of Pareto efficiency to Test case selection. The Pareto efficient approach takes multiple Objectives such as code coverage, past fault-detection history and execution cost, and constructs a group of non-dominating, equivalently optimal Test case subsets. The paper describes the potential bene?ts of Pareto efficient multi-Objective Test case selection, illustrating with empirical studies of two and three Objective formulations.

Hisao Ishibuchi - One of the best experts on this subject based on the ideXlab platform.

  • performance comparison of nsga ii and nsga iii on various many Objective Test problems
    Congress on Evolutionary Computation, 2016
    Co-Authors: Hisao Ishibuchi, Ryo Imada, Yu Setoguchi, Yusuke Nojima
    Abstract:

    Recently NSGA-III has been frequently used for performance comparison of newly proposed evolutionary many-Objective optimization algorithms. That is, NSGA-III has been used as a benchmark algorithm for evolutionary many-Objective optimization. However, unfortunately, its source code is not available from the authors of the NSGA-III paper. This leads to an undesirable situation where a different implementation is used in a different study. Moreover, comparison is usually performed on DTLZ and WFG Test problems. As a result, the performance of NSGA-III on a wide variety of many-Objective Test problems is still unclear whereas it has been frequently used for performance comparison in the literature. In this paper, we evaluate the performance of NSGA-III in comparison with NSGA-II on four totally different types of many-Objective Test problems with 3–10 Objectives: DTLZ1-4 problems, their maximization variants, distance minimization problems, and knapsack problems. We use two different implementations of NSGA-II and NSGA-III. We show through computational experiments that NSGA-III does not always outperform NSGA-II even for ten-Objective problems. That is, their comparison results depend not only on the number of Objectives but also on the type of Test problems. The choice of Test problems has a larger effect on their comparison results than the number of Objectives in our computational experiments. We also demonstrate that totally different results are obtained from different implementations of NSGA-III for some Test problems.

  • Many-Objective and many-variable Test problems for visual examination of multiObjective search
    2013 IEEE Congress on Evolutionary Computation, 2013
    Co-Authors: Hisao Ishibuchi, Masakazu Yamane, Naoya Akedo, Yusuke Nojima
    Abstract:

    In the development of evolutionary multiObjective optimization (EMO) algorithms, it is important to implement a good balancing mechanism between the convergence of solutions towards the Pareto front and their diversity over the Pareto front. When an EMO algorithm is applied to a two-Objective problem, the balance can be easily visualized by showing all solutions at each generation in the two-dimensional Objective space. However, such a visual examination of the multiObjective search is difficult for many-Objective problems with four or more Objectives. The use of many-Objective Test problems with two decision variables has been proposed in some studies to visually examine the search behavior of EMO algorithms. Such Test problems are defined by a number of points in a two-dimensional decision space where the distance minimization from each point is an Objective. Thus the number of Objectives is the same as the number of points. The search behavior of EMO algorithms can be visually examined in the two-dimensional decision space. In this paper, we propose the use of many-Objective Test problems for visual examination of the search behavior in a high-dimensional decision space. More specifically, our m-Objective Test problem with n variables is generated by specifying m points on a plane in an n-dimensional decision space. We examine the behavior of EMO algorithms through computational experiments on such an m-Objective n-variable Test problem. Our experimental results show that the number of variables has a large effect on the search behavior of EMO algorithms with respect to the diversity of solutions.

  • a many Objective Test problem for visually examining diversity maintenance behavior in a decision space
    Genetic and Evolutionary Computation Conference, 2011
    Co-Authors: Hisao Ishibuchi, Naoya Akedo, Yusuke Nojima
    Abstract:

    Recently distance minimization problems in a two-dimensional decision space have been utilized as many-Objective Test problems to visually examine the behavior of evolutionary multi-Objective optimization (EMO) algorithms. Such a Test problem is usually defined by a single polygon where the distance from a solution to each vertex is minimized in the decision space. We can easily generate different Test problems from different polygons. We can also easily generate Test problems with multiple equivalent Pareto optimal regions using multiple polygons of the same shape and the same size. Whereas these Test problems have a number of advantages, they have no clear relevance to real-world situations since they are artificially generated unrealistic Test problems. In this paper, we generate a distance minimization problem from a real-world map. Our Test problem has four Objectives, which are to minimize the distances to the nearest elementary school, junior high school, railway station, and convenience store. Using our Test problem, we examine the behavior of well-known and frequently-used EMO algorithms in terms of their diversity maintenance ability in the two-dimensional decision space.

  • many Objective Test problems to visually examine the behavior of multiObjective evolution in a decision space
    Parallel Problem Solving from Nature, 2010
    Co-Authors: Hisao Ishibuchi, Yasuhiro Hitotsuyanagi, Noritaka Tsukamoto, Yusuke Nojima
    Abstract:

    Many-Objective optimization is a hot issue in the EMO (evolutionary multiObjective optimization) community. Since almost all solutions in the current population are non-dominated with each other in many-Objective EMO algorithms, we may need a different fitness evaluation scheme from the case of two and three Objectives. One difficulty in the design of many-Objective EMO algorithms is that we cannot visually observe the behavior of multiObjective evolution in the Objective space with four or more Objectives. In this paper, we propose the use of many-Objective Test problems in a two- or three-dimensional decision space to visually examine the behavior of multiObjective evolution. Such a visual examination helps us to understand the characteristic features of EMO algorithms for many-Objective optimization. Good understanding of existing EMO algorithms may facilitates their modification and the development of new EMO algorithms for many-Objective optimization.

Shin Yoo - One of the best experts on this subject based on the ideXlab platform.

  • empirical evaluation of pareto efficient multi Objective regression Test case prioritisation
    International Symposium on Software Testing and Analysis, 2015
    Co-Authors: Michael G Epitropakis, Shin Yoo, Mark Harman, Edmund K Burke
    Abstract:

    The aim of Test case prioritisation is to determine an ordering of Test cases that maximises the likelihood of early fault revelation. Previous prioritisation techniques have tended to be single Objective, for which the additional greedy algorithm is the current state-of-the-art. Unlike Test suite minimisation, multi Objective Test case prioritisation has not been thoroughly evaluated. This paper presents an extensive empirical study of the effectiveness of multi Objective Test case prioritisation, evaluating it on multiple versions of five widely-used benchmark programs and a much larger real world system of over 1 million lines of code. The paper also presents a lossless coverage compaction algorithm that dramatically scales the performance of all algorithms studied by between 2 and 4 orders of magnitude, making prioritisation practical for even very demanding problems.

  • highly scalable multi Objective Test suite minimisation using graphics cards
    Symposium on Search Based Software Engineering, 2011
    Co-Authors: Shin Yoo, Mark Harman
    Abstract:

    Despite claims of "embarrassing parallelism" for many optimisation algorithms, there has been very little work on exploiting parallelism as a route for SBSE scalability. This is an important oversight because scalability is so often a critical success factor for Software Engineering work. This paper shows how relatively inexpensive General Purpose computing on Graphical Processing Units (GPGPU) can be used to run suitably adapted optimisation algorithms, opening up the possibility of cheap scalability. The paper develops a search based optimisation approach for multi Objective regression Test optimisation, evaluating it on benchmark problems as well as larger real world problems. The results indicate that speed-ups of over 25x are possible using widely available standard GPUs. It is also encouraging that the results reveal a statistically strong correlation between larger problem instances and the degree of speed up achieved. This is the first time that GPGPU has been used for SBSE scalability.

  • Using hybrid algorithm for Pareto efficient multi-Objective Test suite minimisation
    Journal of Systems and Software, 2010
    Co-Authors: Shin Yoo, Mark Harman
    Abstract:

    Test suite minimisation techniques seek to reduce the effort required for regression Testing by selecting a subset of Test suites. In previous work, the problem has been considered as a single-Objective optimisation problem. However, real world regression Testing can be a complex process in which multiple Testing criteria and constraints are involved. This paper presents the concept of Pareto efficiency for the Test suite minimisation problem. The Pareto-efficient approach is inherently capable of dealing with multiple Objectives, providing the decision maker with a group of solutions that are not dominated by each other. The paper illustrates the benefits of Pareto efficient multi-Objective Test suite minimisation with empirical studies of two and three Objective formulations, in which multiple Objectives such as coverage and past fault-detection history are considered. The paper utilises a hybrid, multi-Objective genetic algorithm that combines the efficient approximation of the greedy approach with the capability of population based genetic algorithm to produce higher-quality Pareto fronts.

  • pareto efficient multi Objective Test case selection
    International Symposium on Software Testing and Analysis, 2007
    Co-Authors: Shin Yoo, Mark Harman
    Abstract:

    Previous work has treated Test case selection as a single Objective optimisation problem. This paper introduces the concept of Pareto efficiency to Test case selection. The Pareto efficient approach takes multiple Objectives such as code coverage, past fault-detection history and execution cost, and constructs a group of non-dominating, equivalently optimal Test case subsets. The paper describes the potential bene?ts of Pareto efficient multi-Objective Test case selection, illustrating with empirical studies of two and three Objective formulations.

Kalyanmoy Deb - One of the best experts on this subject based on the ideXlab platform.

  • difficulty adjustable and scalable constrained multi Objective Test problem toolkit
    arXiv: Neural and Evolutionary Computing, 2016
    Co-Authors: Zhun Fan, Qingfu Zhang, Kalyanmoy Deb, Xinye Cai, Caimin Wei, Erik D Goodman
    Abstract:

    Multi-Objective evolutionary algorithms (MOEAs) have progressed significantly in recent decades, but most of them are designed to solve unconstrained multi-Objective optimization problems. In fact, many real-world multi-Objective problems contain a number of constraints. To promote research on constrained multi-Objective optimization, we first propose a problem classification scheme with three primary types of difficulty, which reflect various types of challenges presented by real-world optimization problems, in order to characterize the constraint functions in constrained multi-Objective optimization problems (CMOPs). These are feasibility-hardness, convergence-hardness and diversity-hardness. We then develop a general toolkit to construct difficulty-adjustable and scalable CMOPs (DAS-CMOPs, or DAS-CMaOPs when the number of Objectives is greater than three) with three types of parameterized constraint functions developed to capture the three proposed types of difficulty. Based on this toolkit, we suggest nine difficulty-adjustable and scalable CMOPs and nine CMaOPs. The experimental results reveal that mechanisms in MOEA/D-CDP may be more effective in solving convergence-hard DAS-CMOPs, while mechanisms of NSGA-II-CDP may be more effective in solving DAS-CMOPs with simultaneous diversity-, feasibility- and convergence-hardness. Mechanisms in C-NSGA-III may be more effective in solving feasibility-hard CMaOPs, while mechanisms of C-MOEA/DD may be more effective in solving CMaOPs with convergence-hardness. In addition, none of them can solve these problems efficiently, which stimulates us to continue to develop new CMOEAs and CMaOEAs to solve the suggested DAS-CMOPs and DAS-CMaOPs.

  • improved pruning of non dominated solutions based on crowding distance for bi Objective optimization problems
    IEEE International Conference on Evolutionary Computation, 2006
    Co-Authors: Saku Kukkonen, Kalyanmoy Deb
    Abstract:

    In this paper an algorithm for pruning a set of non-dominated solutions is proposed. The algorithm is based on the crowding distance calculation used in the elitist non-dominated sorting genetic algorithm (NSGA-II). The time complexity class of the new algorithm is estimated and in most cases it is the same as for the original pruning algorithm. Numerical results also support this estimate. For used bi-Objective Test problems, the proposed pruning algorithm is demonstrated to provide better distribution compared to the original pruning algorithm of NSGA-II. However, with tri-Objective Test problems there is no improvement and this study reveals that crowding distance does not estimate crowdedness well in this case and presumably also in cases of more Objectives.

  • multi Objective Test problems linkages and evolutionary methodologies
    Genetic and Evolutionary Computation Conference, 2006
    Co-Authors: Kalyanmoy Deb, Ankur Sinha, Saku Kukkonen
    Abstract:

    Existing Test problems for multi-Objective optimization are criticized for not having adequate linkages among variables. In most problems, the Pareto-optimal solutions correspond to a fixed value of certain variables and diversity of solutions comes mainly from a random variation of certain other variables. In this paper, we introduce explicit linkages among variables so as to develop difficult two and multi-Objective Test problems along the lines of ZDT and DTLZ problems. On a number of such Test problems, this paper compares the performance of a number of EMO methodologies having (i) variable-wise versus vector-wise recombination operators and (ii) spatial versus unidirectional recombination operators. Interesting and useful conclusions on the use of above operators are made from the study.

  • multi Objective genetic algorithms problem difficulties and construction of Test problems
    Evolutionary Computation, 1999
    Co-Authors: Kalyanmoy Deb
    Abstract:

    In this paper, we study the problem features that may cause a multi-Objective genetic algorithm (GA) difficulty in converging to the true Pareto-optimal front. Identification of such features helps us develop difficult Test problems for multi-Objective optimization. Multi-Objective Test problems are constructed from single-Objective optimization problems, thereby allowing known difficult features of single-Objective problems (such as multi-modality, isolation, or deception) to be directly transferred to the corresponding multi-Objective problem. In addition, Test problems having features specific to multi-Objective optimization are also constructed. More importantly, these difficult Test problems will enable researchers to Test their algorithms for specific aspects of multi-Objective optimization.