The Experts below are selected from a list of 8283 Experts worldwide ranked by ideXlab platform

Liva Ralaivola - One of the best experts on this subject based on the ideXlab platform.

  • pac bayesian generalization bound on confusion matrix for multi class classification
    International Conference on Machine Learning, 2012
    Co-Authors: Emilie Morvant, Sokol Ko O, Liva Ralaivola
    Abstract:

    In this paper, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with performance measure that are richer than mere scalar criterion such as the Misclassification Rate. Thanks to very recent and beautiful results on matrix concentration inequalities, we derive two bounds showing that the true confusion risk of the Gibbs classifier is upper-bounded by its empirical risk plus a term depending on the number of training examples in each class. To the best of our knowledge, this is the first PAC-Bayes bounds based on confusion matrices.

  • pac bayesian generalization bound on confusion matrix for multi class classification
    arXiv: Machine Learning, 2012
    Co-Authors: Emilie Morvant, Sokol Koco, Liva Ralaivola
    Abstract:

    In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with performance measure that are richer than mere scalar criterion such as the Misclassification Rate. Thanks to very recent and beautiful results on matrix concentration inequalities, we derive two bounds showing that the true confusion risk of the Gibbs classifier is upper-bounded by its empirical risk plus a term depending on the number of training examples in each class. To the best of our knowledge, this is the first PAC-Bayes bounds based on confusion matrices.

C J Harris - One of the best experts on this subject based on the ideXlab platform.

  • probability density function estimation based over sampling for imbalanced two class problems
    International Joint Conference on Neural Network, 2012
    Co-Authors: Xia Hong, Sheng Chen, C J Harris
    Abstract:

    A novel probability density function (PDF) estimation based over-sampling approach is proposed for two-class imbalanced classification problems. The Parzen-window kernel function is applied to estimate the PDF of the positive class, from which synthetic instances are geneRated as additional training data to re-balance the class distribution. Utilising the re-balanced over-sampled training data, a radial basis function (RBF) classifier is constructed by applying an orthogonal forward regression, in which the classifier's structure and the parameters of RBF kernels are determined using a particle swarm optimisation algorithm based on the criterion of minimising the leave-one-out Misclassification Rate. The effectiveness of the proposed approach is demonstRated by an empirical study on several imbalanced data sets.

  • a combined smote and pso based rbf classifier for two class imbalanced problems
    Neurocomputing, 2011
    Co-Authors: Ming Gao, Sheng Chen, Xia Hong, C J Harris
    Abstract:

    This contribution proposes a powerful technique for two-class imbalanced classification problems by combining the synthetic minority over-sampling technique (SMOTE) and the particle swarm optimisation (PSO) aided radial basis function (RBF) classifier. In order to enhance the significance of the small and specific region belonging to the positive class in the decision region, the SMOTE is applied to geneRate synthetic instances for the positive class to balance the training data set. Based on the over-sampled training data, the RBF classifier is constructed by applying the orthogonal forward selection procedure, in which the classifier's structure and the parameters of RBF kernels are determined using a PSO algorithm based on the criterion of minimising the leave-one-out Misclassification Rate. The experimental results obtained on a simulated imbalanced data set and three real imbalanced data sets are presented to demonstRate the effectiveness of our proposed algorithm.

  • radial basis function classifier construction using particle swarm optimisation aided orthogonal forward regression
    International Joint Conference on Neural Network, 2010
    Co-Authors: Sheng Chen, Xia Hong, C J Harris
    Abstract:

    We develop a particle swarm optimisation (PSO) aided orthogonal forward regression (OFR) approach for constructing radial basis function (RBF) classifiers with tunable nodes. At each stage of the OFR construction process, the centre vector and diagonal covariance matrix of one RBF node is determined efficiently by minimising the leave-one-out (LOO) Misclassification Rate (MR) using a PSO algorithm. Compared with the state-of-the-art regularisation assisted orthogonal least square algorithm based on the LOO MR for selecting fixed-node RBF classifiers, the proposed PSO aided OFR algorithm for constructing tunable-node RBF classifiers offers significant advantages in terms of better generalisation performance and smaller model size as well as imposes lower computational complexity in classifier construction process. Moreover, the proposed algorithm does not have any hyperparameter that requires costly tuning based on cross validation.

  • a fast linear in the parameters classifier construction algorithm using orthogonal forward selection to minimize leave one out Misclassification Rate
    International Journal of Systems Science, 2008
    Co-Authors: Xia Hong, Sheng Chen, C J Harris
    Abstract:

    We propose a simple and computationally efficient construction algorithm for two class linear-in-the-parameters classifiers. In order to optimize model generalization, a forward orthogonal selection (OFS) procedure is used for minimizing the leave-one-out (LOO) Misclassification Rate directly. An analytic formula and a set of forward recursive updating formula of the LOO Misclassification Rate are developed and applied in the proposed algorithm. Numerical examples are used to demonstRate that the proposed algorithm is an excellent alternative approach to construct sparse two class classifiers in terms of performance and computational efficiency.

Emilie Morvant - One of the best experts on this subject based on the ideXlab platform.

  • pac bayesian generalization bound on confusion matrix for multi class classification
    International Conference on Machine Learning, 2012
    Co-Authors: Emilie Morvant, Sokol Ko O, Liva Ralaivola
    Abstract:

    In this paper, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with performance measure that are richer than mere scalar criterion such as the Misclassification Rate. Thanks to very recent and beautiful results on matrix concentration inequalities, we derive two bounds showing that the true confusion risk of the Gibbs classifier is upper-bounded by its empirical risk plus a term depending on the number of training examples in each class. To the best of our knowledge, this is the first PAC-Bayes bounds based on confusion matrices.

  • pac bayesian generalization bound on confusion matrix for multi class classification
    arXiv: Machine Learning, 2012
    Co-Authors: Emilie Morvant, Sokol Koco, Liva Ralaivola
    Abstract:

    In this work, we propose a PAC-Bayes bound for the generalization risk of the Gibbs classifier in the multi-class classification framework. The novelty of our work is the critical use of the confusion matrix of a classifier as an error measure; this puts our contribution in the line of work aiming at dealing with performance measure that are richer than mere scalar criterion such as the Misclassification Rate. Thanks to very recent and beautiful results on matrix concentration inequalities, we derive two bounds showing that the true confusion risk of the Gibbs classifier is upper-bounded by its empirical risk plus a term depending on the number of training examples in each class. To the best of our knowledge, this is the first PAC-Bayes bounds based on confusion matrices.

Xia Hong - One of the best experts on this subject based on the ideXlab platform.

  • probability density function estimation based over sampling for imbalanced two class problems
    International Joint Conference on Neural Network, 2012
    Co-Authors: Xia Hong, Sheng Chen, C J Harris
    Abstract:

    A novel probability density function (PDF) estimation based over-sampling approach is proposed for two-class imbalanced classification problems. The Parzen-window kernel function is applied to estimate the PDF of the positive class, from which synthetic instances are geneRated as additional training data to re-balance the class distribution. Utilising the re-balanced over-sampled training data, a radial basis function (RBF) classifier is constructed by applying an orthogonal forward regression, in which the classifier's structure and the parameters of RBF kernels are determined using a particle swarm optimisation algorithm based on the criterion of minimising the leave-one-out Misclassification Rate. The effectiveness of the proposed approach is demonstRated by an empirical study on several imbalanced data sets.

  • a combined smote and pso based rbf classifier for two class imbalanced problems
    Neurocomputing, 2011
    Co-Authors: Ming Gao, Sheng Chen, Xia Hong, C J Harris
    Abstract:

    This contribution proposes a powerful technique for two-class imbalanced classification problems by combining the synthetic minority over-sampling technique (SMOTE) and the particle swarm optimisation (PSO) aided radial basis function (RBF) classifier. In order to enhance the significance of the small and specific region belonging to the positive class in the decision region, the SMOTE is applied to geneRate synthetic instances for the positive class to balance the training data set. Based on the over-sampled training data, the RBF classifier is constructed by applying the orthogonal forward selection procedure, in which the classifier's structure and the parameters of RBF kernels are determined using a PSO algorithm based on the criterion of minimising the leave-one-out Misclassification Rate. The experimental results obtained on a simulated imbalanced data set and three real imbalanced data sets are presented to demonstRate the effectiveness of our proposed algorithm.

  • radial basis function classifier construction using particle swarm optimisation aided orthogonal forward regression
    International Joint Conference on Neural Network, 2010
    Co-Authors: Sheng Chen, Xia Hong, C J Harris
    Abstract:

    We develop a particle swarm optimisation (PSO) aided orthogonal forward regression (OFR) approach for constructing radial basis function (RBF) classifiers with tunable nodes. At each stage of the OFR construction process, the centre vector and diagonal covariance matrix of one RBF node is determined efficiently by minimising the leave-one-out (LOO) Misclassification Rate (MR) using a PSO algorithm. Compared with the state-of-the-art regularisation assisted orthogonal least square algorithm based on the LOO MR for selecting fixed-node RBF classifiers, the proposed PSO aided OFR algorithm for constructing tunable-node RBF classifiers offers significant advantages in terms of better generalisation performance and smaller model size as well as imposes lower computational complexity in classifier construction process. Moreover, the proposed algorithm does not have any hyperparameter that requires costly tuning based on cross validation.

  • Construction of Tunable Radial Basis Function Networks Using Orthogonal Forward Selection
    IEEE transactions on systems man and cybernetics. Part B Cybernetics : a publication of the IEEE Systems Man and Cybernetics Society, 2008
    Co-Authors: Sheng Chen, Xia Hong, B.l. Luk, Chris J. Harris
    Abstract:

    An orthogonal forward selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO Misclassification Rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstRated using examples taken from both regression and classification applications.

  • a fast linear in the parameters classifier construction algorithm using orthogonal forward selection to minimize leave one out Misclassification Rate
    International Journal of Systems Science, 2008
    Co-Authors: Xia Hong, Sheng Chen, C J Harris
    Abstract:

    We propose a simple and computationally efficient construction algorithm for two class linear-in-the-parameters classifiers. In order to optimize model generalization, a forward orthogonal selection (OFS) procedure is used for minimizing the leave-one-out (LOO) Misclassification Rate directly. An analytic formula and a set of forward recursive updating formula of the LOO Misclassification Rate are developed and applied in the proposed algorithm. Numerical examples are used to demonstRate that the proposed algorithm is an excellent alternative approach to construct sparse two class classifiers in terms of performance and computational efficiency.

Sheng Chen - One of the best experts on this subject based on the ideXlab platform.

  • probability density function estimation based over sampling for imbalanced two class problems
    International Joint Conference on Neural Network, 2012
    Co-Authors: Xia Hong, Sheng Chen, C J Harris
    Abstract:

    A novel probability density function (PDF) estimation based over-sampling approach is proposed for two-class imbalanced classification problems. The Parzen-window kernel function is applied to estimate the PDF of the positive class, from which synthetic instances are geneRated as additional training data to re-balance the class distribution. Utilising the re-balanced over-sampled training data, a radial basis function (RBF) classifier is constructed by applying an orthogonal forward regression, in which the classifier's structure and the parameters of RBF kernels are determined using a particle swarm optimisation algorithm based on the criterion of minimising the leave-one-out Misclassification Rate. The effectiveness of the proposed approach is demonstRated by an empirical study on several imbalanced data sets.

  • a combined smote and pso based rbf classifier for two class imbalanced problems
    Neurocomputing, 2011
    Co-Authors: Ming Gao, Sheng Chen, Xia Hong, C J Harris
    Abstract:

    This contribution proposes a powerful technique for two-class imbalanced classification problems by combining the synthetic minority over-sampling technique (SMOTE) and the particle swarm optimisation (PSO) aided radial basis function (RBF) classifier. In order to enhance the significance of the small and specific region belonging to the positive class in the decision region, the SMOTE is applied to geneRate synthetic instances for the positive class to balance the training data set. Based on the over-sampled training data, the RBF classifier is constructed by applying the orthogonal forward selection procedure, in which the classifier's structure and the parameters of RBF kernels are determined using a PSO algorithm based on the criterion of minimising the leave-one-out Misclassification Rate. The experimental results obtained on a simulated imbalanced data set and three real imbalanced data sets are presented to demonstRate the effectiveness of our proposed algorithm.

  • radial basis function classifier construction using particle swarm optimisation aided orthogonal forward regression
    International Joint Conference on Neural Network, 2010
    Co-Authors: Sheng Chen, Xia Hong, C J Harris
    Abstract:

    We develop a particle swarm optimisation (PSO) aided orthogonal forward regression (OFR) approach for constructing radial basis function (RBF) classifiers with tunable nodes. At each stage of the OFR construction process, the centre vector and diagonal covariance matrix of one RBF node is determined efficiently by minimising the leave-one-out (LOO) Misclassification Rate (MR) using a PSO algorithm. Compared with the state-of-the-art regularisation assisted orthogonal least square algorithm based on the LOO MR for selecting fixed-node RBF classifiers, the proposed PSO aided OFR algorithm for constructing tunable-node RBF classifiers offers significant advantages in terms of better generalisation performance and smaller model size as well as imposes lower computational complexity in classifier construction process. Moreover, the proposed algorithm does not have any hyperparameter that requires costly tuning based on cross validation.

  • Construction of Tunable Radial Basis Function Networks Using Orthogonal Forward Selection
    IEEE transactions on systems man and cybernetics. Part B Cybernetics : a publication of the IEEE Systems Man and Cybernetics Society, 2008
    Co-Authors: Sheng Chen, Xia Hong, B.l. Luk, Chris J. Harris
    Abstract:

    An orthogonal forward selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO Misclassification Rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstRated using examples taken from both regression and classification applications.

  • a fast linear in the parameters classifier construction algorithm using orthogonal forward selection to minimize leave one out Misclassification Rate
    International Journal of Systems Science, 2008
    Co-Authors: Xia Hong, Sheng Chen, C J Harris
    Abstract:

    We propose a simple and computationally efficient construction algorithm for two class linear-in-the-parameters classifiers. In order to optimize model generalization, a forward orthogonal selection (OFS) procedure is used for minimizing the leave-one-out (LOO) Misclassification Rate directly. An analytic formula and a set of forward recursive updating formula of the LOO Misclassification Rate are developed and applied in the proposed algorithm. Numerical examples are used to demonstRate that the proposed algorithm is an excellent alternative approach to construct sparse two class classifiers in terms of performance and computational efficiency.