The Experts below are selected from a list of 53721 Experts worldwide ranked by ideXlab platform

Matthias Seeger - One of the best experts on this subject based on the ideXlab platform.

  • Fast Sparse Gaussian Process Methods: The Informative Vector Machine
    Advances in Neural Information Processing Systems 15, 2003
    Co-Authors: Neil Lawrence, Matthias Seeger, Neil D. Lawrence, Ralf Herbrich
    Abstract:

    We present a framework for sparse Gaussian process (GP) methods which uses Forward Selection with criteria based on informationtheoretic principles, previously suggested for active learning. Our goal is not only to learn dsparse predictors (which can be evaluated in O(d) rather than O(n), d n, n the number of training points), but also to perform training under strong restrictions on time and memory requirements. The scaling of our method is at most O(n d2), and in large real-world classi cation experiments we show that it can match prediction performance of the popular support vector machine (SVM), yet can be signi cantly faster in training. In contrast to the SVM, our approximation produces estimates of predictive probabilities (`error bars'), allows for Bayesian model Selection and is less complex in implementation.

  • fast Forward Selection to speed up sparse gaussian process regression
    International Conference on Artificial Intelligence and Statistics, 2003
    Co-Authors: Matthias Seeger, Christopher K I Williams, Neil D. Lawrence
    Abstract:

    We present a method for the sparse greedy approximation of Bayesian Gaussian process regression, featuring a novel heuristic for very fast Forward Selection. Our method is essentially as fast as an equivalent one which selects the "support" patterns at random, yet it can outperform random Selection on hard curve fitting tasks. More importantly, it leads to a sufficiently stable approximation of the log marginal likelihood of the training data, which can be optimised to adjust a large number of hyperparameters automatically. We demonstrate the model Selection capabilities of the algorithm in a range of experiments. In line with the development of our method, we present a simple view on sparse approximations for GP models and their underlying assumptions and show relations to other methods.

Kezhi Mao - One of the best experts on this subject based on the ideXlab platform.

  • Orthogonal Forward Selection and backward elimination algorithms for feature subset Selection
    IEEE transactions on systems man and cybernetics. Part B Cybernetics : a publication of the IEEE Systems Man and Cybernetics Society, 2004
    Co-Authors: Kezhi Mao
    Abstract:

    Sequential Forward Selection (SFS) and sequential backward elimination (SBE) are two commonly used search methods in feature subset Selection. In the present study, we derive an orthogonal Forward Selection (OFS) and an orthogonal backward elimination (OBE) algorithms for feature subset Selection by incorporating Gram-Schmidt and Givens orthogonal transforms into Forward Selection and backward elimination procedures, respectively. The basic idea of the orthogonal feature subset Selection algorithms is to find an orthogonal space in which to express features and to perform feature subset Selection. After Selection, the physically meaningless features in the orthogonal space are linked back to the same number of input variables in the original measurement space. The strength of employing orthogonal transforms is that features are decorrelated in the orthogonal space, hence individual features can be evaluated and selected independently. The effectiveness of our algorithms to deal with real world problems is finally demonstrated.

  • fast orthogonal Forward Selection algorithm for feature subset Selection
    IEEE Transactions on Neural Networks, 2002
    Co-Authors: Kezhi Mao
    Abstract:

    Feature Selection is an important issue in pattern classification. In the presented study, we develop a fast orthogonal Forward Selection (FOFS) algorithm for feature subset Selection. The FOFS algorithm employs an orthogonal transform to decompose correlations among candidate features, but it performs the orthogonal decomposition in an implicit way. Consequently, the fast algorithm demands less computational effort as compared with conventional orthogonal Forward Selection (OFS).

Neil D. Lawrence - One of the best experts on this subject based on the ideXlab platform.

  • Fast Sparse Gaussian Process Methods: The Informative Vector Machine
    Advances in Neural Information Processing Systems 15, 2003
    Co-Authors: Neil Lawrence, Matthias Seeger, Neil D. Lawrence, Ralf Herbrich
    Abstract:

    We present a framework for sparse Gaussian process (GP) methods which uses Forward Selection with criteria based on informationtheoretic principles, previously suggested for active learning. Our goal is not only to learn dsparse predictors (which can be evaluated in O(d) rather than O(n), d n, n the number of training points), but also to perform training under strong restrictions on time and memory requirements. The scaling of our method is at most O(n d2), and in large real-world classi cation experiments we show that it can match prediction performance of the popular support vector machine (SVM), yet can be signi cantly faster in training. In contrast to the SVM, our approximation produces estimates of predictive probabilities (`error bars'), allows for Bayesian model Selection and is less complex in implementation.

  • fast Forward Selection to speed up sparse gaussian process regression
    International Conference on Artificial Intelligence and Statistics, 2003
    Co-Authors: Matthias Seeger, Christopher K I Williams, Neil D. Lawrence
    Abstract:

    We present a method for the sparse greedy approximation of Bayesian Gaussian process regression, featuring a novel heuristic for very fast Forward Selection. Our method is essentially as fast as an equivalent one which selects the "support" patterns at random, yet it can outperform random Selection on hard curve fitting tasks. More importantly, it leads to a sufficiently stable approximation of the log marginal likelihood of the training data, which can be optimised to adjust a large number of hyperparameters automatically. We demonstrate the model Selection capabilities of the algorithm in a range of experiments. In line with the development of our method, we present a simple view on sparse approximations for GP models and their underlying assumptions and show relations to other methods.

Chris J. Harris - One of the best experts on this subject based on the ideXlab platform.

  • Construction of Tunable Radial Basis Function Networks Using Orthogonal Forward Selection
    IEEE transactions on systems man and cybernetics. Part B Cybernetics : a publication of the IEEE Systems Man and Cybernetics Society, 2008
    Co-Authors: Sheng Chen, Xia Hong, B.l. Luk, Chris J. Harris
    Abstract:

    An orthogonal Forward Selection (OFS) algorithm based on leave-one-out (LOO) criteria is proposed for the construction of radial basis function (RBF) networks with tunable nodes. Each stage of the construction process determines an RBF node, namely, its center vector and diagonal covariance matrix, by minimizing the LOO statistics. For regression application, the LOO criterion is chosen to be the LOO mean-square error, while the LOO misclassification rate is adopted in two-class classification application. This OFS-LOO algorithm is computationally efficient, and it is capable of constructing parsimonious RBF networks that generalize well. Moreover, the proposed algorithm is fully automatic, and the user does not need to specify a termination criterion for the construction process. The effectiveness of the proposed RBF network construction procedure is demonstrated using examples taken from both regression and classification applications.

  • a fast linear in the parameters classifier construction algorithm using orthogonal Forward Selection to minimize leave one out misclassification rate
    International Journal of Systems Science, 2008
    Co-Authors: Xia Hong, Sheng Chen, Chris J. Harris
    Abstract:

    We propose a simple and computationally efficient construction algorithm for two class linear-in-the-parameters classifiers. In order to optimize model generalization, a Forward orthogonal Selection (OFS) procedure is used for minimizing the leave-one-out (LOO) misclassification rate directly. An analytic formula and a set of Forward recursive updating formula of the LOO misclassification rate are developed and applied in the proposed algorithm. Numerical examples are used to demonstrate that the proposed algorithm is an excellent alternative approach to construct sparse two class classifiers in terms of performance and computational efficiency.

  • orthogonal Forward Selection for constructing the radial basis function network with tunable nodes
    International Conference on Intelligent Computing, 2005
    Co-Authors: Sheng Chen, Xia Hong, Chris J. Harris
    Abstract:

    An orthogonal Forward Selection (OFS) algorithm based on the leave-one-out (LOO) criterion is proposed for the construction of radial basis function (RBF) networks with tunable nodes. This OFS-LOO algorithm is computationally efficient and is capable of identifying parsimonious RBF networks that generalise well. Moreover, the proposed algorithm is fully automatic and the user does not need to specify a termination criterion for the construction process.

Constantine Kotropoulos - One of the best experts on this subject based on the ideXlab platform.

  • emotional speech classification using gaussian mixture models and the sequential floating Forward Selection algorithm
    International Conference on Multimedia and Expo, 2005
    Co-Authors: Dimitrios Ververidis, Constantine Kotropoulos
    Abstract:

    Emotional speech classification can be treated as a supervised learning task where the statistical properties of emotional speech segments are the features and the emotional styles form the labels. The Akaike criterion is used for estimating automatically the number of Gaussian densities that model the probability density function of the emotional speech features. A procedure for reducing the computational burden of crossvalidation in sequential floating Forward Selection algorithm is proposed that applies the t-test on the probability of correct classification for the Bayes classifier designed for various feature sets. For the Bayes classifier, the sequential floating Forward Selection algorithm is found to yield a higher probability of correct classification by 3% than that of the sequential Forward Selection algorithm either taking into account the gender information or ignoring it. The experimental results indicate that the utterances from isolated words and sentences are more colored emotional than those from paragraphs. Without taking into account the gender information, the probability of correct classification for the Bayes classifier admits a maximum when the probability density function of emotional speech features extracted from the aforementioned utterances is modeled as a mixture of 2 Gaussian densities