The Experts below are selected from a list of 111597 Experts worldwide ranked by ideXlab platform
Neil D. Lawrence - One of the best experts on this subject based on the ideXlab platform.
-
variational Gaussian Process dynamical systems
arXiv: Machine Learning, 2011Co-Authors: Andreas Damianou, Michalis K. Titsias, Neil D. LawrenceAbstract:High dimensional time series are endemic in applications of machine learning such as robotics (sensor data), computational biology (gene expression data), vision (video sequences) and graphics (motion capture data). Practical nonlinear probabilistic approaches to this data are required. In this paper we introduce the variational Gaussian Process dynamical system. Our work builds on recent variational approximations for Gaussian Process latent variable models to allow for nonlinear dimensionality reduction simultaneously with learning a dynamical prior in the latent space. The approach also allows for the appropriate dimensionality of the latent space to be automatically determined. We demonstrate the model on a human motion capture data set and a series of high resolution video sequences.
-
bayesian Gaussian Process latent variable model
International Conference on Artificial Intelligence and Statistics, 2010Co-Authors: Michalis K. Titsias, Neil D. LawrenceAbstract:We introduce a variational inference framework for training the Gaussian Process latent variable model and thus performing Bayesian nonlinear dimensionality reduction. This method allows us to variationally integrate out the input variables of the Gaussian Process and compute a lower bound on the exact marginal likelihood of the nonlinear latent variable model. The maximization of the variational lower bound provides a Bayesian training procedure that is robust to overfitting and can automatically select the dimensionality of the nonlinear latent space. We demonstrate our method on real world datasets. The focus in this paper is on dimensionality reduction problems, but the methodology is more general. For example, our algorithm is immediately applicable for training Gaussian Process models in the presence of missing or uncertain inputs.
-
hierarchical Gaussian Process latent variable models
International Conference on Machine Learning, 2007Co-Authors: Neil D. Lawrence, Andrew J. MooreAbstract:The Gaussian Process latent variable model (GP-LVM) is a powerful approach for probabilistic modelling of high dimensional data through dimensional reduction. In this paper we extend the GP-LVM through hierarchies. A hierarchical model (such as a tree) allows us to express conditional independencies in the data as well as the manifold structure. We first introduce Gaussian Process hierarchies through a simple dynamical model, we then extend the approach to a more complex hierarchy which is applied to the visualisation of human motion data sets.
-
learning for larger datasets with the Gaussian Process latent variable model
International Conference on Artificial Intelligence and Statistics, 2007Co-Authors: Neil D. LawrenceAbstract:In this paper we apply the latest techniques in sparse Gaussian Process regression (GPR) to the Gaussian Process latent variable model (GPLVM). We review three techniques and discuss how they may be implemented in the context of the GP-LVM. Each approach is then implemented on a well known benchmark data set and compared with earlier attempts to sparsify the model.
Carl Edward Rasmussen - One of the best experts on this subject based on the ideXlab platform.
-
approximate inference for fully bayesian Gaussian Process regression
arXiv: Machine Learning, 2019Co-Authors: Vidhi Lalchand, Carl Edward RasmussenAbstract:Learning in Gaussian Process models occurs through the adaptation of hyperparameters of the mean and the covariance function. The classical approach entails maximizing the marginal likelihood yielding fixed point estimates (an approach called \textit{Type II maximum likelihood} or ML-II). An alternative learning procedure is to infer the posterior over hyperparameters in a hierarchical specification of GPs we call \textit{Fully Bayesian Gaussian Process Regression} (GPR). This work considers two approximation schemes for the intractable hyperparameter posterior: 1) Hamiltonian Monte Carlo (HMC) yielding a sampling-based approximation and 2) Variational Inference (VI) where the posterior over hyperparameters is approximated by a factorized Gaussian (mean-field) or a full-rank Gaussian accounting for correlations between hyperparameters. We analyze the predictive performance for fully Bayesian GPR on a range of benchmark data sets.
-
sparse spectrum Gaussian Process regression
Journal of Machine Learning Research, 2010Co-Authors: Miguel Lazarogredilla, Carl Edward Rasmussen, Joaquin Quinonerocandela, Anibal R FigueirasvidalAbstract:We present a new sparse Gaussian Process (GP) model for regression. The key novel idea is to sparsify the spectral representation of the GP. This leads to a simple, practical algorithm for regression tasks. We compare the achievable trade-offs between predictive accuracy and computational requirements, and show that these are typically superior to existing state-of-the-art sparse approximations. We discuss both the weight space and function space representations, and note that the new construction implies priors over functions which are always stationary, and can approximate any covariance function in this class.
-
assessing approximate inference for binary Gaussian Process classification
Journal of Machine Learning Research, 2005Co-Authors: Malte Kuss, Carl Edward RasmussenAbstract:Gaussian Process priors can be used to define flexible, probabilistic classification models. Unfortunately exact Bayesian inference is analytically intractable and various approximation techniques have been proposed. In this work we review and compare Laplace's method and Expectation Propagation for approximate Bayesian inference in the binary Gaussian Process classification model. We present a comprehensive comparison of the approximations, their predictive performance and marginal likelihood estimates to results obtained by MCMC sampling. We explain theoretically and corroborate empirically the advantages of Expectation Propagation compared to Laplace's method.
Zoubin Ghahramani - One of the best experts on this subject based on the ideXlab platform.
-
Gaussian Process Behaviour in Wide Deep Neural Networks
arXiv: Machine Learning, 2018Co-Authors: Alexander G. De G. Matthews, Mark Rowland, Jiri Hron, Richard E. Turner, Zoubin GhahramaniAbstract:Whilst deep neural networks have shown great empirical success, there is still much work to be done to understand their theoretical properties. In this paper, we study the relationship between random, wide, fully connected, feedforward networks with more than one hidden layer and Gaussian Processes with a recursive kernel definition. We show that, under broad conditions, as we make the architecture increasingly wide, the implied random function converges in distribution to a Gaussian Process, formalising and extending existing results by Neal (1996) to deep networks. To evaluate convergence rates empirically, we use maximum mean discrepancy. We then compare finite Bayesian deep networks from the literature to Gaussian Processes in terms of the key predictive quantities of interest, finding that in some cases the agreement can be very close. We discuss the desirability of Gaussian Process behaviour and review non-Gaussian alternative models from the literature.
-
gpflow a Gaussian Process library using tensorflow
arXiv: Machine Learning, 2016Co-Authors: Alexander G. De G. Matthews, Zoubin Ghahramani, Mark Van Der Wilk, Tom Nickson, Keisuke Fujii, Alexis Boukouvalas, Pablo Leonvillagra, James HensmanAbstract:GPflow is a Gaussian Process library that uses TensorFlow for its core computations and Python for its front end. The distinguishing features of GPflow are that it uses variational inference as the primary approximation method, provides concise code through the use of automatic differentiation, has been engineered with a particular emphasis on software testing and is able to exploit GPU hardware.
-
scalable variational Gaussian Process classification
arXiv: Machine Learning, 2014Co-Authors: James Hensman, Alexander G. De G. Matthews, Zoubin GhahramaniAbstract:Gaussian Process classification is a popular method with a number of appealing properties. We show how to scale the model within a variational inducing point framework, outperforming the state of the art on benchmark datasets. Importantly, the variational formulation can be exploited to allow classification in problems with millions of data points, as we demonstrate in experiments.
Michalis K. Titsias - One of the best experts on this subject based on the ideXlab platform.
-
variational inference for mahalanobis distance metrics in Gaussian Process regression
Neural Information Processing Systems, 2013Co-Authors: Michalis K. Titsias, Miguel LazarogredillaAbstract:We introduce a novel variational method that allows to approximately integrate out kernel hyperparameters, such as length-scales, in Gaussian Process regression. This approach consists of a novel variant of the variational framework that has been recently developed for the Gaussian Process latent variable model which additionally makes use of a standardised representation of the Gaussian Process. We consider this technique for learning Mahalanobis distance metrics in a Gaussian Process regression setting and provide experimental evaluations and comparisons with existing methods by considering datasets with high-dimensional inputs.
-
variational Gaussian Process dynamical systems
arXiv: Machine Learning, 2011Co-Authors: Andreas Damianou, Michalis K. Titsias, Neil D. LawrenceAbstract:High dimensional time series are endemic in applications of machine learning such as robotics (sensor data), computational biology (gene expression data), vision (video sequences) and graphics (motion capture data). Practical nonlinear probabilistic approaches to this data are required. In this paper we introduce the variational Gaussian Process dynamical system. Our work builds on recent variational approximations for Gaussian Process latent variable models to allow for nonlinear dimensionality reduction simultaneously with learning a dynamical prior in the latent space. The approach also allows for the appropriate dimensionality of the latent space to be automatically determined. We demonstrate the model on a human motion capture data set and a series of high resolution video sequences.
-
bayesian Gaussian Process latent variable model
International Conference on Artificial Intelligence and Statistics, 2010Co-Authors: Michalis K. Titsias, Neil D. LawrenceAbstract:We introduce a variational inference framework for training the Gaussian Process latent variable model and thus performing Bayesian nonlinear dimensionality reduction. This method allows us to variationally integrate out the input variables of the Gaussian Process and compute a lower bound on the exact marginal likelihood of the nonlinear latent variable model. The maximization of the variational lower bound provides a Bayesian training procedure that is robust to overfitting and can automatically select the dimensionality of the nonlinear latent space. We demonstrate our method on real world datasets. The focus in this paper is on dimensionality reduction problems, but the methodology is more general. For example, our algorithm is immediately applicable for training Gaussian Process models in the presence of missing or uncertain inputs.
Junichi Imura - One of the best experts on this subject based on the ideXlab platform.
-
a Gaussian Process based incremental neural network for online regression
International Conference on Neural Information Processing, 2020Co-Authors: Xiaoyu Wang, Lucian Gheorghe, Junichi ImuraAbstract:This paper proposes a Gaussian Process-based incremental neural network algorithm to handle the online regression problem. It can extract prototypes by an incremental neural network, where 1) Gaussian Process approximations are adopted to update the threshold regions and the posterior distribution of the dependent variable at the weight vectors of nodes and 2) the optimal bandwidth matrix is derived for adapting to network structure. Besides, we discuss some properties of the proposed approach, and the experimental results show that our approach achieves remarkable accuracy improvement in extracting prototypes for online regression on noisy data.
-
a Gaussian Process based self organizing incremental neural network
International Joint Conference on Neural Network, 2019Co-Authors: Xiaoyu Wang, Giona Casiraghi, Yan Zhang, Junichi ImuraAbstract:This paper proposes a Gaussian Process-based self-organizing incremental neural network (GPINN) to address the density estimation problem of online unsupervised learning. First, we adopt Gaussian Process models with adaptive kernels that map the distribution of the neighbors of each node to its link relationship. Second, combining GPINN and kernel density estimation, we derive the bandwidth matrix updating rule for adapting to the generated network. We theoretically analyze the advantages of the proposed approach in determining threshold regions over using distance measures. The experimental results on both synthetic data sets and real-world data sets show that our method achieves remarkable improvement in density estimation accuracy for large noisy data.