The Experts below are selected from a list of 10917 Experts worldwide ranked by ideXlab platform
Stefano Carrazza - One of the best experts on this subject based on the ideXlab platform.
-
sampling the riemann theta Boltzmann Machine
Computer Physics Communications, 2020Co-Authors: Stefano Carrazza, Daniel KreflAbstract:Abstract We show that the visible sector probability density function of the Riemann-Theta Boltzmann Machine corresponds to a Gaussian mixture model consisting of an infinite number of component multi-variate Gaussians. The weights of the mixture are given by a discrete multi-variate Gaussian over the hidden state space. This allows us to sample the visible sector density function in a straightforward manner. Furthermore, we show that the visible sector probability density function possesses an affine transform property, similar to the multi-variate Gaussian density.
-
Riemann-Theta Boltzmann Machine
Neurocomputing, 2020Co-Authors: Daniel Krefl, Jens Kahlen, Babak Haghighat, Stefano CarrazzaAbstract:Abstract A general Boltzmann Machine with continuous visible and discrete integer valued hidden states is introduced. Under mild assumptions about the connection matrices, the probability density function of the visible units can be solved for analytically, yielding a novel parametric density function involving a ratio of Riemann-Theta functions. The conditional expectation of a hidden state for given visible states can also be calculated analytically, yielding a derivative of the logarithmic Riemann-Theta function. The conditional expectation can be used as activation function in a feedforward neural network, thereby increasing the modelling capacity of the network. Both the Boltzmann Machine and the derived feedforward neural network can be successfully trained via standard gradient- and non-gradient-based optimization techniques.
Daniel Krefl - One of the best experts on this subject based on the ideXlab platform.
-
sampling the riemann theta Boltzmann Machine
Computer Physics Communications, 2020Co-Authors: Stefano Carrazza, Daniel KreflAbstract:Abstract We show that the visible sector probability density function of the Riemann-Theta Boltzmann Machine corresponds to a Gaussian mixture model consisting of an infinite number of component multi-variate Gaussians. The weights of the mixture are given by a discrete multi-variate Gaussian over the hidden state space. This allows us to sample the visible sector density function in a straightforward manner. Furthermore, we show that the visible sector probability density function possesses an affine transform property, similar to the multi-variate Gaussian density.
-
Riemann-Theta Boltzmann Machine
Neurocomputing, 2020Co-Authors: Daniel Krefl, Jens Kahlen, Babak Haghighat, Stefano CarrazzaAbstract:Abstract A general Boltzmann Machine with continuous visible and discrete integer valued hidden states is introduced. Under mild assumptions about the connection matrices, the probability density function of the visible units can be solved for analytically, yielding a novel parametric density function involving a ratio of Riemann-Theta functions. The conditional expectation of a hidden state for given visible states can also be calculated analytically, yielding a derivative of the logarithmic Riemann-Theta function. The conditional expectation can be used as activation function in a feedforward neural network, thereby increasing the modelling capacity of the network. Both the Boltzmann Machine and the derived feedforward neural network can be successfully trained via standard gradient- and non-gradient-based optimization techniques.
Graham W Taylor - One of the best experts on this subject based on the ideXlab platform.
-
modeling pigeon behavior using a conditional restricted Boltzmann Machine
The European Symposium on Artificial Neural Networks, 2009Co-Authors: Matthew D Zeiler, Graham W Taylor, Nikolaus F Troje, Geoffrey E HintonAbstract:In an effort to better understand the complex courtship be- haviour of pigeons, we have built a model learned from motion capture data. We employ a Conditional Restricted Boltzmann Machine (CRBM) with binary latent features and real-valued visible units. The units are conditioned on information from previous time steps to capture dynam- ics. We validate a trained model by quantifying the characteristic "head- bobbing" present in pigeons. We also show how to predict missing data by marginalizing out the hidden variables and minimizing free energy.
-
the recurrent temporal restricted Boltzmann Machine
Neural Information Processing Systems, 2008Co-Authors: Ilya Sutskever, Geoffrey E Hinton, Graham W TaylorAbstract:The Temporal Restricted Boltzmann Machine (TRBM) is a probabilistic model for sequences that is able to successfully model (i.e., generate nice-looking samples of) several very high dimensional sequences, such as motion capture data and the pixels of low resolution videos of balls bouncing in a box. The major disadvantage of the TRBM is that exact inference is extremely hard, since even computing a Gibbs update for a single variable of the posterior is exponentially expensive. This difficulty has necessitated the use of a heuristic inference procedure, that nonetheless was accurate enough for successful learning. In this paper we introduce the Recurrent TRBM, which is a very slight modification of the TRBM for which exact inference is very easy and exact gradient learning is almost tractable. We demonstrate that the RTRBM is better than an analogous TRBM at generating motion capture and videos of bouncing balls.
Geoffrey E Hinton - One of the best experts on this subject based on the ideXlab platform.
-
modeling documents with a deep Boltzmann Machine
Uncertainty in Artificial Intelligence, 2013Co-Authors: Nitish Srivastava, Ruslan Salakhutdinov, Geoffrey E HintonAbstract:We introduce a type of Deep Boltzmann Machine (DBM) that is suitable for extracting distributed semantic representations from a large unstructured collection of documents. We overcome the apparent difficulty of training a DBM with judicious parameter tying. This enables an efficient pretraining algorithm and a state initialization scheme for fast inference. The model can be trained just as efficiently as a standard Restricted Boltzmann Machine. Our experiments show that the model assigns better log probability to unseen data than the Replicated Softmax model. Features extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classification tasks.
-
fast inference and learning for modeling documents with a deep Boltzmann Machine
2013Co-Authors: Nitish Srivastava, Ruslan Salakhutdinov, Geoffrey E HintonAbstract:We introduce a type of Deep Boltzmann Machine (DBM) that is suitable for extracting distributed semantic representations from a large unstructured collection of documents. We propose an approximate inference method that interacts with learning in a way that makes it possible to train the DBM more eciently than previously proposed methods. Even though the model has two hidden layers, it can be trained just as eciently as a standard Restricted Boltzmann Machine. Our experiments show that the model assigns better log probability to unseen data than the Replicated Softmax model. Features extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classication tasks.
-
phone recognition with the mean covariance restricted Boltzmann Machine
Neural Information Processing Systems, 2010Co-Authors: George E Dahl, Marcaurelio Ranzato, Abdelrahman Mohamed, Geoffrey E HintonAbstract:Straightforward application of Deep Belief Nets (DBNs) to acoustic modeling produces a rich distributed representation of speech data that is useful for recognition and yields impressive results on the speaker-independent TIMIT phone recognition task. However, the first-layer Gaussian-Bernoulli Restricted Boltzmann Machine (GRBM) has an important limitation, shared with mixtures of diagonal-covariance Gaussians: GRBMs treat different components of the acoustic input vector as conditionally independent given the hidden state. The mean-covariance restricted Boltzmann Machine (mcRBM), first introduced for modeling natural images, is a much more representationally efficient and powerful way of modeling the covariance structure of speech data. Every configuration of the precision units of the mcRBM specifies a different precision matrix for the conditional distribution over the acoustic space. In this work, we use the mcRBM to learn features of speech data that serve as input into a standard DBN. The mcRBM features combined with DBNs allow us to achieve a phone error rate of 20.5%, which is superior to all published results on speaker-independent TIMIT to date.
-
learning to combine foveal glimpses with a third order Boltzmann Machine
Neural Information Processing Systems, 2010Co-Authors: Hugo Larochelle, Geoffrey E HintonAbstract:We describe a model based on a Boltzmann Machine with third-order connections that can learn how to accumulate information about a shape over several fixations. The model uses a retina that only has enough high resolution pixels to cover a small area of the image, so it must decide on a sequence of fixations and it must combine the "glimpse" at each fixation with the location of the fixation before integrating the information with information from other glimpses of the same object. We evaluate this model on a synthetic dataset and two image classification datasets, showing that it can perform at least as well as a model trained on whole images.
-
modeling pigeon behavior using a conditional restricted Boltzmann Machine
The European Symposium on Artificial Neural Networks, 2009Co-Authors: Matthew D Zeiler, Graham W Taylor, Nikolaus F Troje, Geoffrey E HintonAbstract:In an effort to better understand the complex courtship be- haviour of pigeons, we have built a model learned from motion capture data. We employ a Conditional Restricted Boltzmann Machine (CRBM) with binary latent features and real-valued visible units. The units are conditioned on information from previous time steps to capture dynam- ics. We validate a trained model by quantifying the characteristic "head- bobbing" present in pigeons. We also show how to predict missing data by marginalizing out the hidden variables and minimizing free energy.
Roderick Edwards - One of the best experts on this subject based on the ideXlab platform.
-
Universal approximation results for the temporal restricted Boltzmann Machine and the recurrent temporal restricted Boltzmann Machine
Journal of Machine Learning Research, 2016Co-Authors: Simon Odense, Roderick EdwardsAbstract:The Restricted Boltzmann Machine (RBM) has proved to be a powerful tool in Machine learning, both on its own and as the building block for Deep Belief Networks (multi-layer generative graphical models). The RBM and Deep Belief Network have been shown to be universal approximators for probability distributions on binary vectors. In this paper we prove several similar universal approximation results for two variations of the Restricted Boltzmann Machine with time dependence, the Temporal Restricted Boltzmann Machine (TRBM) and the Recurrent Temporal Restricted Boltzmann Machine (RTRBM). We show that the TRBM is a universal approximator for Markov chains and generalize the theorem to sequences with longer time dependence. We then prove that the RTRBM is a universal approximator for stochastic processes with _nite time dependence. We conclude with a discussion on efficiency and how the constructions developed could explain some previous experimental results.