The Experts below are selected from a list of 87903 Experts worldwide ranked by ideXlab platform
David M. Blei - One of the best experts on this subject based on the ideXlab platform.
-
black box variational Inference
2014Co-Authors: Rajesh Ranganath, Sean Gerrish, David M. BleiAbstract:Variational Inference has become a widely used method to approximate posteriors in complex latent variables models. However, deriving a variational Inference Algorithm generally requires signicant model-specic analysis. These eorts can hinder and deter us from quickly developing and exploring a variety of models for a problem at hand. In this paper, we present a \black box" variational Inference Algorithm, one that can be quickly applied to many models with little additional derivation. Our method is based on a stochastic optimization of the variational objective where the noisy gradient is computed from Monte Carlo samples from the variational distribution. We develop a number of methods to reduce the variance of the gradient, always maintaining the criterion that we want to avoid dicult model-based derivations. We evaluate our method against the corresponding black box sampling based methods. We nd that our method reaches better predictive likelihoods much faster than sampling methods. Finally, we demonstrate that Black Box Variational Inference lets us easily explore a wide space of models by quickly constructing and evaluating several models of longitudinal healthcare data.
-
Truncation-free stochastic variational Inference for Bayesian nonparametric models
2012Co-Authors: Chong Wang, David M. BleiAbstract:We present a truncation-free stochastic variational Inference Algorithm for Bayesian nonparametric models. While traditional variational Inference Algorithms require truncations for the model or the variational distribution, our method adapts model complexity on the fly. We studied our method with Dirichlet process mixture models and hierarchical Dirichlet process topic models on two large data sets. Our method performs better than previous stochastic variational Inference Algorithms.
-
the discrete innite logistic normal distribution
2012Co-Authors: John Paisley, Chong Wang, David M. BleiAbstract:We present the discrete innite logistic normal distribution (DILN), a Bayesian nonparametric prior for mixed membership models. DILN generalizes the hierarchical Dirichlet process (HDP) to model correlation structure between the weights of the atoms at the group level. We derive a representation of DILN as a normalized collection of gamma-distributed random variables and study its statistical properties. We derive a variational Inference Algorithm for approximate posterior Inference. We apply DILN to topic modeling of documents and study its empirical performance on four corpora, comparing performance with the HDP and the correlated topic model (CTM). To compute with large-scale data, we develop a stochastic variational Inference Algorithm for DILN and compare with similar Algorithms for HDP and latent Dirichlet allocation (LDA) on a collection of 350; 000 articles from Nature.
-
truncation free online variational Inference for bayesian nonparametric models
2012Co-Authors: Chong Wang, David M. BleiAbstract:We present a truncation-free online variational Inference Algorithm for Bayesian nonparametric models. Unlike traditional (online) variational Inference Algorithms that require truncations for the model or the variational distribution, our method adapts model complexity on the fly. Our experiments for Dirichlet process mixture models and hierarchical Dirichlet process topic models on two large-scale data sets show better performance than previous online variational Inference Algorithms.
-
variational Inference for stick breaking beta process priors
2011Co-Authors: Lawrence Carin, David M. Blei, John PaisleyAbstract:We present a variational Bayesian Inference Algorithm for the stick-breaking construction of the beta process. We derive an alternate representation of the beta process that is amenable to variational Inference, and present a bound relating the truncated beta process to its infinite counterpart. We assess performance on two matrix factorization problems, using a non-negative factorization model and a linear-Gaussian model.
Chaoyang Zhang - One of the best experts on this subject based on the ideXlab platform.
-
a novel gene network Inference Algorithm using predictive minimum description length approach
2010Co-Authors: Vijender Chaitankar, Preetam Ghosh, Edward J Perkins, Ping Gong, Youping Deng, Chaoyang ZhangAbstract:Background: Reverse engineering of gene regulatory networks using information theory models has received much attention due to its simplicity, low computational cost, and capability of inferring large networks. One of the major problems with information theory models is to determine the threshold which defines the regulatory relationships between genes. The minimum description length (MDL) principle has been implemented to overcome this problem. The description length of the MDL principle is the sum of model length and data encoding length. A user-specified fine tuning parameter is used as control mechanism between model and data encoding, but it is difficult to find the optimal parameter. In this work, we proposed a new Inference Algorithm which incorporated mutual information (MI), conditional mutual information (CMI) and predictive minimum description length (PMDL) principle to infer gene regulatory networks from DNA microarray data. In this Algorithm, the information theoretic quantities MI and CMI determine the regulatory relationships between genes and the PMDL principle method attempts to determine the best MI threshold without the need of a user-specified fine tuning parameter. Results: The performance of the proposed Algorithm was evaluated using both synthetic time series data sets and a biological time series data set for the yeast Saccharomyces cerevisiae. The benchmark quantities precision and recall were used as performance measures. The results show that the proposed Algorithm produced less false edges and significantly improved the precision, as compared to the existing Algorithm. For further analysis the performance of the Algorithms was observed over different sizes of data. Conclusions: We have proposed a new Algorithm that implements the PMDL principle for inferring gene regulatory networks from time series DNA microarray data that eliminates the need of a fine tuning parameter. The evaluation results obtained from both synthetic and actual biological data sets show that the PMDL principle is effective in determining the MI threshold and the developed Algorithm improves precision of gene regulatory network Inference. Based on the sensitivity analysis of all tested cases, an optimal CMI threshold value has been identified. Finally it was observed that the performance of the Algorithms saturates at a certain threshold of data size.
Max Welling - One of the best experts on this subject based on the ideXlab platform.
-
a collapsed variational bayesian Inference Algorithm for latent dirichlet allocation
2006Co-Authors: Yee Whye Teh, David Newman, Max WellingAbstract:Latent Dirichlet allocation (LDA) is a Bayesian network that has recently gained much popularity in applications ranging from document modeling to computer vision. Due to the large scale nature of these applications, current Inference procedures like variational Bayes and Gibbs sampling have been found lacking. In this paper we propose the collapsed variational Bayesian Inference Algorithm for LDA, and show that it is computationally efficient, easy to implement and significantly more accurate than standard variational Bayesian Inference for LDA.
Dhruv Batra - One of the best experts on this subject based on the ideXlab platform.
-
bidirectional beam search forward backward Inference in neural sequence models for fill in the blank image captioning
2017Co-Authors: Qing Sun, Stefan Lee, Dhruv BatraAbstract:We develop the first approximate Inference Algorithm for 1-Best (and M-Best) decoding in bidirectional neural sequence models by extending Beam Search (BS) to reason about both forward and backward time dependencies. Beam Search (BS) is a widely used approximate Inference Algorithm for decoding sequences from unidirectional neural sequence models. Interestingly, approximate Inference in bidirectional models remains an open problem, despite their significant advantage in modeling information from both the past and future. To enable the use of bidirectional models, we present Bidirectional Beam Search (BiBS), an efficient Algorithm for approximate bidirectional Inference. To evaluate our method and as an interesting problem in its own right, we introduce a novel Fill-in-the-Blank Image Captioning task which requires reasoning about both past and future sentence structure to reconstruct sensible image descriptions. We use this task as well as the Visual Madlibs dataset to demonstrate the effectiveness of our approach, consistently outperforming all baseline methods.
-
bidirectional beam search forward backward Inference in neural sequence models for fill in the blank image captioning
2017Co-Authors: Qing Sun, Stefan Lee, Dhruv BatraAbstract:We develop the first approximate Inference Algorithm for 1-Best (and M-Best) decoding in bidirectional neural sequence models by extending Beam Search (BS) to reason about both forward and backward time dependencies. Beam Search (BS) is a widely used approximate Inference Algorithm for decoding sequences from unidirectional neural sequence models. Interestingly, approximate Inference in bidirectional models remains an open problem, despite their significant advantage in modeling information from both the past and future. To enable the use of bidirectional models, we present Bidirectional Beam Search (BiBS), an efficient Algorithm for approximate bidirectional this http URL evaluate our method and as an interesting problem in its own right, we introduce a novel Fill-in-the-Blank Image Captioning task which requires reasoning about both past and future sentence structure to reconstruct sensible image descriptions. We use this task as well as the Visual Madlibs dataset to demonstrate the effectiveness of our approach, consistently outperforming all baseline methods.
Yee Whye Teh - One of the best experts on this subject based on the ideXlab platform.
-
beam sampling for the infinite hidden markov model
2008Co-Authors: Jurgen Van Gael, Yee Whye Teh, Yunus Saatci, Zoubin GhahramaniAbstract:The infinite hidden Markov model is a non-parametric extension of the widely used hidden Markov model. Our paper introduces a new Inference Algorithm for the infinite Hidden Markov model called beam sampling. Beam sampling combines slice sampling, which limits the number of states considered at each time step to a finite number, with dynamic programming, which samples whole state trajectories efficiently. Our Algorithm typically outperforms the Gibbs sampler and is more robust. We present applications of iHMM Inference using the beam sampler on changepoint detection and text prediction problems.
-
a collapsed variational bayesian Inference Algorithm for latent dirichlet allocation
2006Co-Authors: Yee Whye Teh, David Newman, Max WellingAbstract:Latent Dirichlet allocation (LDA) is a Bayesian network that has recently gained much popularity in applications ranging from document modeling to computer vision. Due to the large scale nature of these applications, current Inference procedures like variational Bayes and Gibbs sampling have been found lacking. In this paper we propose the collapsed variational Bayesian Inference Algorithm for LDA, and show that it is computationally efficient, easy to implement and significantly more accurate than standard variational Bayesian Inference for LDA.