The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform

Michael I Jordan - One of the best experts on this subject based on the ideXlab platform.

  • variational consensus monte carlo
    Neural Information Processing Systems, 2015
    Co-Authors: Maxim Rabinovich, Elaine Angelino, Michael I Jordan
    Abstract:

    Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel [22]. A fixed Aggregation Function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over Aggregation Functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC—achieving near-ideal speedup in some instances.

  • variational consensus monte carlo
    arXiv: Machine Learning, 2015
    Co-Authors: Maxim Rabinovich, Elaine Angelino, Michael I Jordan
    Abstract:

    Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel (Scott et al, 2013). A fixed Aggregation Function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over Aggregation Functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC, achieving near-ideal speedup in some instances.

Maxim Rabinovich - One of the best experts on this subject based on the ideXlab platform.

  • variational consensus monte carlo
    Neural Information Processing Systems, 2015
    Co-Authors: Maxim Rabinovich, Elaine Angelino, Michael I Jordan
    Abstract:

    Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel [22]. A fixed Aggregation Function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over Aggregation Functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC—achieving near-ideal speedup in some instances.

  • variational consensus monte carlo
    arXiv: Machine Learning, 2015
    Co-Authors: Maxim Rabinovich, Elaine Angelino, Michael I Jordan
    Abstract:

    Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel (Scott et al, 2013). A fixed Aggregation Function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over Aggregation Functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC, achieving near-ideal speedup in some instances.

Elaine Angelino - One of the best experts on this subject based on the ideXlab platform.

  • variational consensus monte carlo
    Neural Information Processing Systems, 2015
    Co-Authors: Maxim Rabinovich, Elaine Angelino, Michael I Jordan
    Abstract:

    Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel [22]. A fixed Aggregation Function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over Aggregation Functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC—achieving near-ideal speedup in some instances.

  • variational consensus monte carlo
    arXiv: Machine Learning, 2015
    Co-Authors: Maxim Rabinovich, Elaine Angelino, Michael I Jordan
    Abstract:

    Practitioners of Bayesian statistics have long depended on Markov chain Monte Carlo (MCMC) to obtain samples from intractable posterior distributions. Unfortunately, MCMC algorithms are typically serial, and do not scale to the large datasets typical of modern machine learning. The recently proposed consensus Monte Carlo algorithm removes this limitation by partitioning the data and drawing samples conditional on each partition in parallel (Scott et al, 2013). A fixed Aggregation Function then combines these samples, yielding approximate posterior samples. We introduce variational consensus Monte Carlo (VCMC), a variational Bayes algorithm that optimizes over Aggregation Functions to obtain samples from a distribution that better approximates the target. The resulting objective contains an intractable entropy term; we therefore derive a relaxation of the objective and show that the relaxed problem is blockwise concave under mild conditions. We illustrate the advantages of our algorithm on three inference tasks from the literature, demonstrating both the superior quality of the posterior approximation and the moderate overhead of the optimization step. Our algorithm achieves a relative error reduction (measured against serial MCMC) of up to 39% compared to consensus Monte Carlo on the task of estimating 300-dimensional probit regression parameter expectations; similarly, it achieves an error reduction of 92% on the task of estimating cluster comembership probabilities in a Gaussian mixture model with 8 components in 8 dimensions. Furthermore, these gains come at moderate cost compared to the runtime of serial MCMC, achieving near-ideal speedup in some instances.

Chee Peng Lim - One of the best experts on this subject based on the ideXlab platform.

  • parametric conditions for a monotone tsk fuzzy inference system to be an n ary Aggregation Function
    IEEE Transactions on Fuzzy Systems, 2021
    Co-Authors: Yi Wen Kerk, Chin Ying Teh, Kai Meng Tay, Chee Peng Lim
    Abstract:

    Despite the popularity and practical importance of the fuzzy inference system (FIS), the use of an FIS model as an n -ary Aggregation Function, which is characterized by both the monotonicity and boundary properties, is yet to be established. This is because research on ensuring that FIS models satisfy the monotonicity property, i.e., monotone FIS, is relatively new, not to mention the additional requirement of satisfying the boundary property. The aim of this article, therefore, is to establish the parametric conditions for the Takagi–Sugeno–Kang (TSK) FIS model to operate as an n -ary Aggregation Function (hereafter denoted as n -TSK-FIS) via the specifications of fuzzy membership Functions and fuzzy rules. An absorption property with fuzzy rules interpretation is outlined, and the use of n -TSK-FIS as a uninorm is explained. Exploiting the established parametric conditions, a framework for which an n -TSK-FIS model can be constructed from data samples is formulated and analyzed, along with a number of remarks. Synthetic data sets and a benchmark example on education assessment are presented and discussed. To be best of the authors’ knowledge, this article serves as the first use of the TSK-FIS model as an n -ary Aggregation Function.

Lorenz T Biegler - One of the best experts on this subject based on the ideXlab platform.

  • optimal process design with model parameter uncertainty and process variability
    Aiche Journal, 2003
    Co-Authors: William C Rooney, Lorenz T Biegler
    Abstract:

    Optimal design under unknown information is a key task in process systems engineering. This study considers formulations that incorporate two types of unknown input parameters, uncertain model parameters, and variable process parameters. In the former case, a process must be designed that is feasible over the entire domain of uncertain parameters, while in the latter case, control variables can be adjusted during process operation to compensate for variable process parameters. To address this problem we extend the two-stage formulation for design under uncertainty and derive new formulations for the multiperiod and feasibility problems. Moreover, to simplify the feasibility problem in the two-stage algorithm, we also introduce a KS constraint Aggregation Function and derive a single, smooth nonlinear program that approximates the feasibility problem. Three case studies are presented to demonstrate the proposed approach.

  • new strategies for flexibility analysis and design under uncertainty
    Computers & Chemical Engineering, 2000
    Co-Authors: Claudia Gabriela Raspanti, Jose Alberto Bandoni, Lorenz T Biegler
    Abstract:

    Abstract Process flexibility and design under uncertainty have been researched extensively in the literature. Problem formulations for flexibility include nested optimization problems and these can often be refined by substituting the optimality conditions for these nested problems. However, these reformulations are highly constrained and can be expensive to solve. In this paper we extend algorithms to solve these reformulated NLP problem under uncertainty by introducing two contributions to this approach. These are the use of a Constraint Aggregation Function (KS Function) and Smoothing Functions. We begin with basic properties of KS Function. Next, we review a class of parametric smooth Functions, used to replace the complementarity conditions of the KKT conditions with a well-behaved, smoothed nonlinear equality constraint. In this paper we apply the previous strategies to two specific problems: i) the'worst case algorithm', that assesses design under uncertainty and, ii) the flexibility index and feasibility test formulations. In the first case, two new algorithms are derived, one of them being single level optimization problem. Next using similar ideas, both flexibility index and feasibility test are reformulated leading to a single non linear programming problem instead of a mixed integer non linear programming one. The new formulations are demonstrated on five different example problems where a CPU time reduction of more than 70 and 80% is demonstrated.