The Experts below are selected from a list of 19554 Experts worldwide ranked by ideXlab platform

Tommi S Jaakkola - One of the best experts on this subject based on the ideXlab platform.

  • high dimensional inference with random maximum a posteriori perturbations
    IEEE Transactions on Information Theory, 2019
    Co-Authors: Tamir Hazan, Francesco Orabona, Anand D Sarwate, Subhransu Maji, Tommi S Jaakkola
    Abstract:

    This paper presents a new approach, called perturb-max, for high-dimensional statistical inference in graphical models that is based on applying random perturbations followed by optimization. This framework injects randomness into maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic result from extreme value statistics asserts that perturb-max operations generate unbiased samples from the Gibbs Distribution using high-dimensional perturbations. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. However, when the perturbations are of low dimension, sampling the perturb-max prediction is as efficient as MAP optimization. This paper shows that the expected value of perturb-max inference with low dimensional perturbations can be used sequentially to generate unbiased samples from the Gibbs Distribution. Furthermore the expected value of the maximal perturbations is a natural bound on the entropy of such perturb-max models. A measure concentration result for perturb-max values shows that the deviation of their sampled average from its expectation decays exponentially in the number of samples, allowing effective approximation of the expectation.

  • high dimensional inference with random maximum a posteriori perturbations
    arXiv: Learning, 2016
    Co-Authors: Tamir Hazan, Francesco Orabona, Anand D Sarwate, Subhransu Maji, Tommi S Jaakkola
    Abstract:

    This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic result from extreme value statistics asserts that perturb-max operations generate unbiased samples from the Gibbs Distribution using high-dimensional perturbations. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. However, when the perturbations are of low dimension, sampling the perturb-max prediction is as efficient as MAP optimization. This paper shows that the expected value of perturb-max inference with low dimensional perturbations can be used sequentially to generate unbiased samples from the Gibbs Distribution. Furthermore the expected value of the maximal perturbations is a natural bound on the entropy of such perturb-max models. A measure concentration result for perturb-max values shows that the deviation of their sampled average from its expectation decays exponentially in the number of samples, allowing effective approximation of the expectation.

  • on measure concentration of random maximum a posteriori perturbations perturbations
    International Conference on Machine Learning, 2014
    Co-Authors: Francesco Orabona, Tamir Hazan, Anand D Sarwate, Tommi S Jaakkola
    Abstract:

    The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased samples from the Gibbs Distribution. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. More efficient algorithms use sequential sampling strategies based on the expected value of low dimensional MAP perturbations. This paper develops new measure concentration inequalities that bound the number of samples needed to estimate such expected values. Applying the general result to MAP perturbations can yield a more efficient algorithm to approximate sampling from the Gibbs Distribution. The measure concentration result is of general interest and may be applicable to other areas involving Monte Carlo estimation of expectations.

  • learning with maximum a posteriori perturbation models
    International Conference on Artificial Intelligence and Statistics, 2014
    Co-Authors: Andreea Gane, Tamir Hazan, Tommi S Jaakkola
    Abstract:

    Perturbation models are families of Distributions induced from perturbations. They combine randomization of the parameters with maximization to draw unbiased samples. Unlike Gibbs’ Distributions, a perturbation model dened on the basis of low order statistics still gives rise to high order dependencies. In this paper, we analyze, extend and seek to estimate such dependencies from data. In particular, we shift the modelling focus from the parameters of the Gibbs’ Distribution used as a base model to the space of perturbations. We estimate dependent perturbations over the parameters using a hardEM approach, cast in the form of inverse convex programs. Each inverse program connes the randomization to the parameter polytope responsible for generating the observed answer. We illustrate the method on several computer vision problems.

  • on measure concentration of random maximum a posteriori perturbations
    arXiv: Learning, 2013
    Co-Authors: Francesco Orabona, Tamir Hazan, Anand D Sarwate, Tommi S Jaakkola
    Abstract:

    The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased samples from the Gibbs Distribution. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. More efficient algorithms use sequential sampling strategies based on the expected value of low dimensional MAP perturbations. This paper develops new measure concentration inequalities that bound the number of samples needed to estimate such expected values. Applying the general result to MAP perturbations can yield a more efficient algorithm to approximate sampling from the Gibbs Distribution. The measure concentration result is of general interest and may be applicable to other areas involving expected estimations.

Tamir Hazan - One of the best experts on this subject based on the ideXlab platform.

  • high dimensional inference with random maximum a posteriori perturbations
    IEEE Transactions on Information Theory, 2019
    Co-Authors: Tamir Hazan, Francesco Orabona, Anand D Sarwate, Subhransu Maji, Tommi S Jaakkola
    Abstract:

    This paper presents a new approach, called perturb-max, for high-dimensional statistical inference in graphical models that is based on applying random perturbations followed by optimization. This framework injects randomness into maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic result from extreme value statistics asserts that perturb-max operations generate unbiased samples from the Gibbs Distribution using high-dimensional perturbations. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. However, when the perturbations are of low dimension, sampling the perturb-max prediction is as efficient as MAP optimization. This paper shows that the expected value of perturb-max inference with low dimensional perturbations can be used sequentially to generate unbiased samples from the Gibbs Distribution. Furthermore the expected value of the maximal perturbations is a natural bound on the entropy of such perturb-max models. A measure concentration result for perturb-max values shows that the deviation of their sampled average from its expectation decays exponentially in the number of samples, allowing effective approximation of the expectation.

  • high dimensional inference with random maximum a posteriori perturbations
    arXiv: Learning, 2016
    Co-Authors: Tamir Hazan, Francesco Orabona, Anand D Sarwate, Subhransu Maji, Tommi S Jaakkola
    Abstract:

    This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic result from extreme value statistics asserts that perturb-max operations generate unbiased samples from the Gibbs Distribution using high-dimensional perturbations. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. However, when the perturbations are of low dimension, sampling the perturb-max prediction is as efficient as MAP optimization. This paper shows that the expected value of perturb-max inference with low dimensional perturbations can be used sequentially to generate unbiased samples from the Gibbs Distribution. Furthermore the expected value of the maximal perturbations is a natural bound on the entropy of such perturb-max models. A measure concentration result for perturb-max values shows that the deviation of their sampled average from its expectation decays exponentially in the number of samples, allowing effective approximation of the expectation.

  • on measure concentration of random maximum a posteriori perturbations perturbations
    International Conference on Machine Learning, 2014
    Co-Authors: Francesco Orabona, Tamir Hazan, Anand D Sarwate, Tommi S Jaakkola
    Abstract:

    The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased samples from the Gibbs Distribution. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. More efficient algorithms use sequential sampling strategies based on the expected value of low dimensional MAP perturbations. This paper develops new measure concentration inequalities that bound the number of samples needed to estimate such expected values. Applying the general result to MAP perturbations can yield a more efficient algorithm to approximate sampling from the Gibbs Distribution. The measure concentration result is of general interest and may be applicable to other areas involving Monte Carlo estimation of expectations.

  • learning with maximum a posteriori perturbation models
    International Conference on Artificial Intelligence and Statistics, 2014
    Co-Authors: Andreea Gane, Tamir Hazan, Tommi S Jaakkola
    Abstract:

    Perturbation models are families of Distributions induced from perturbations. They combine randomization of the parameters with maximization to draw unbiased samples. Unlike Gibbs’ Distributions, a perturbation model dened on the basis of low order statistics still gives rise to high order dependencies. In this paper, we analyze, extend and seek to estimate such dependencies from data. In particular, we shift the modelling focus from the parameters of the Gibbs’ Distribution used as a base model to the space of perturbations. We estimate dependent perturbations over the parameters using a hardEM approach, cast in the form of inverse convex programs. Each inverse program connes the randomization to the parameter polytope responsible for generating the observed answer. We illustrate the method on several computer vision problems.

  • on measure concentration of random maximum a posteriori perturbations
    arXiv: Learning, 2013
    Co-Authors: Francesco Orabona, Tamir Hazan, Anand D Sarwate, Tommi S Jaakkola
    Abstract:

    The maximum a-posteriori (MAP) perturbation framework has emerged as a useful approach for inference and learning in high dimensional complex models. By maximizing a randomly perturbed potential function, MAP perturbations generate unbiased samples from the Gibbs Distribution. Unfortunately, the computational cost of generating so many high-dimensional random variables can be prohibitive. More efficient algorithms use sequential sampling strategies based on the expected value of low dimensional MAP perturbations. This paper develops new measure concentration inequalities that bound the number of samples needed to estimate such expected values. Applying the general result to MAP perturbations can yield a more efficient algorithm to approximate sampling from the Gibbs Distribution. The measure concentration result is of general interest and may be applicable to other areas involving expected estimations.

Masaru Hongo - One of the best experts on this subject based on the ideXlab platform.

  • relativistic hydrodynamics from quantum field theory on the basis of the generalized Gibbs ensemble method
    Physical Review D, 2015
    Co-Authors: Tomoya Hayata, Yoshimasa Hidaka, Toshifumi Noumi, Masaru Hongo
    Abstract:

    We derive relativistic hydrodynamics from quantum field theories by assuming that the density operator is given by a local Gibbs Distribution at initial time. We decompose the energy-momentum tensor and particle current into nondissipative and dissipative parts, and analyze their time evolution in detail. Performing the path-integral formulation of the local Gibbs Distribution, we microscopically derive the generating functional for the nondissipative hydrodynamics.We also construct a basis to study dissipative corrections. In particular, we derive the first-order dissipative hydrodynamic equations without a choice of frame such as the Landau-Lifshitz or Eckart frame.

Eric Vigoda - One of the best experts on this subject based on the ideXlab platform.

  • convergence of mcmc and loopy bp in the tree uniqueness region for the hard core model
    arXiv: Discrete Mathematics, 2016
    Co-Authors: Charilaos Efthymiou, Thomas P Hayes, Daniel Stefankovic, Eric Vigoda, Yitong Yin
    Abstract:

    We study the hard-core model defined on independent sets of an input graph where the independent sets are weighted by a parameter $\lambda>0$. For constant $\Delta$, previous work of Weitz (2006) established an FPTAS for the partition function for graphs of maximum degree $\Delta$ when $\lambda \lambda_c(\Delta)$. The running time of Weitz's algorithm is exponential in $\log(\Delta)$. Here we present an FPRAS for the partition function whose running time is $O^*(n^2)$. We analyze the simple single-site Glauber dynamics for sampling from the associated Gibbs Distribution. We prove there exists a constant $\Delta_0$ such that for all graphs with maximum degree $\Delta\geq\Delta_0$ and girth $\geq 7$, the mixing time of the Glauber dynamics is $O(n\log(n))$ when $\lambda<\lambda_c(\Delta)$. Our work complements that of Weitz which applies for constant $\Delta$ whereas our work applies for all $\Delta \geq \Delta_0$. We utilize loopy BP (belief propagation), a widely-used inference algorithm. A novel aspect of our work is using the principal eigenvector for the BP operator to design a distance function which contracts in expectation for pairs of states that behave like the BP fixed point. We also prove that the Glauber dynamics behaves locally like loopy BP. As a byproduct we obtain that the Glauber dynamics converges, after a short burn-in period, close to the BP fixed point, and this implies that the fixed point of loopy BP is a close approximation to the Gibbs Distribution. Using these connections we establish that loopy BP quickly converges to the Gibbs Distribution when the girth $\geq 6$ and $\lambda<\lambda_c(\Delta)$.

  • Convergence of MCMC and Loopy BP in the Tree Uniqueness Region for the Hard-Core Model
    2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), 2016
    Co-Authors: Charilaos Efthymiou, Thomas P Hayes, Daniel Stefankovic, Eric Vigoda
    Abstract:

    We study the hard-core (gas) model defined on independent sets of an input graph where the independent sets are weighted by a parameter (aka fugacity) λ > 0. For constant Δ, previous work of Weitz (2006) established an FPTAS for the partition function for graphs of maximum degree Δ when λ λc(Δ). The threshold λc(Δ) is the critical point for the statistical physics phase transition for uniqueness/non-uniqueness on the infinite Δ-regular tree. The running time of Weitz's algorithm is exponential in log Δ. Here we present an FPRAS for the partition function whose running time is O* (n2). We analyze the simple single-site Markov chain known as the Glauber dynamics for sampling from the associated Gibbs Distribution. We prove there exists a constant Δ0 such that for all graphs with maximum degree Δ > Δ0 and girth > 7 (i.e., no cycles of length ≤ 6), the mixing time of the Glauber dynamics is O(nlog n) when λ

  • coupling with the stationary Distribution and improved sampling for colorings and independent sets
    Annals of Applied Probability, 2006
    Co-Authors: Thomas P Hayes, Eric Vigoda
    Abstract:

    We present an improved coupling technique for analyzing the mixing time of Markov chains. Using our technique, we simplify and extend previous results for sampling colorings and independent sets. Our approach uses properties of the stationary Distribution to avoid worst-case configurations which arise in the traditional approach. As an application, we show that for $k/\Delta >1.764$, the Glauber dynamics on $k$-colorings of a graph on $n$ vertices with maximum degree $\Delta$ converges in $O(n\log n)$ steps, assuming $\Delta =\Omega(\log n)$ and that the graph is triangle-free. Previously, girth $\ge 5$ was needed. As a second application, we give a polynomial-time algorithm for sampling weighted independent sets from the Gibbs Distribution of the hard-core lattice gas model at fugacity $\lambda <(1-\epsilon)e/\Delta$, on a regular graph $G$ on $n$ vertices of degree $\Delta =\Omega(\log n)$ and girth $\ge 6$. The best known algorithm for general graphs currently assumes $\lambda <2/(\Delta -2)$.

  • coupling with the stationary Distribution and improved sampling for colorings and independent sets
    Symposium on Discrete Algorithms, 2005
    Co-Authors: Thomas P Hayes, Eric Vigoda
    Abstract:

    We present an improved coupling technique for analyzing the mixing time of Markov chains. Using our technique, we simplify and extend previous results for sampling colorings and independent sets. Our approach uses properties of the stationary Distribution to avoid worst-case configurations which arise in the traditional approach.As an application, we show that for k/Δ > 1.764, the Glauber dynamics on k-colorings of a graph on n vertices with maximum degree Δ converges in O(n log n) steps, assuming Δ = Ω(log n) and that the graph is triangle-free. Previously, girth ≥ 5 was needed.As a second application, we give a polynomial-time algorithm for sampling weighted independent sets from the Gibbs Distribution of the hard-core lattice gas model at fugacity λ < (1 - e)e/Δ, on a regular graph G on n vertices of degree Δ = Ω(log n) and girth ≥ 6. The best known algorithm for general graphs currently assumes λ < 2/(Δ - 2).

M I Kalinin - One of the best experts on this subject based on the ideXlab platform.

  • on the completeness of description of an equilibrium canonical ensemble using a reduced s particle Distribution function
    Journal of Statistical Mechanics: Theory and Experiment, 2009
    Co-Authors: M I Kalinin
    Abstract:

    In this paper it is shown that for a classical equilibrium canonical ensemble of molecules with sufficiently small s-body interaction, the full Gibbs Distribution can be uniquely expressed in terms of a reduced s-particle Distribution function. This means that whenever the number of particles N and the volume V of such a system are fixed, the reduced s-particle Distribution function contains as much information about the equilibrium system as the canonical Gibbs Distribution function. The latter is represented as an absolutely convergent power series relative to the reduced s-particle Distribution function. As an example, a linear term of this expansion is calculated. It is also shown that reduced Distribution functions of order less than s do not possess such a property and, to all appearances, do not contain all of the information about the system under consideration.

  • on the completeness of describing an equilibrium canonical ensemble using a pair Distribution function
    arXiv: Statistical Mechanics, 2004
    Co-Authors: M I Kalinin
    Abstract:

    It is shown that in equilibrium a canonical ensemble of particles with two-particle interaction the Gibbs Distribution function may be expressed uniquely through a pair Distribution function. It means, that for given values of the particle number $N$, volume $V$, and temperature $T$, the pair Distribution function contains as many information about the system as a full Gibbs Distribution. The latter is represented as a series expansion in the pair Distribution function. A recurrence relation system is constructed, which allows all terms of this expansion to be calculated successively.