The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform

Mokshay Madiman - One of the best experts on this subject based on the ideXlab platform.

  • beyond the Entropy power inequality via rearrangements
    IEEE Transactions on Information Theory, 2014
    Co-Authors: Liyao Wang, Mokshay Madiman
    Abstract:

    A lower bound on the Renyi Differential Entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon Entropy, this lower bound is better than that given by the Entropy power inequality. Several applications are discussed, including a new proof of the classical Entropy power inequality and an Entropy inequality involving symmetrization of Levy processes.

  • sumset and inverse sumset inequalities for Differential Entropy and mutual information
    IEEE Transactions on Information Theory, 2014
    Co-Authors: Ioannis Kontoyiannis, Mokshay Madiman
    Abstract:

    The sumset and inverse sumset theories of Freiman, Plunnecke and Ruzsa, give bounds connecting the cardinality of the sumset A +B =fa +b ; a2 A; b2 Bg of two discrete sets A;B, to the cardinalities (or the ner structure) of the original sets A;B. For example, the sum-dierence

  • sumset inequalities for Differential Entropy and mutual information
    International Symposium on Information Theory, 2012
    Co-Authors: Ioannis Kontoyiannis, Mokshay Madiman
    Abstract:

    The Plunnecke-Ruzsa sumset theory gives bounds connecting the cardinality of the sumset A + B defined as {a + b; a ∊ A, b ∊ B} with the cardinalities of the original sets A, B. For example, the sum-difference bound states that, |A+B| |A| |B| ≤ |A−B|3, where A−B = {a−b; a ∊ A, b ∊ B}. Interpreting the Differential Entropy h(X) as (the logarithm of) the size of the effective support of X, the main results here are a series of natural information-theoretic analogs for these bounds. For example, the sum-difference bound becomes the new inequality, h(X + Y) + h(X) + h(Y) ≤ 3h(X − Y), for independent X, Y. Our results include Differential-Entropy versions of Ruzsa's triangle inequality, the Plunnecke-Ruzsa inequality, and the Balog-Szemeredi-Gowers lemma. Versions of most of these results for the discrete Entropy H(X) were recently proved by Tao, relying heavily on a strong, functional form of the submodularity property of H(X). Since Differential Entropy is not functionally submodular, in the continuous case many of the corresponding discrete proofs fail, in several cases requiring substantially new proof strategies. The basic property that naturally replaces functional submodularity is the data processing property of mutual information.

  • sumset and inverse sumset inequalities for Differential Entropy and mutual information
    arXiv: Information Theory, 2012
    Co-Authors: Ioannis Kontoyiannis, Mokshay Madiman
    Abstract:

    The sumset and inverse sumset theories of Freiman, Pl\"{u}nnecke and Ruzsa, give bounds connecting the cardinality of the sumset $A+B=\{a+b\;;\;a\in A,\,b\in B\}$ of two discrete sets $A,B$, to the cardinalities (or the finer structure) of the original sets $A,B$. For example, the sum-difference bound of Ruzsa states that, $|A+B|\,|A|\,|B|\leq|A-B|^3$, where the difference set $A-B= \{a-b\;;\;a\in A,\,b\in B\}$. Interpreting the Differential Entropy $h(X)$ of a continuous random variable $X$ as (the logarithm of) the size of the effective support of $X$, the main contribution of this paper is a series of natural information-theoretic analogs for these results. For example, the Ruzsa sum-difference bound becomes the new inequality, $h(X+Y)+h(X)+h(Y)\leq 3h(X-Y)$, for any pair of independent continuous random variables $X$ and $Y$. Our results include Differential-Entropy versions of Ruzsa's triangle inequality, the Pl\"{u}nnecke-Ruzsa inequality, and the Balog-Szemer\'{e}di-Gowers lemma. Also we give a Differential Entropy version of the Freiman-Green-Ruzsa inverse-sumset theorem, which can be seen as a quantitative converse to the Entropy power inequality. Versions of most of these results for the discrete Entropy $H(X)$ were recently proved by Tao, relying heavily on a strong, functional form of the submodularity property of $H(X)$. Since Differential Entropy is {\em not} functionally submodular, in the continuous case many of the corresponding discrete proofs fail, in many cases requiring substantially new proof strategies. We find that the basic property that naturally replaces the discrete functional submodularity, is the data processing property of mutual information.

Ashok Vardhan Makkuva - One of the best experts on this subject based on the ideXlab platform.

  • equivalence of additive combinatorial linear inequalities for shannon Entropy and Differential Entropy
    IEEE Transactions on Information Theory, 2018
    Co-Authors: Ashok Vardhan Makkuva
    Abstract:

    This paper addresses the correspondence between linear inequalities for Shannon Entropy and Differential Entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero) linear inequality for Shannon Entropy holds if and only if its Differential Entropy counterpart also holds; moreover, any linear inequality for Differential Entropy must be balanced. In particular, our result shows that recently proved Differential Entropy inequalities by Kontoyiannis and Madiman can be deduced from their discrete counterparts due to Tao in a unified manner. Generalizations to certain abelian groups are also obtained. Our proof of extending inequalities for Shannon Entropy to Differential Entropy relies on a result of Renyi which relates the Shannon Entropy of a finely discretized random variable to its Differential Entropy and also helps in establishing that the Entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum; the converse uses the asymptotics of the Differential Entropy of convolutions with weak additive noise.

  • on additive combinatorial affine inequalities for shannon Entropy and Differential Entropy
    International Symposium on Information Theory, 2016
    Co-Authors: Ashok Vardhan Makkuva
    Abstract:

    To be considered for the 2016 IEEE Jack Keil Wolf ISIT Student Paper Award. This paper addresses the question of to what extent do discrete Entropy inequalities for weighted sums of independent group-valued random variables continue to hold for Differential entropies. We show that all balanced affine inequalities (with the sum of coefficients being zero) of Shannon Entropy extend to Differential Entropy; conversely, any affine inequality for Differential Entropy must be balanced. In particular, this result recovers recently proved Differential Entropy inequalities by Kontoyiannis and Madiman [1] from their discrete counterparts due to Tao [2] in a unified manner. Our proof relies on a result of Renyi which relates the Shannon Entropy of a finely discretized random variable to its Differential Entropy and also helps in establishing the Entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum.

  • equivalence of additive combinatorial linear inequalities for shannon Entropy and Differential Entropy
    arXiv: Information Theory, 2016
    Co-Authors: Ashok Vardhan Makkuva
    Abstract:

    This paper addresses the correspondence between linear inequalities of Shannon Entropy and Differential Entropy for sums of independent group-valued random variables. We show that any balanced (with the sum of coefficients being zero) linear inequality of Shannon Entropy holds if and only if its Differential Entropy counterpart also holds; moreover, any linear inequality for Differential Entropy must be balanced. In particular, our result shows that recently proved Differential Entropy inequalities by Kontoyiannis and Madiman \cite{KM14} can be deduced from their discrete counterparts due to Tao \cite{Tao10} in a unified manner. Generalizations to certain abelian groups are also obtained. Our proof of extending inequalities of Shannon Entropy to Differential Entropy relies on a result of Renyi \cite{Renyi59} which relates the Shannon Entropy of a finely discretized random variable to its Differential Entropy and also helps in establishing the Entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum; the converse uses the asymptotics of the Differential Entropy of convolutions with weak additive noise.

  • on additive combinatorial affine inequalities for shannon Entropy and Differential Entropy
    arXiv: Information Theory, 2016
    Co-Authors: Ashok Vardhan Makkuva
    Abstract:

    This paper addresses the question of to what extent do discrete Entropy inequalities for weighted sums of independent group-valued random variables continue to hold for Differential entropies. We show that all balanced (with the sum of coefficients being zero) affine inequalities of Shannon Entropy extend to Differential Entropy; conversely, any affine inequality for Differential Entropy must be balanced. In particular, this result shows that recently proved Differential Entropy inequalities by Kontoyiannis and Madiman \cite{KM14} can be deduced from their discrete counterparts due to Tao \cite{Tao10} in a unified manner. Generalizations to certain abelian groups are also obtained. Our proof relies on a result of R\'enyi \cite{Renyi59} which relates the Shannon Entropy of a finely discretized random variable to its Differential Entropy and also helps in establishing the Entropy of the sum of quantized random variables is asymptotically equal to that of the quantized sum.

T R A Magee - One of the best experts on this subject based on the ideXlab platform.

  • water sorption isotherms of starch powders part 2 thermodynamic characteristics
    Journal of Food Engineering, 2004
    Co-Authors: Alaa H Almuhtaseb, W A M Mcminn, T R A Magee
    Abstract:

    Abstract A thermodynamic approach was used to interpret the experimental adsorption and desorption isotherm data for potato, highly amylopectin and highly amylose powders starch. Calculation of the thermodynamic properties (Differential enthalpy, integral enthalpy, Differential Entropy and integral Entropy) provides an understanding of the properties of water and energy requirements associated with the sorption behaviour. The isosteric heat of sorption (Differential enthalpy) was determined from the equilibrium adsorption and desorption data, using the Clausius–Clapeyron equation in the temperature range 30–60 °C. The Differential and integral enthalpy decreased with increasing moisture content. The integral Entropy was found to be negative in magnitude and increased with moisture content, reaching the Entropy of free water at a moisture content of approximately 1.20 kg kg −1 dry basis. The exponential trend of the Differential Entropy with moisture content was found to be similar to that of Differential enthalpy. The spreading pressure increased with increasing water activity, and decreased with increasing temperature. The Entropy production during the adsorption and desorption process clearly showed that the sorption process in starch materials is irreversible.

Ioannis Kontoyiannis - One of the best experts on this subject based on the ideXlab platform.

  • sumset and inverse sumset inequalities for Differential Entropy and mutual information
    IEEE Transactions on Information Theory, 2014
    Co-Authors: Ioannis Kontoyiannis, Mokshay Madiman
    Abstract:

    The sumset and inverse sumset theories of Freiman, Plunnecke and Ruzsa, give bounds connecting the cardinality of the sumset A +B =fa +b ; a2 A; b2 Bg of two discrete sets A;B, to the cardinalities (or the ner structure) of the original sets A;B. For example, the sum-dierence

  • sumset inequalities for Differential Entropy and mutual information
    International Symposium on Information Theory, 2012
    Co-Authors: Ioannis Kontoyiannis, Mokshay Madiman
    Abstract:

    The Plunnecke-Ruzsa sumset theory gives bounds connecting the cardinality of the sumset A + B defined as {a + b; a ∊ A, b ∊ B} with the cardinalities of the original sets A, B. For example, the sum-difference bound states that, |A+B| |A| |B| ≤ |A−B|3, where A−B = {a−b; a ∊ A, b ∊ B}. Interpreting the Differential Entropy h(X) as (the logarithm of) the size of the effective support of X, the main results here are a series of natural information-theoretic analogs for these bounds. For example, the sum-difference bound becomes the new inequality, h(X + Y) + h(X) + h(Y) ≤ 3h(X − Y), for independent X, Y. Our results include Differential-Entropy versions of Ruzsa's triangle inequality, the Plunnecke-Ruzsa inequality, and the Balog-Szemeredi-Gowers lemma. Versions of most of these results for the discrete Entropy H(X) were recently proved by Tao, relying heavily on a strong, functional form of the submodularity property of H(X). Since Differential Entropy is not functionally submodular, in the continuous case many of the corresponding discrete proofs fail, in several cases requiring substantially new proof strategies. The basic property that naturally replaces functional submodularity is the data processing property of mutual information.

  • sumset and inverse sumset inequalities for Differential Entropy and mutual information
    arXiv: Information Theory, 2012
    Co-Authors: Ioannis Kontoyiannis, Mokshay Madiman
    Abstract:

    The sumset and inverse sumset theories of Freiman, Pl\"{u}nnecke and Ruzsa, give bounds connecting the cardinality of the sumset $A+B=\{a+b\;;\;a\in A,\,b\in B\}$ of two discrete sets $A,B$, to the cardinalities (or the finer structure) of the original sets $A,B$. For example, the sum-difference bound of Ruzsa states that, $|A+B|\,|A|\,|B|\leq|A-B|^3$, where the difference set $A-B= \{a-b\;;\;a\in A,\,b\in B\}$. Interpreting the Differential Entropy $h(X)$ of a continuous random variable $X$ as (the logarithm of) the size of the effective support of $X$, the main contribution of this paper is a series of natural information-theoretic analogs for these results. For example, the Ruzsa sum-difference bound becomes the new inequality, $h(X+Y)+h(X)+h(Y)\leq 3h(X-Y)$, for any pair of independent continuous random variables $X$ and $Y$. Our results include Differential-Entropy versions of Ruzsa's triangle inequality, the Pl\"{u}nnecke-Ruzsa inequality, and the Balog-Szemer\'{e}di-Gowers lemma. Also we give a Differential Entropy version of the Freiman-Green-Ruzsa inverse-sumset theorem, which can be seen as a quantitative converse to the Entropy power inequality. Versions of most of these results for the discrete Entropy $H(X)$ were recently proved by Tao, relying heavily on a strong, functional form of the submodularity property of $H(X)$. Since Differential Entropy is {\em not} functionally submodular, in the continuous case many of the corresponding discrete proofs fail, in many cases requiring substantially new proof strategies. We find that the basic property that naturally replaces the discrete functional submodularity, is the data processing property of mutual information.

Pramod Viswanath - One of the best experts on this subject based on the ideXlab platform.